Test Plano Plan of action for testing the entire applications. This
covers resources, resource allocation, budget, breakdown of features, timeline,
etc
Test spec
o A specification for testing of a specific
feature area. This goes into more depth about a specific feature then the test
plan
Test case
o A specific test that is run on the
application to verify proper functionality
Bug
o A product
defect
Test Harness
o A program woven into the source code
that allows testers to script out automated test cases to be run on an
application in development
Halo Testing
o Testing around an
existing test case or bug to see if there are other defects that may all be
linked
Sunday, December 2, 2012
Common Test Areas
• Functional testing
o Verify the basic functionality of an application
o Example: Can you unscrew the cap off of the water bottle?
• Stress testing
o Hostile User Scenarios that happen to an application on a daily basis
o Example: Can you squeeze the bottle without breaking it?
• Safety / Security
o Is the application safe to run on your home computer? Is it secure from outside intrusion?
o Example: Does the bottles safety seal break when the you apply pressure to the cap on the water bottle, without twisting?
• Localization / Globalization Testing
o Is the UI in the correct language for your particular market? Can the grammatical / language content be changed to a different language without breaking the application?
o Example: Is the text on the label of the bottle grammatically correct when it is translated into Chinese for sale in the Chinese market?
• Boundary
o Does the application crash / perform badly when you push the boundaries of it’s intended use?
o Example: Does the bottle hold the amount of water specified in the spec?
• Scalability
o Is the application the proper size for it’s intended audience?
o Example: Does the bottle fit into it’s packaging?
• Usability
o Can anyone pick up the application and easily use it for its intended purpose?
o Example: Can anyone pick up this bottle and figure out how to open it?
• Setup / Uninstall
o Does the application install / uninstall correctly?
o Example: Dose the setup program copy all necessary files to the computer hard drive during installation? Does uninstall remove the correct files during un-installation?
• Ap / compat
o Is the application compatible with other applications while running?
o Example: When you copy a word for a Word Pad document into an Excel spreadsheet, does it cause either program to hang / crash?
o Verify the basic functionality of an application
o Example: Can you unscrew the cap off of the water bottle?
• Stress testing
o Hostile User Scenarios that happen to an application on a daily basis
o Example: Can you squeeze the bottle without breaking it?
• Safety / Security
o Is the application safe to run on your home computer? Is it secure from outside intrusion?
o Example: Does the bottles safety seal break when the you apply pressure to the cap on the water bottle, without twisting?
• Localization / Globalization Testing
o Is the UI in the correct language for your particular market? Can the grammatical / language content be changed to a different language without breaking the application?
o Example: Is the text on the label of the bottle grammatically correct when it is translated into Chinese for sale in the Chinese market?
• Boundary
o Does the application crash / perform badly when you push the boundaries of it’s intended use?
o Example: Does the bottle hold the amount of water specified in the spec?
• Scalability
o Is the application the proper size for it’s intended audience?
o Example: Does the bottle fit into it’s packaging?
• Usability
o Can anyone pick up the application and easily use it for its intended purpose?
o Example: Can anyone pick up this bottle and figure out how to open it?
• Setup / Uninstall
o Does the application install / uninstall correctly?
o Example: Dose the setup program copy all necessary files to the computer hard drive during installation? Does uninstall remove the correct files during un-installation?
• Ap / compat
o Is the application compatible with other applications while running?
o Example: When you copy a word for a Word Pad document into an Excel spreadsheet, does it cause either program to hang / crash?
Priority and Severity
What’s the difference between priority and severity?
“Priority” is associated with scheduling, and “severity” is associated with standards.
“Priority” means something is afforded or deserves prior attention; a precedence
established by order of importance (or urgency). “Severity” is the state or quality of
being severe; severe implies adherence to rigorous standards or high principles and
often suggests harshness; severe is marked by or requires strict adherence to
rigorous standards or high principles, e.g. a severe code of behavior. The words
priority and severity do come up in bug tracking. A variety of commercial, problemtracking/
management software tools are available. These tools, with the detailed
input of software test engineers, give the team complete information so developers can understand the bug, get an idea of its ’severity’, reproduce it and fix it. The fixes are based on project ‘priorities’ and ’severity’ of bugs. The ’severity’ of a problem is defined in accordance to the customer’s risk assessment and recorded in their selected tracking tool. A buggy software can ’severely’ affect schedules, which, in turn can lead to a reassessment and renegotiation of ‘priorities’.
“Priority” is associated with scheduling, and “severity” is associated with standards.
“Priority” means something is afforded or deserves prior attention; a precedence
established by order of importance (or urgency). “Severity” is the state or quality of
being severe; severe implies adherence to rigorous standards or high principles and
often suggests harshness; severe is marked by or requires strict adherence to
rigorous standards or high principles, e.g. a severe code of behavior. The words
priority and severity do come up in bug tracking. A variety of commercial, problemtracking/
management software tools are available. These tools, with the detailed
input of software test engineers, give the team complete information so developers can understand the bug, get an idea of its ’severity’, reproduce it and fix it. The fixes are based on project ‘priorities’ and ’severity’ of bugs. The ’severity’ of a problem is defined in accordance to the customer’s risk assessment and recorded in their selected tracking tool. A buggy software can ’severely’ affect schedules, which, in turn can lead to a reassessment and renegotiation of ‘priorities’.
Test change mechanism for a vending machine
Question: Ask applicant to test a change mechanism for a vending machine.
Specification - input = nickels, dimes, quarters and dollar bills, price range
of items is 55 to 75 cents.
Answer: A good candidate should cover many of the different testing methodologies with this question and should be coming up with interesting/creative test cases in relatively rapid succession for 5 minutes or more. No hires generally run out of ideas within 1-4 minutes. An exceptional candidate may think of different test cases for 15 minutes or more. Some of the various test areas and possible answers are listed below:
Functional: testing all combinations of coins for 55 and 75 cents. For example, 11 nickels, 5 dimes and a nickel, 2 quarters and a nickel, 1 dollar...any combination that adds up to 55 cents and the same for 75 cents. Advanced testers may think of designing a finite-state diagram to cover all combinations. Test change return option as well.
Boundary: no input value, less than expected value (i.e. less than 55 cents), more than expected value (i.e. $2.00).
Stress: Excessively high input values. For example, if I inserted 10 dollars worth of dimes, how does the machine respond? What if I run out of nickels for change, how does the machine detect this and distribute change. What is the wear and tear on the internal mechanism over time. Will machine function constantly/reliable for 10 years. What if the machine is full with a particular change value. For example, dimes and the machine cannot accept any more dime values. What about environmental stress conditions such as extreme heat or cold, or altitude changes. How does it respond to rain.
Error: No change available in machine. Is customer notified. If I input invalid coins, how does the machine react (i.e. pennies). What if I unplug and plug in the machine to simulate a power outage. Does it reset itself? Does it remember input values up to that point? What if machine can't accept any value since it is so full it requires service.
Security: Use of crowbar to attempt to open machine. What about a photocopied dollar bill. Is this accepted by machine?
Performance: Does the system respond quickly even under stress conditions listed above.
Usability: Is everything clearly labeled? Do customer's understand how to use change mechanism?
Accessibility: Can a blind person use the system? Does system have brail?
Localization: Can system be reset to accept coins from different countries? For example, Canada, Japan?
Integration Testing: Applicant may consider testing coin mechanism in conjunction with other parts.
Answer: A good candidate should cover many of the different testing methodologies with this question and should be coming up with interesting/creative test cases in relatively rapid succession for 5 minutes or more. No hires generally run out of ideas within 1-4 minutes. An exceptional candidate may think of different test cases for 15 minutes or more. Some of the various test areas and possible answers are listed below:
Functional: testing all combinations of coins for 55 and 75 cents. For example, 11 nickels, 5 dimes and a nickel, 2 quarters and a nickel, 1 dollar...any combination that adds up to 55 cents and the same for 75 cents. Advanced testers may think of designing a finite-state diagram to cover all combinations. Test change return option as well.
Boundary: no input value, less than expected value (i.e. less than 55 cents), more than expected value (i.e. $2.00).
Stress: Excessively high input values. For example, if I inserted 10 dollars worth of dimes, how does the machine respond? What if I run out of nickels for change, how does the machine detect this and distribute change. What is the wear and tear on the internal mechanism over time. Will machine function constantly/reliable for 10 years. What if the machine is full with a particular change value. For example, dimes and the machine cannot accept any more dime values. What about environmental stress conditions such as extreme heat or cold, or altitude changes. How does it respond to rain.
Error: No change available in machine. Is customer notified. If I input invalid coins, how does the machine react (i.e. pennies). What if I unplug and plug in the machine to simulate a power outage. Does it reset itself? Does it remember input values up to that point? What if machine can't accept any value since it is so full it requires service.
Security: Use of crowbar to attempt to open machine. What about a photocopied dollar bill. Is this accepted by machine?
Performance: Does the system respond quickly even under stress conditions listed above.
Usability: Is everything clearly labeled? Do customer's understand how to use change mechanism?
Accessibility: Can a blind person use the system? Does system have brail?
Localization: Can system be reset to accept coins from different countries? For example, Canada, Japan?
Integration Testing: Applicant may consider testing coin mechanism in conjunction with other parts.
Good test plan to test a Juicer
Before testing, always read the products spec or owner’s manual. This will give
a tester an understanding of the intended functionality and
limitations.
If product has an owner’s manual, one can also test the owner’s manual itself.
Examples:
az Q: How would you test a juicer (or any household appliances).
A:
Areas to test:
• Hardware
• Assembly (pitcher, blade, washers, locking ring)
• Controls
• Motor (cord)
• Functionality
• Safety (UL approved, child safe)
• Manuals (clear concise accurate)
Expected behavior Scenarios
• Blend liquids.
• Crush ice.
Boundary cases and why
Find limits and or failure conditions
• Water
• Peanut butter (something very viscous)
• Ice
• Wooden spoon
• Baseball
• Marbles
• Nails (will they test beyond the reasonable limits)
• How long will it run continuously?
Unusual user scenarios
• Flour
• Hamster
• Dirt
Expected Failure Points
• Power cord
• Controls
• Washers
• User error
• External influence ( wrong power voltage,
International
• Power supply
• Control decals (understandable, non offensive)
• Varying safety regulation.
If product has an owner’s manual, one can also test the owner’s manual itself.
Examples:
az Q: How would you test a juicer (or any household appliances).
A:
Areas to test:
• Hardware
• Assembly (pitcher, blade, washers, locking ring)
• Controls
• Motor (cord)
• Functionality
• Safety (UL approved, child safe)
• Manuals (clear concise accurate)
Expected behavior Scenarios
• Blend liquids.
• Crush ice.
Boundary cases and why
Find limits and or failure conditions
• Water
• Peanut butter (something very viscous)
• Ice
• Wooden spoon
• Baseball
• Marbles
• Nails (will they test beyond the reasonable limits)
• How long will it run continuously?
Unusual user scenarios
• Flour
• Hamster
• Dirt
Expected Failure Points
• Power cord
• Controls
• Washers
• User error
• External influence ( wrong power voltage,
International
• Power supply
• Control decals (understandable, non offensive)
• Varying safety regulation.
Good Test Plan
Good Test Plans Meet the following criteria
Complete – Test for All Categories
1. Functional
Performance
Boundary
Invalid
Expected Failure Points
Bonus -
Additional Test Areas depending on product:
Accessibility, Usability, International, Regression, etc….
2. Detailed - How many ways can you test each given test scenario. If one is testing assembling the blender, a good tester would evaluate whether it is easy, intuitive, safe, secure and functional. Don’t just state “ I would test assembling the blender” That is an example of a poor test scenario.
3. Creative – Impress the manager by thinking outside the box.
Basically, each specific test case should answer the following three questions
1. What am I going to test for?
2. How am I going to test it?
3. What are the expected results?
Complete – Test for All Categories
1. Functional
Performance
Boundary
Invalid
Expected Failure Points
Bonus -
Additional Test Areas depending on product:
Accessibility, Usability, International, Regression, etc….
2. Detailed - How many ways can you test each given test scenario. If one is testing assembling the blender, a good tester would evaluate whether it is easy, intuitive, safe, secure and functional. Don’t just state “ I would test assembling the blender” That is an example of a poor test scenario.
3. Creative – Impress the manager by thinking outside the box.
Basically, each specific test case should answer the following three questions
1. What am I going to test for?
2. How am I going to test it?
3. What are the expected results?
Subscribe to:
Posts (Atom)