Risk - Definition
Risk is the possibility of the negative or undesirable outcome.
A risk is something that has not happened yet and it may never happen; it is a potential problem.
Not all projects are subject to the same risks.
Test items can also have risks associated with them.
Importance
We are concerned about risks because, if one of them did happen, we’d feel a negative impact.Not all software systems carry the same level of risk and not all problems have same impact when they occur.
Risk Levels:
Level of Risk has three components to it
-Potential Consequences:Impact
-Likelihood of Occurrence: Probability
*More likely the outcome is, the worse the risk
*In Future: Between 0% and 100% (possibility)
*In Past: Either 0% or 100%
-Detection Capability
*How well we can identify the risk?
-Every testing effort has risks associated with it.
*Testing software with a high degree of criticality, complexity, or a tight delivery deadline all impose risks that may have negative impacts on project goals.
Risks - Classification:
We can classify risks into
*Project Risk
*Product Risk
*Project Risk:
-factors relating to the way work is carried out
*Product Risk:
-factors relating to what is produced by work.
*Product risk might influence project risk.
Risks - Options
For any risk, we have four typical options:
-Mitigate: Take steps in advance to reduce the likelihood (possibly the impact) of the risk.
-Contingency: Have a plan in place to reduce the impact should the risk become an outcome.
-Transfer: Convince the member of the team or project stakeholder to reduce the likelihood or accept the impact of the risk.
-Ignore: Do nothing about the risk when there is little that can be done or when the likelihood and impact are low.
Risk Management - Steps
*Identify risks related to testing
Project Plan, Requirements, Design & Implementation Phases
Risk Repository
*Prioritize and assess the risks
Use Six Sigma Prioritization Tool : FMEA
*Identify mitigation / contingency / transfer plan
*Monitor & Track risks throughout using Risk Register.
-Use GCRAILS document
Monday
Tuesday
Cyclomatic complexity
Cyclomatic complexity is a software metric (measurement) in computational complexity theory. It was developed by Thomas McCabe and is used to measure the complexity of a program. It directly measures the number of linearly independent paths through a program's source code.
The concept, although not the method, is somewhat similar to that of general text complexity measured by the Flesch-Kincaid Readability Test.
Cyclomatic complexity is computed using a graph that describes the control flow of the program. The nodes of the graph correspond to the commands of a program. A directed edge connects two nodes if the second command might be executed immediately after the first command.
Definition
M = E − N + P
where
M = cyclomatic complexity
E = the number of edges of the graph
N = the number of nodes of the graph
P = the number of connected components.
"M" is alternatively defined to be one larger than the number of decision points (IFs, UNTILs, ENDs...) in a module (function, procedure, chart node, etc.), or more generally a system.
Alternative definition
v(G) = e − n + 2
G is a program's flowgraph
e is the number of arcs in the flowgraph
n is the number of nodes in the flowgraph
Alternative way
There is another simple way to determine the cyclomatic number. This is done by counting the number of closed loops in the flow graph, and incrementing that number by one.
i.e.
M = Number of closed loops + 1
Where
M = Cyclomatic number.
Note that there should be at least M test cases for testing the particular flow graph. This is the bare-minimum necessity to test the flow graph.
Key Concept
McCabe's rule of thumb is that "modules" with M of ten or more are error-prone. CASE structures are an exception. Due to their structure, they can have more than ten decisions without increasing the likelihood of errors.
The cyclomatic complexity of a section of source code is the count of the number of paths through the source code. For instance if the source code contained no decision points such as IF statements or FOR loops, the complexity would be 1 since there is only a single path through the code. If the code had a single IF statement there would be two paths through the code, one path where the IF statement is evaluated as TRUE and one path where the IF statement is evaluated as FALSE.
Cyclomatic complexity is normally calculated by creating a graph of the source code with each line of source code being a node on the graph and arrows between the nodes showing the execution pathways. As some programming languages can be quite terse and compact, a source code statement when developing the graph may actually create several nodes in the graph (for instance when using the C and C++ language "?" conditional operator (also known as the "ternary operator") within a function call interface).
In general, in order to fully test a module all execution paths through the module should be exercised. This implies a module with a high complexity number requires more testing effort than a module with a lower value since the higher complexity number indicates more pathways through the code. This also implies that a module with higher complexity is more difficult for a programmer to understand since the programmer must understand the different pathways and the results of those pathways.
One would also expect that a module with higher complexity would tend to have lower cohesion (less than functional cohesion) than a module with lower complexity. The possible correlation between higher complexity measure with a lower level of cohesion is predicated on a module with more decision points generally implementing more than a single well defined function. However there are certain types of modules that one would expect to have a high complexity number, such as user interface (UI) modules containing source code for data validation and error recovery.
Cyclomatic complexity (cc) is used in white box
testing for calculating the complexity of the code.
(1) cc = No. of direct path.
(2) cc = No. of closed loops + 1
(3) cc = No. of edges - no. of nodes + 2
The concept, although not the method, is somewhat similar to that of general text complexity measured by the Flesch-Kincaid Readability Test.
Cyclomatic complexity is computed using a graph that describes the control flow of the program. The nodes of the graph correspond to the commands of a program. A directed edge connects two nodes if the second command might be executed immediately after the first command.
Definition
M = E − N + P
where
M = cyclomatic complexity
E = the number of edges of the graph
N = the number of nodes of the graph
P = the number of connected components.
"M" is alternatively defined to be one larger than the number of decision points (IFs, UNTILs, ENDs...) in a module (function, procedure, chart node, etc.), or more generally a system.
Alternative definition
v(G) = e − n + 2
G is a program's flowgraph
e is the number of arcs in the flowgraph
n is the number of nodes in the flowgraph
Alternative way
There is another simple way to determine the cyclomatic number. This is done by counting the number of closed loops in the flow graph, and incrementing that number by one.
i.e.
M = Number of closed loops + 1
Where
M = Cyclomatic number.
Note that there should be at least M test cases for testing the particular flow graph. This is the bare-minimum necessity to test the flow graph.
Key Concept
McCabe's rule of thumb is that "modules" with M of ten or more are error-prone. CASE structures are an exception. Due to their structure, they can have more than ten decisions without increasing the likelihood of errors.
The cyclomatic complexity of a section of source code is the count of the number of paths through the source code. For instance if the source code contained no decision points such as IF statements or FOR loops, the complexity would be 1 since there is only a single path through the code. If the code had a single IF statement there would be two paths through the code, one path where the IF statement is evaluated as TRUE and one path where the IF statement is evaluated as FALSE.
Cyclomatic complexity is normally calculated by creating a graph of the source code with each line of source code being a node on the graph and arrows between the nodes showing the execution pathways. As some programming languages can be quite terse and compact, a source code statement when developing the graph may actually create several nodes in the graph (for instance when using the C and C++ language "?" conditional operator (also known as the "ternary operator") within a function call interface).
In general, in order to fully test a module all execution paths through the module should be exercised. This implies a module with a high complexity number requires more testing effort than a module with a lower value since the higher complexity number indicates more pathways through the code. This also implies that a module with higher complexity is more difficult for a programmer to understand since the programmer must understand the different pathways and the results of those pathways.
One would also expect that a module with higher complexity would tend to have lower cohesion (less than functional cohesion) than a module with lower complexity. The possible correlation between higher complexity measure with a lower level of cohesion is predicated on a module with more decision points generally implementing more than a single well defined function. However there are certain types of modules that one would expect to have a high complexity number, such as user interface (UI) modules containing source code for data validation and error recovery.
Cyclomatic complexity (cc) is used in white box
testing for calculating the complexity of the code.
(1) cc = No. of direct path.
(2) cc = No. of closed loops + 1
(3) cc = No. of edges - no. of nodes + 2
Monday
How Web Servers Work?
Web servers are responsible for storing and exchanging information with other machines. Because of this, at least two participants are required for each exchange of information: a client, which requests the information, and a server, which stores it. Each side also requires a piece of software to negotiate the exchange of data; in the case of the client, a browser like Netscape or Internet Explorer is used.
A simple exchange between the client machine and Web Server goes like this:
1. The client's browser dissects the URL in to a number of separate parts, including address, path name and protocol.
2. A Domain Name Server (DNS) translates the domain name the user has entered in to its IP address, a numeric combination that represents the site's true address on the Internet (a domain name is merely a "front" to make site addresses easier to remember).
3. The browser now determines which protocol (the language client machines use to communicate with servers) should be used. Examples of protocols include FTP, or File Transfer Protocol, and HTTP, Hypertext Transfer Protocol.
4. The server sends a GET request to Web Server to retrieve the address it has been given. For example, when a user types http://www.example.com/1.jpg, the browser sends a GET 1.jpg command to example.com and waits for a response. The server now responds to the browser's requests. It verifies that the given address exists, finds the necessary files, runs the appropriate scripts, exchanges cookies if necessary, and returns the results back to the browser. If it cannot locate the file, the server sends an error message to the client.
5. The browser translates the data it has been given in to HTML and displays the results to the user. This process is repeated until the client browser leaves the site.
A simple exchange between the client machine and Web Server goes like this:
1. The client's browser dissects the URL in to a number of separate parts, including address, path name and protocol.
2. A Domain Name Server (DNS) translates the domain name the user has entered in to its IP address, a numeric combination that represents the site's true address on the Internet (a domain name is merely a "front" to make site addresses easier to remember).
3. The browser now determines which protocol (the language client machines use to communicate with servers) should be used. Examples of protocols include FTP, or File Transfer Protocol, and HTTP, Hypertext Transfer Protocol.
4. The server sends a GET request to Web Server to retrieve the address it has been given. For example, when a user types http://www.example.com/1.jpg, the browser sends a GET 1.jpg command to example.com and waits for a response. The server now responds to the browser's requests. It verifies that the given address exists, finds the necessary files, runs the appropriate scripts, exchanges cookies if necessary, and returns the results back to the browser. If it cannot locate the file, the server sends an error message to the client.
5. The browser translates the data it has been given in to HTML and displays the results to the user. This process is repeated until the client browser leaves the site.
Sunday
Software Test Estimation Approaches
The 'best approach' is highly dependent on the particular organization and project and the experience of the personnel involved.
For example, given two software projects of similar complexity and size, the appropriate test effort for one project might be very large if it was for life-critical medical equipment software, but might be much smaller for the other project if it was for a low-cost computer game. A test estimation approach that only considered size and complexity might be appropriate for one project but not for the other.
Following are some approaches to consider.
Implicit Risk Context Approach:
A typical approach to test estimation is for a project manager or QA manager to implicitly use risk context, in combination with past personal experiences in the organization, to choose a level of resources to allocate to testing. In many organizations, the 'risk context' is assumed to be similar from one project to the next, so there is no explicit consideration of risk context. (Risk context might include factors such as the organization's typical software quality levels, the software's intended use, the experience level of developers and testers, etc.) This is essentially an intuitive guess based on experience.
Metrics-Based Approach:
A useful approach is to track past experience of an organization's various projects and the associated test effort that worked well for projects. Once there is a set of data covering characteristics for a reasonable number of projects, then this 'past experience' information can be used for future test project planning. (Determining and collecting useful project metrics over time can be an extremely difficult task.) For each particular new project, the 'expected' required test time can be adjusted based on whatever metrics or other information is available, such as function point count, number of external system interfaces, unit testing done by developers, risk levels of the project, etc. In the end, this is essentially 'judgement based on documented experience', and is not easy to do successfully.
Test Work Breakdown Approach:
Another common approach is to decompose the expected testing tasks into a collection of small tasks for which estimates can, at least in theory, be made with reasonable accuracy. This of course assumes that an accurate and predictable breakdown of testing tasks and their estimated effort is feasible. In many large projects, this is not the case. For example, if a large number of bugs are being found in a project, this will add to the time required for testing, retesting, bug analysis and reporting. It will also add to the time required for development, and if development schedules and efforts do not go as planned, this will further impact testing.
Iterative Approach:
In this approach for large test efforts, an initial rough testing estimate is made. Once testing begins, a more refined estimate is made after a small percentage (eg, 1%) of the first estimate's work is done. At this point testers have obtained additional test project knowledge and a better understanding of issues, general software quality, and risk. Test plans and schedules can be refactored if necessary and a new estimate provided. Then a yet-more-refined estimate is made after a somewhat larger percentage (eg, 2%) of the new work estimate is done. Repeat the cycle as necessary/appropriate.
Percentage-of-Development Approach:
Some organizations utilize a quick estimation method for testing based on the estimated programming effort. For example, if a project is estimated to require 1000 hours of programming effort, and the organization normally finds that a 40% ratio for testing is appropriate, then an estimate of 400 hours for testing would be used. This approach may or may not be useful depending on the project-to-project variations in risk, personnel, types of applications, levels of complexity, etc.
Successful test estimation is a challenge for most organizations, since few can accurately estimate software project development efforts, much less the testing effort of a project. It is also difficult to attempt testing estimates without first having detailed information about a project, including detailed requirements, the organization's experience with similar projects in the past, and an understanding of what should be included in a 'testing' estimation for a project (functional testing? unit testing? reviews? inspections? load testing? security testing?)
With agile software development approaches, test effort estimations may be unnecessary if pure test-driven development is utilized. In general, agile-based projects by their nature will not be heavily dependent on large testing efforts, since they emphasize the construction of releasable software in very short iteration cycles. Therefore test effort estimates may not be as difficult and the impact of inaccurate estimates will be minimized.
For example, given two software projects of similar complexity and size, the appropriate test effort for one project might be very large if it was for life-critical medical equipment software, but might be much smaller for the other project if it was for a low-cost computer game. A test estimation approach that only considered size and complexity might be appropriate for one project but not for the other.
Following are some approaches to consider.
Implicit Risk Context Approach:
A typical approach to test estimation is for a project manager or QA manager to implicitly use risk context, in combination with past personal experiences in the organization, to choose a level of resources to allocate to testing. In many organizations, the 'risk context' is assumed to be similar from one project to the next, so there is no explicit consideration of risk context. (Risk context might include factors such as the organization's typical software quality levels, the software's intended use, the experience level of developers and testers, etc.) This is essentially an intuitive guess based on experience.
Metrics-Based Approach:
A useful approach is to track past experience of an organization's various projects and the associated test effort that worked well for projects. Once there is a set of data covering characteristics for a reasonable number of projects, then this 'past experience' information can be used for future test project planning. (Determining and collecting useful project metrics over time can be an extremely difficult task.) For each particular new project, the 'expected' required test time can be adjusted based on whatever metrics or other information is available, such as function point count, number of external system interfaces, unit testing done by developers, risk levels of the project, etc. In the end, this is essentially 'judgement based on documented experience', and is not easy to do successfully.
Test Work Breakdown Approach:
Another common approach is to decompose the expected testing tasks into a collection of small tasks for which estimates can, at least in theory, be made with reasonable accuracy. This of course assumes that an accurate and predictable breakdown of testing tasks and their estimated effort is feasible. In many large projects, this is not the case. For example, if a large number of bugs are being found in a project, this will add to the time required for testing, retesting, bug analysis and reporting. It will also add to the time required for development, and if development schedules and efforts do not go as planned, this will further impact testing.
Iterative Approach:
In this approach for large test efforts, an initial rough testing estimate is made. Once testing begins, a more refined estimate is made after a small percentage (eg, 1%) of the first estimate's work is done. At this point testers have obtained additional test project knowledge and a better understanding of issues, general software quality, and risk. Test plans and schedules can be refactored if necessary and a new estimate provided. Then a yet-more-refined estimate is made after a somewhat larger percentage (eg, 2%) of the new work estimate is done. Repeat the cycle as necessary/appropriate.
Percentage-of-Development Approach:
Some organizations utilize a quick estimation method for testing based on the estimated programming effort. For example, if a project is estimated to require 1000 hours of programming effort, and the organization normally finds that a 40% ratio for testing is appropriate, then an estimate of 400 hours for testing would be used. This approach may or may not be useful depending on the project-to-project variations in risk, personnel, types of applications, levels of complexity, etc.
Successful test estimation is a challenge for most organizations, since few can accurately estimate software project development efforts, much less the testing effort of a project. It is also difficult to attempt testing estimates without first having detailed information about a project, including detailed requirements, the organization's experience with similar projects in the past, and an understanding of what should be included in a 'testing' estimation for a project (functional testing? unit testing? reviews? inspections? load testing? security testing?)
With agile software development approaches, test effort estimations may be unnecessary if pure test-driven development is utilized. In general, agile-based projects by their nature will not be heavily dependent on large testing efforts, since they emphasize the construction of releasable software in very short iteration cycles. Therefore test effort estimates may not be as difficult and the impact of inaccurate estimates will be minimized.
Wednesday
Testing Techniques
1. White Box Testing (Unit Testing)
a. Statement Coverage
i. Flow Graph Notation
ii. Cyclomatic Complexity
iii. Deriving Test cases
iv. Graph Matrices.
b. Decision Coverage
c. Condition Coverage
d. Decision / Condition Coverage
e. Multiple Condition Coverage
f. Transaction flow Testing
g. Control Structure Testing
i. Condition Testing
ii. Data Flow Testing
iii. Loop Testing
h. Syntax Testing
i. Cause – Effect graphing Techniques
j. State Transition Testing
k. Logic Based Testing
2. Black Box Testing
a. Equivalence Partitioning
b. Boundary Value Analysis
c. Error Guessing
d. Comparison or Back-to-Back Testing
e. Functional Testing
3. Human Testing
a. Inspections
b. Walkthroughs
c. Desk checking
4. Integration (Incremental)
a. Top Bottom Integration
b. Bottom Up Integration
c. Big Bang Testing
d. Sandwich Integration Strategy
5. Validation
6. System Testing
a. Recovery Testing,
b. Security Testing,
c. Load / Volume Testing,
d. Stress Testing,
e. Performance Testing,
f. Alpha and Beta Testing,
g. Compliance Testing
h. Usability Testing
i. Compatibility Testing
j. Reliability Testing
k. Device & configuration testing
l. Multitasking Testing
m. Serviceability Testing
n. Installation Testing
o. Install – Un-installation Testing
p. Module/Unit Testing
q. Function Testing
r. Operations Testing
7. Installation Testing
8. Acceptance Testing
9. Regression Testing
10. Static / Dynamic Testing
11. Manual / Automatic
12. Sanity Testing
13. Thread Testing
14. Exploratory Testing.
15. Functional Testing Techniques
a. Requirements Testing
b. Regression Testing
c. Error Handling Testing
d. Intersystem Testing
e. Parallel Testing
16. Maintenance Testing
a. Corrective maintenance Testing
b. Perfective Maintenance Testing
c. Adaptive Maintenance Testing
17. Object Oriented Testing Techniques
a. Fault Based Testing
b. Class level Testing
i. Random Testing
ii. Partition Testing
1. State based partitioning
2. Attribute based partitioning
3. Category based partitioning
c. Scenario based Testing
18. Risk based Testing
19. Input validation Testing
20. End-to-end testing
21. Ad-hoc testing
22. Mutation Testing
23. Install/Uninstall Testing
24. Site Testing
a. Statement Coverage
i. Flow Graph Notation
ii. Cyclomatic Complexity
iii. Deriving Test cases
iv. Graph Matrices.
b. Decision Coverage
c. Condition Coverage
d. Decision / Condition Coverage
e. Multiple Condition Coverage
f. Transaction flow Testing
g. Control Structure Testing
i. Condition Testing
ii. Data Flow Testing
iii. Loop Testing
h. Syntax Testing
i. Cause – Effect graphing Techniques
j. State Transition Testing
k. Logic Based Testing
2. Black Box Testing
a. Equivalence Partitioning
b. Boundary Value Analysis
c. Error Guessing
d. Comparison or Back-to-Back Testing
e. Functional Testing
3. Human Testing
a. Inspections
b. Walkthroughs
c. Desk checking
4. Integration (Incremental)
a. Top Bottom Integration
b. Bottom Up Integration
c. Big Bang Testing
d. Sandwich Integration Strategy
5. Validation
6. System Testing
a. Recovery Testing,
b. Security Testing,
c. Load / Volume Testing,
d. Stress Testing,
e. Performance Testing,
f. Alpha and Beta Testing,
g. Compliance Testing
h. Usability Testing
i. Compatibility Testing
j. Reliability Testing
k. Device & configuration testing
l. Multitasking Testing
m. Serviceability Testing
n. Installation Testing
o. Install – Un-installation Testing
p. Module/Unit Testing
q. Function Testing
r. Operations Testing
7. Installation Testing
8. Acceptance Testing
9. Regression Testing
10. Static / Dynamic Testing
11. Manual / Automatic
12. Sanity Testing
13. Thread Testing
14. Exploratory Testing.
15. Functional Testing Techniques
a. Requirements Testing
b. Regression Testing
c. Error Handling Testing
d. Intersystem Testing
e. Parallel Testing
16. Maintenance Testing
a. Corrective maintenance Testing
b. Perfective Maintenance Testing
c. Adaptive Maintenance Testing
17. Object Oriented Testing Techniques
a. Fault Based Testing
b. Class level Testing
i. Random Testing
ii. Partition Testing
1. State based partitioning
2. Attribute based partitioning
3. Category based partitioning
c. Scenario based Testing
18. Risk based Testing
19. Input validation Testing
20. End-to-end testing
21. Ad-hoc testing
22. Mutation Testing
23. Install/Uninstall Testing
24. Site Testing
Standard Test Plan
There is no one universally accepted standard for test planning. However, there is great consistency between the different organizations that have defined a test plan standard. This section will begin with a discussion of what is normally contained in a test plan, and then provide an example of a test plan standard that is consistent with the test plan standards provided by major standard-setting bodies such as the Institute of Electrical and Electronics Engineers (IEEE) and the National Institute of Standards in Technology (NIST) a part of the U.S. government.Test Plans and their formats vary from company to company, but the best examples contain most of the elements discussed here. The Table of Contents of a test plan might contain the
following:
• Test Scope
•Test Objectives
• Assumptions
• Risk Analysis
• Test Design
• Roles & Responsibilities
• Test Schedule & Resources
• Test Data Management
• Test Environment
• Communication Approach
• Test Tools
1 Test Scope
• This section answers two equally important questions: “What will be covered in the
test?” and “What will not be covered in the test?” The answers to either of these
questions might include:
• Specific functional or structural requirements
• System interfaces
• Infrastructure components (e.g., network stability)
• Supplemental deliverables, such as application documentation
2 Test Objectives
A test objective is simply a testing “goal.” It is a statement of what the tester is expected to accomplish or validate during a specific testing activity. Test objectives:
• Guide the development of test cases, procedures, and test data.
• Enable the tester and project managers to gauge testing progress and success.
• Enhance communication both within and outside of the project team by helping to
define the scope of the testing effort.Each objective should include a high-level description of the expected test results in measurable terms, and should be prioritized. In cases where test time is cut short, test cases supporting the highest priority objectives would be executed first.
3 Assumptions
These assumptions document test prerequisites, which if not met, could have a negative impact on the test. The test plan should communicate the risk that is introduced if these expectations are not met. Examples of assumptions include:
• Skill level of test resources
• Test budget
• State of the application at the start of testing
• Tools available
• Availability of test equipment
Entrance and exit criteria for each stage of testing could be documented here.
4 Risk Analysis
Although the test manager should work with the project team to identify risks to the project,this section of the plan documents test risks and their possible impact on the test effort. Some teams may incorporate these risks into project risk documentation if available. Risks that could impact testing include:
• Availability of downstream application test resources to perform system Integration or regression testing
• Implementation of new test automation tools
• Sequence and increments of code delivery
• New technology
5 Test Design
The test design details the following:
• The types of tests that must be conducted
• The stages of testing that are required (e.g., Unit, Integration, System, Performance,and Usability)
• Outlines the sequence and timing of tests
6 Roles & Responsibilities
This section of the test plan defines who is responsible for each stage or type of testing. A responsibility matrix is an effective means of documenting these assignments. Note that although the Test Manager usually writes the test plan, it does not just include information on tests that the test team will execute.
7 Test Schedule & Planned Resources
The test schedule section includes the following:
• Major test activities
• Sequence of tests
• Dependence on other project activities
• Initial estimates for each activity
The plan should not be maintained separately, but incorporated into the overall Project Plan.
Test resource planning includes:
• People, tools, and facilities
• An analysis of skill sets so that training requirements
8 Test Data Management
This section of the plan defines the data required for testing, as well as the infrastructure requirements to manage test data. It includes:
• Methods for preparing test data
• Backup and rollback procedures
• High-level data requirements, data sources, and methods for preparation (production
extract or test data generation)
• Whether data conditioning or conversion will be required
• Data security issues
9 Test Environment
Environment requirements for each stage and type of testing should be outlined in this section of the plan, for example:
• Unit testing may be conducted in the development environment, while separate
environments may be needed for integration and system testing
• Procedures for configuration management, release, and version control should be
outlined
• Requirements for hardware and software configurations
• The location of individual test events
• The defect tracking mechanisms to be used
10 Communication Approach
In the complex, matrix environment required for testing in most companies, various
communication mechanisms are required. These avenues should include
• Formal and informal meetings
• Working sessions
• Processes, such as defect tracking
• Tools, such as issue and defect tracking, electronic bulletin boards, notes databases,and Intranet sites
• Techniques, such as escalation procedures or the use of white boards for posting
current state of testing (e.g., test environment down)
• Miscellaneous items such as project contact lists, meeting audiences, and frequency of defect reporting.
11 ToolsAny tools that will be needed to support the testing process should be included here. Tools are usually used for:
• Workplan development
• Test planning and management
Configuration management
• Test script development
• Test data conditioning
• Test execution
• Automated test tools
• Stress/load testing
• Results verification
• Defect tracking
The information outlined here cannot usually all be completed at once, but is captured in greater levels of detail as the project progresses through the life cycle.
following:
• Test Scope
•Test Objectives
• Assumptions
• Risk Analysis
• Test Design
• Roles & Responsibilities
• Test Schedule & Resources
• Test Data Management
• Test Environment
• Communication Approach
• Test Tools
1 Test Scope
• This section answers two equally important questions: “What will be covered in the
test?” and “What will not be covered in the test?” The answers to either of these
questions might include:
• Specific functional or structural requirements
• System interfaces
• Infrastructure components (e.g., network stability)
• Supplemental deliverables, such as application documentation
2 Test Objectives
A test objective is simply a testing “goal.” It is a statement of what the tester is expected to accomplish or validate during a specific testing activity. Test objectives:
• Guide the development of test cases, procedures, and test data.
• Enable the tester and project managers to gauge testing progress and success.
• Enhance communication both within and outside of the project team by helping to
define the scope of the testing effort.Each objective should include a high-level description of the expected test results in measurable terms, and should be prioritized. In cases where test time is cut short, test cases supporting the highest priority objectives would be executed first.
3 Assumptions
These assumptions document test prerequisites, which if not met, could have a negative impact on the test. The test plan should communicate the risk that is introduced if these expectations are not met. Examples of assumptions include:
• Skill level of test resources
• Test budget
• State of the application at the start of testing
• Tools available
• Availability of test equipment
Entrance and exit criteria for each stage of testing could be documented here.
4 Risk Analysis
Although the test manager should work with the project team to identify risks to the project,this section of the plan documents test risks and their possible impact on the test effort. Some teams may incorporate these risks into project risk documentation if available. Risks that could impact testing include:
• Availability of downstream application test resources to perform system Integration or regression testing
• Implementation of new test automation tools
• Sequence and increments of code delivery
• New technology
5 Test Design
The test design details the following:
• The types of tests that must be conducted
• The stages of testing that are required (e.g., Unit, Integration, System, Performance,and Usability)
• Outlines the sequence and timing of tests
6 Roles & Responsibilities
This section of the test plan defines who is responsible for each stage or type of testing. A responsibility matrix is an effective means of documenting these assignments. Note that although the Test Manager usually writes the test plan, it does not just include information on tests that the test team will execute.
7 Test Schedule & Planned Resources
The test schedule section includes the following:
• Major test activities
• Sequence of tests
• Dependence on other project activities
• Initial estimates for each activity
The plan should not be maintained separately, but incorporated into the overall Project Plan.
Test resource planning includes:
• People, tools, and facilities
• An analysis of skill sets so that training requirements
8 Test Data Management
This section of the plan defines the data required for testing, as well as the infrastructure requirements to manage test data. It includes:
• Methods for preparing test data
• Backup and rollback procedures
• High-level data requirements, data sources, and methods for preparation (production
extract or test data generation)
• Whether data conditioning or conversion will be required
• Data security issues
9 Test Environment
Environment requirements for each stage and type of testing should be outlined in this section of the plan, for example:
• Unit testing may be conducted in the development environment, while separate
environments may be needed for integration and system testing
• Procedures for configuration management, release, and version control should be
outlined
• Requirements for hardware and software configurations
• The location of individual test events
• The defect tracking mechanisms to be used
10 Communication Approach
In the complex, matrix environment required for testing in most companies, various
communication mechanisms are required. These avenues should include
• Formal and informal meetings
• Working sessions
• Processes, such as defect tracking
• Tools, such as issue and defect tracking, electronic bulletin boards, notes databases,and Intranet sites
• Techniques, such as escalation procedures or the use of white boards for posting
current state of testing (e.g., test environment down)
• Miscellaneous items such as project contact lists, meeting audiences, and frequency of defect reporting.
11 ToolsAny tools that will be needed to support the testing process should be included here. Tools are usually used for:
• Workplan development
• Test planning and management
Configuration management
• Test script development
• Test data conditioning
• Test execution
• Automated test tools
• Stress/load testing
• Results verification
• Defect tracking
The information outlined here cannot usually all be completed at once, but is captured in greater levels of detail as the project progresses through the life cycle.
Tuesday
Database Testing Updates
Data Integrity Test :
Once a value undergoes any of the above actions (update / Delete / Insert) the database should be verified for the changes performed on related entities i.e., Foreign key / Primary key and all dependent entities.
Store Procedure Test :
Every Stored Procedure is to be tested separately for its functionality ( Based on Separate functions it performs ) Stored procedures need to be broken up into Action Items based on Functions and then Each action item needs to be tested separately as the results of Complete Stored procedure.
Execution may differ from the results obtained by partial execution. This also helps in validating the modularity of Code (White Box).
In the case of stored procedures, to come up with test cases one can consider the following:
1. The no. of arguments being passed
2. The data type of each of the arguments being passed
3. The order of the arguments being passed
4. The return value
5. The data type of the return value
Based on these you can write both positive and negative test cases, consider a simple example of a stored procedure taking 2 numbers as input and returning the sum of the 2 numbers.
Type Test :
This test is performed to verify that the data types used by the DBA are same as expected by agreed upon by the developer. Often the data types chosen by developers are not the same as suggested by database administrators , especially for the fields like
A) Phone (Num./Text)
B) Description (Large Text)
These kinds of mismatches often do not effect the functionality and normal execution of code, but prove to be very Costly during Updating of product or during addition of features to the existing modules.
Data Size Test :
Performance of Data size testing is often done only at the front end during the unit testing , but it is essential to perform it at back end separately .
This ensures smooth transition while appending functionality and integrating modules as during these Phases , the data is passed to the system with direct user interaction and bypassing front end validation.
Environment Driven Item Test :
Event Driven Actions (Triggers or Scheduled Actions) needs to be tested on two parameters.
A) Events that trigger these actions: Here QA needs to check the events on which any of the trigger can get fired/executed . This testing can be done with the help of DBA also .
B) Actions performed by the above stated events. Here the contents of such stored procedures or scheduled actions are verified for the functionality .
Input Item Verification :
This is the process of verification of the input items (Though this is not totally a part of database testing , but this has to be performed essentially during database testing of the Web based applications.
Often it is seen that the input items (Text Box / RTB / Combination Box / Active-X controls) are tested for validation only at front end (Screen testing) but these are again to be tested with junk Character values to confirm that they do not push in such characters which the databases Often misrepresent or Replace with other characters (this testing can partially be performed during Unit testing also by the developer.
Once a value undergoes any of the above actions (update / Delete / Insert) the database should be verified for the changes performed on related entities i.e., Foreign key / Primary key and all dependent entities.
Store Procedure Test :
Every Stored Procedure is to be tested separately for its functionality ( Based on Separate functions it performs ) Stored procedures need to be broken up into Action Items based on Functions and then Each action item needs to be tested separately as the results of Complete Stored procedure.
Execution may differ from the results obtained by partial execution. This also helps in validating the modularity of Code (White Box).
In the case of stored procedures, to come up with test cases one can consider the following:
1. The no. of arguments being passed
2. The data type of each of the arguments being passed
3. The order of the arguments being passed
4. The return value
5. The data type of the return value
Based on these you can write both positive and negative test cases, consider a simple example of a stored procedure taking 2 numbers as input and returning the sum of the 2 numbers.
Type Test :
This test is performed to verify that the data types used by the DBA are same as expected by agreed upon by the developer. Often the data types chosen by developers are not the same as suggested by database administrators , especially for the fields like
A) Phone (Num./Text)
B) Description (Large Text)
These kinds of mismatches often do not effect the functionality and normal execution of code, but prove to be very Costly during Updating of product or during addition of features to the existing modules.
Data Size Test :
Performance of Data size testing is often done only at the front end during the unit testing , but it is essential to perform it at back end separately .
This ensures smooth transition while appending functionality and integrating modules as during these Phases , the data is passed to the system with direct user interaction and bypassing front end validation.
Environment Driven Item Test :
Event Driven Actions (Triggers or Scheduled Actions) needs to be tested on two parameters.
A) Events that trigger these actions: Here QA needs to check the events on which any of the trigger can get fired/executed . This testing can be done with the help of DBA also .
B) Actions performed by the above stated events. Here the contents of such stored procedures or scheduled actions are verified for the functionality .
Input Item Verification :
This is the process of verification of the input items (Though this is not totally a part of database testing , but this has to be performed essentially during database testing of the Web based applications.
Often it is seen that the input items (Text Box / RTB / Combination Box / Active-X controls) are tested for validation only at front end (Screen testing) but these are again to be tested with junk Character values to confirm that they do not push in such characters which the databases Often misrepresent or Replace with other characters (this testing can partially be performed during Unit testing also by the developer.
Installation Testing
Installation testing is often the most under tested area in testing. This type of testing is performed to ensure that all Installed features and options function properly. It is also performed to verify that all necessary components of the application are, indeed, installed. Installation testing should take care of the following points:
1. To check if while installing product checks for the dependent software / patches say Service pack3.
2. The product should check for the version of the same product on the target machine, say the previous version should not be over installed on the newer version. 3. Installer should give a default installation path say “C:\programs\.”
4. Installer should allow user to install at location other then the default installation path.
5. Check if the product can be installed “Over the Network”
6. Installation should start automatically when the CD is inserted.
7. Installer should give the remove / Repair options.
8. When uninstalling, check that all the registry keys, files, Dll, shortcuts, active X components are removed from the system.
9. Try to install the software without administrative privileges (login as guest).
10. Try installing on different operating system.
11. Try installing on system having non-compliant configuration such as less memory / RAM / HDD.
1. To check if while installing product checks for the dependent software / patches say Service pack3.
2. The product should check for the version of the same product on the target machine, say the previous version should not be over installed on the newer version. 3. Installer should give a default installation path say “C:\programs\.”
4. Installer should allow user to install at location other then the default installation path.
5. Check if the product can be installed “Over the Network”
6. Installation should start automatically when the CD is inserted.
7. Installer should give the remove / Repair options.
8. When uninstalling, check that all the registry keys, files, Dll, shortcuts, active X components are removed from the system.
9. Try to install the software without administrative privileges (login as guest).
10. Try installing on different operating system.
11. Try installing on system having non-compliant configuration such as less memory / RAM / HDD.
Wednesday
Client/Server Architectures
Two tier architectures :With two tier client/server architectures, the user system interface is usually located in the user's desktop environment and the database management services are usually in a server that is a more powerful machine that services many clients. Processing management is split between the user system interface environment and the database management server environment. The database management server provides stored procedures and triggers. There are a number of software vendors that provide tools to simplify development of applications for the two tier client/server architecture.
The two tier client/server architecture is a good solution for distributed computing when work groups are defined as a dozen to 100 people interacting on a LAN simultaneously. It does have a number of limitations. When the number of users exceeds 100, performance begins to deteriorate. This limitation is a result of the server maintaining a connection via "keep-alive" messages with each client, even when no work is being done. A second limitation of the two tier architecture is that implementation of processing management services using vendor proprietary database procedures restricts flexibility and choice of DBMS for applications. Finally, current implementations of the two tier architecture provide limited flexibility in moving (repartitioning) program functionality from one server to another without manually regenerating procedural code.
Three tier architectures :The three tier architecture (also referred to as the multi-tier architecture) emerged to overcome the limitations of the two tier architecture. In the three tier architecture, a middle tier was added between the user system interface client environment and the database management server environment. There are a variety of ways of implementing this middle tier, such as transaction processing monitors, message servers, or application servers. The middle tier can perform queuing, application execution, and database staging. For example, if the middle tier provides queuing, the client can deliver its request to the middle layer and disengage because the middle tier will access the data and return the answer to the client. In addition the middle layer adds scheduling and prioritization for work in progress. The three tier client/server architecture has been shown to improve performance for groups with a large number of users (in the thousands) and improves flexibility when compared to the two tier approach. Flexibility in partitioning can be a simple as "dragging and dropping" application code modules onto different computers in some three tier architectures. A limitation with three tier architectures is that the development environment is reportedly more difficult to use than the visually-oriented development of two tier applications. Recently, mainframes have found a new use as servers in three tier architectures (see Mainframe Server Software Architectures).
The two tier client/server architecture is a good solution for distributed computing when work groups are defined as a dozen to 100 people interacting on a LAN simultaneously. It does have a number of limitations. When the number of users exceeds 100, performance begins to deteriorate. This limitation is a result of the server maintaining a connection via "keep-alive" messages with each client, even when no work is being done. A second limitation of the two tier architecture is that implementation of processing management services using vendor proprietary database procedures restricts flexibility and choice of DBMS for applications. Finally, current implementations of the two tier architecture provide limited flexibility in moving (repartitioning) program functionality from one server to another without manually regenerating procedural code.
Three tier architectures :The three tier architecture (also referred to as the multi-tier architecture) emerged to overcome the limitations of the two tier architecture. In the three tier architecture, a middle tier was added between the user system interface client environment and the database management server environment. There are a variety of ways of implementing this middle tier, such as transaction processing monitors, message servers, or application servers. The middle tier can perform queuing, application execution, and database staging. For example, if the middle tier provides queuing, the client can deliver its request to the middle layer and disengage because the middle tier will access the data and return the answer to the client. In addition the middle layer adds scheduling and prioritization for work in progress. The three tier client/server architecture has been shown to improve performance for groups with a large number of users (in the thousands) and improves flexibility when compared to the two tier approach. Flexibility in partitioning can be a simple as "dragging and dropping" application code modules onto different computers in some three tier architectures. A limitation with three tier architectures is that the development environment is reportedly more difficult to use than the visually-oriented development of two tier applications. Recently, mainframes have found a new use as servers in three tier architectures (see Mainframe Server Software Architectures).
Subscribe to:
Posts (Atom)