The products developed into different phases of software testing life cycle and shared with the stake holders are known as Test Artifacts. Generally the software test team should prepare these artifacts and they are supposed to take sign off on those artifacts from the stake holders to make sure that there is no communication gap between customer and test team. Any change requirement during the testing process can also be tracked easily through these test artifacts. Some of the deliverable test artifacts are discussed in details below:
3.1 Test Strategy
Test Strategy is generally a static document prepared by the higher management of the testing team to define the objectives of all test stages and the techniques that apply. The testing strategy also forms the basis for the creation of a standardized documentation set and facilitates communication of the test processes. Test Approach/ Test Architecture are synonyms for Test Strategy.
To develop the test strategy the following things should be considered:
· Objectives of testing
· guidelines for testing
. requirements for testing such as functional specifications, acceptance criteria and test scenarios
· Approach for testing, for example Requirement Driven Testing
· Roles and responsibilities
· What are the levels of testing
· Test requirements i.e. test artifacts such as functional specifications, acceptance criteria and test scenarios
· Entry and exit criteria
· management of defects i.e. what to do when a defect is reported
· What test reports will be provided
· Test environment information and migration procedures
· Test Constraints
· Test Risk including project and product risks
3.2 Test Plan
Test Plan can be defined as a document that describes the scope, approach , resources and schedule of intended test activities. Generally the Test Lead or the Test Manager prepares this document and this actually identifies test items, the features to be tested, the testing tasks , resources and risks involved through the process. The main purpose of preparing the test plan is to make all stake holders aware about the scope, responsibilities, timeline and deliverables for the project under test. Test Plan is not a static document and any change (either in the requirement or schedule) should be incorporated within the Test Plan and a ‘Revision History’ section should be maintained to track the changes easily. A review sign off is very important for a Test Plan since it means that everyone agree the contents of the test plan and any dispute in later phase of the project can be avoided.
To develop the test strategy the following things should be considered:
· Scope Of Testing
· Testing Approach
· Test Entry Criteria
· Test Exit Criteria
· Test Schedules
· Hardware/Software Requirements
· Risks & Mitigation Plans
· Test Artifacts
· Revision History
3.3 Test Scenario
The two terms ‘Test Scenario’ and ‘Test Case’ are often used synonymously but these two are different. A test scenario is a small statement to verify one particular area of an application and these are mainly prepared after reviewing the functional requirements. A particular area or module of an application can have one or many scenarios based on the complexity of the module. From one Test Scenario one or many Test cases can be formed
Consider testing on a secure web application and it consists of many web pages. Now ‘Validating the Login Page/Screen’ can be considered as an example of one Test Scenario. Here a tester needs to validate all the objects (URL, Input, button etc.) and the respective functionalities within the page and it all comes as a sub part of the test scenario mentioned.
Developing Test Scenarios and by analyzing the complexities of each scenario Test team can provide a better estimation to the stake holders
3.4 Test Case
A Test Case is a single or set of conditions or steps that a tester should follow to determine whether a particular feature or functionality of the software or application is working as per business requirement or not. A formal test-case is characterized by a known input and by an expected output, which is worked out before the test execution and as per requirement.
Now there are two ways of writing test case through which tester can verify the functionalities of an object within the software application. Positive Test Cases are written on a system to verify the application behavior on positive test data input. On the other hand Negative Test Cases are written on a system to verify the system behavior on negative or invalid test data input.
For example consider a text box within a web application which can accept only numbers from 1-999. Any value not apart from the range or any alphanumeric value should be discarded by the system. To verify the functionality of the input box properly a tester should write two test cases – one with any input value within 1 – 999 (positive) and the other with input any invalid alphanumeric data / any numeric data not within 1-999 (negative).
Generally a well written test case should contain the following component:
· Test Case ID
· Test Case Name
· Test Case Description
· Expected Result
· Actual Result
· Application/Release Name
3.5 Traceability Matrix
Traceability Matrix is a document maintained in table format which correlates two documents and also gives the relationship between them. As a part of software testing, Requirement Tracing is often used to link between detailed requirements and test design and the end product is known as Requirement Traceability Matrix (RTM). A RTM may be used to determine whether all the current project requirements are verified or covered through the designed test cases.
Generally the Requirement Traceability Matrix contains the following parameters:
· Requirement ID
· Requirement Type (Functional/Business)
· Requirement Description
· Linked Test Cases
· Test Case Result
The Requirement Traceability Matrix is bi-directional, as it tracks the requirement forward by examining the output of the deliverables and backward by looking at the business requirement that was specified for a particular area of the software
3.6 Test Report
Testing Team generally sends out various reports at the conclusion of each test activity to aware the stake holders or customers about the update on each phase. These test reports are designed to document the results of testing as defined in the test plan.
The test report has one immediate and two future purposes. The immediate purpose is to provide information to the customers about the stability of the recent build so that they can redesign or make some immediate changes based on the defects if required. Regarding the long term purposes the first purpose is to plan corrective actions before the application deployed at production based on the test reports. The second purpose is to use the data to analyze the rework process for the future changes based on the defects reported during different test reports. The defect prone component of the application can be also identified early and preventive measures can be taken to minimize the defects in future releases.
Some of the reports are discussed below:
3.6.1 Daily Test Execution Report
When different testers are testing different module of the application, this report is prepared on daily basis to provide the information about test progress to the higher management and also to the customer. Generally this kind of report contains the following items:
· Key Items of Test
· Percentage of Test completion
· Test Environment Details
· Number of Test Cases Planned for Execution Vs. Number of actual Test Cases executed per day
· Overall Test Execution Result
· Defect Details
· Risks or Issues ( Environment downtime or any show stopper)
For example following is a metric and graph that we can use to show planned vs. actual test execution within daily report:
3.6.2 Integration & System Test Report
Through Integration Testing the interfaces between individual projects are verified. Generally during test planning the interfaces and related test conditions are identified. This test report helps the customer to identify the defect prone application interface and take the corrective measures in advance
A system test plan actually mentions about the objectives of testing, test scope, how was it to be tested and test schedule. The system test report should depict the result of executing the test plan.