Test management is an essential part for Software Testing Life Cycle. To a great degree we can conclude that test management depends on proper estimation and judgment. Many external and internal factors impact the project as well as the project timeline and this is where the test estimation and risk analysis meet together. Estimation involves risk at all level and mainly the following factors influence test estimation:
- Available resources
- Test Data
- Fund available
By definition Estimation means something that can be changed and it is the basis on which higher management can make decisions. This is why the project managers continuously monitor the progress of the project and change the estimate accordingly. They also need to send the updates to the respective stake holders. A number of methods and metrics have been proposed and applied to estimating the size and effort of software projects. Many of these methods are also used to estimate the effort of software testing activities. Some of the estimation methods are discussed below:
5.1 Function Point Analysis
Function point analysis is one of the most commonly used estimation technique in the industry. It is an ISO recognized method to measure the functional size of an information system. Function Point metric can be used to estimate the cost or effort required to design, code and test the software or process. This analysis measures the software size based on five components including external inputs (EIs), external outputs (EOs), external inquiries (EQs), internal logical files (ILFs), and external interface files (EIFs). The size output, whose unit is Function Point, is derived using an empirical relationship based on countable measures of software’s information domain and qualitative assessments of software complexity. The values of information domain are defined on the five factors listed above. Apart from that organizations had some weighting factor (simple, average, complex) for each of the entries in the information domain. Now to compute the function points (FP), the following relationship is used:
FP=count total * [0.65+0.01*∑(F i)]
Where count total is the sum of all FP entries and Fi (i=1 to 14) is the value adjustment factor (VAF) determined based on some responses to the organization specific questions or factors on the developed process.
Now consider an example of a small function in a software system which takes 2 external inputs along with 2 external inquiries and 1 internal logical file. The function itself has 2 external outputs and connected with 3 external interfaces. Here the total count is determined as per the table shown below:
Information Domain Value
External Inputs (EIs)
External Outputs (EOs)
External Inquiries (EQs)
Internal Logical Files (ILFs)
External Interface Files (EIFs)
Here all the entries for the information domain are considered Simple for this example and assume ∑ Fi = 50 (a moderately complex system)
Therefore estimated number of FP = 38 * [ 0.65 + (0.01 * 50 )] = 44
Now for an organization consider average productivity for softwares for this type is 4 FP / Per month and assuming the approximate labour rate of $ 500 per month , the cost of developing this module becomes $ 5500 and estimated effort is 11 person months. Now by adding resources we can reduce the total time if required.
5.2 Test Case Point Analysis
The Test Case Point is a measure of estimating the software testing size and effort. The Test Case Point Analysis uses test cases as input and generates Test Case Point count for the test cases being used as input. The TCP counts are nothing but ranking the requirements and the impacted test cases for those requirements into simple, average and complex and quantifying it into a complexity measure.
Any application can be divided into several modules and any module can be classified as Simple, Average and Complex based on the number and complexity of the associated requirements for that module. Now the test cases associated for a particular requirement can also be classified into Simple, Average and Complex based on the following factors:
- Complexity of the test case
- Number of verification points or check points
- Interface with other test cases
- Precondition and test data
For simplicity a test case can be considered as complex if it contains a calculation in any of it check point or any verification point which interfaces with or interacts with another application. But depending on the respective project, the complexity calculation may be different.
Based on the type of the test case an adjustment factor is allocated for simple , average , and complex test cases. This adjustment factor has been derived after a thorough study and analyzing the historical data of many testing projects and this factor is organization specific. The weighing factor should be highest for the Complex and lowest for the Simple test case. From this portion and break up we can easily identify the number of simple, average and complex requirements associated with the test cases within the project. Now the Test Case Point for each type can be calculated as following:
Simple Test Case Point = Number of simple requirements in the project * Adjustment factor for Simple requirements ( A1 )
Average Test Case Points = Number of average requirements in the project * Adjustment factor for average requirements ( A2 )
Complex Test Case Points = Number of complex requirements in the project * Adjustment factor for complex requirements ( A3 )
Hence the total test case point for the system under test = A1 + A2 + A3
From estimation perspective if the productivity index is measured as the number of Test Case points completed per person month then the total effort required can be easily calculated from this productivity index and total test case point.
Now testing activities can be categorized into four parts – test planning, test designing, test execution and defect reporting. From these four, test execution and defect reporting can be performed multiple times for a single test case within a project . However the effort or size estimated into Test Case Point takes into account all of these activities assuming that each activity is performed once. So to make the estimation more perfect a distribution of effort during each of these phases is required.