Tools Journal - Metrics...

A Quick Re-Cap

It is said that you cant control what you cant measure. Metrics have become an integral part of a software development process and the challenge that had been and is today as well despite software metrics defined and prevailing from decades is to implement them, determine which metrics are required to be measured for a project and specifically what they mean and how to measure them. This article collates and rewinds key metrics that matters the most at one place throughout project lifecycle.

What is a METRIC?

By definition asoftware metric is a measure of some property of a piece ofsoftwareor its specifications. This article will look into few key metrics related to project management, testing and defects.

1/6

Tools Journal - Metrics... A Quick Re-Cap

Project Management Metrics

Effort variance A simple metric which describes how much have you deviated from your estimated effort for a particular piece of work. It can be calculated as follows: Effort Variance = {(Actual Efforts-Estimated Efforts) / Estimated Efforts} *100. Schedule Variance Schedule variance describes how much have you deviated from estimated number of days to complete your work. Just like above formula it is similarly calculated as. Schedule Variance = {(Actual Duration - Estimated Duration)/Estimated Duration} *100 Schedule Slippage When a task has been delayed from its original baseline schedule then the amount of time that it has taken is defined as schedule slippage. Its calculation is as simple as. Schedule Slippage = (Actual End date - Estimated End date) / (Planned End Date Planned Start Date) * 100 Cost Variance Cost Variance describes how much have you deviated from your estimated cost to complete a piece of work. It is calculated as follows Cost Variance = {(Actual Cost - Estimated Cost)/Estimated Cost} *100 Cost Of Quality Costs Of Quality is a measure of total of the cost incurred by: - Investing in the prevention of nonconformance to requirements. - Appraising a product or service for conformance to requirements. - Failing to meet requirements.

Cost of quality comprises of four elements:

External Failure Cost cost associated with defects found after the customer receives the product or serviceex: processing customer complaints, customer returns, warranty claims, product recalls.

2/6

Tools Journal - Metrics... A Quick Re-Cap

Internal Failure Cost cost associated with defects found before the customer receives the product or serviceex: scrap, rework, re-inspection, re-testing, material review, material downgrades. Appraisal (Inspection) Cost cost incurred to determine the degree of conformance to quality requirements (measuring, evaluating or auditing)ex: inspection, testing, process or service audits, calibration of measuring and test equipment. Prevention Cost cost incurred to prevent (keep failure and appraisal cost to a minimum) poor qualityex: new product review, quality planning, supplier surveys, process reviews, quality improvement teams, education and training.

Cost Of Quality = Prevention Cost + Failure Cost + Appraisal Cost

Defect Metrics

Defect Density The Number of Known/confirmed Defects is the count of total defects identified against a particular software entity, during a particular time period. For Ex: - defect to date since the creation of module - defects found in a program during an inspection - defects to date since the shipment of a release to the customer

Size is a normalizer that allows comparisons between different software entities (i.e., modules, releases, products). Size is typically counted either in Lines of Code or Function Points.

Defect Density = Total Number of Confirmed Defects /Size of project

3/6

Tools Journal - Metrics... A Quick Re-Cap

Weighted Defect Density Weighted defect density is an accurate representation of defect severity wise defect density within a project. Weighted Defect Density = (5*Count of fatal defects)+(3*Count of Major defects)+(1*Count of minor defects) Where are the values 5,3,1 corresponds to the severities of the detect Defect Age Defect Age can be calculated either in time or project phases. Its a measure of how long a defect has been left in the project lifecycle since its detection. In other words it also represents quality of project phases. Defect Age (in Time) is the difference in time between the date a defect is detected and the current date (if the defect is still open) or the date the defect was fixed (if the defect is already fixed). Defect Age = Defect Fix Date (Or Current date) Defect Detection Date Defect Age (in Phases) is the difference in phases between the defect injection phase and the defect detection phase. Defect Age = Defect Fix Phase (Or Current Phase) Defect Detection Phase Defect Removal Efficiency (DRE) DRE is the measure of the defect removal ability of quality assurance and control activities as they are applied throughout all project phases. The computation of DRE can be done as follows DRE = (Defect Found by Internal Team/Total no of defects found) * 100

Testing Metrics

Test Case Adequacy This is a measure of adequacy of test cases generated within a project for testing the software. This defines the number of actual test cases created to effectively cover the test scope vs estimated test cases at the end of test case preparation phase. It is calculated as Test Case Adequacy = No. of actual test cases created / No. of test cases estimated

4/6

Tools Journal - Metrics... A Quick Re-Cap

Test Case Effectiveness This metric is used to measure the effectiveness of test cases generated in detecting defects. It is calculated as Test Case Effectiveness = No. of defects detected using test cases*100/Total no. of defects detected Test Effectiveness Test effectiveness is a relative ability of your testing process or strategy to find bugs. There are many methods to calculate this but the one we follow usually is Test effectivenss = Total number of defects reported in testing/( Total number of defects reported in testing + Defects reported in acceptance testing) Cost of finding a defect in testing This can be used as indicator for measuring testing phase defect finding cost spanned over a period of testing effort. Cost of finding a defect = Total effort spent on testing / defects found in testing P.S. Total time spent on testing must include time to create, review, rework, execute the test cases and record the defects. This should not include time spent in fixing the defects. Rework Effort Ratio This is a ratio of rework actual undertaken with respect to total estimated effort for software testing. Rework Effort Ratio = {(Actual rework efforts spent in testing phase / Total estimated efforts spent for testing phase)} * 100 Requirement Stability Index It is the ratio of new approved changes incorporated to software vs. total number of scope planned initially before project start. This can be represented as: Requirement Stability Index = {1 - (Total No. of changes /No of initial requirements)} Requirements Creep It is the ratio of changes to initial requirements initially scoped for project. Requirements Creep = (Total No. of requirements added / No of initial requirements) * 100

Conclusion

5/6

Tools Journal - Metrics... A Quick Re-Cap

Despite the metrics defined decades ago and organizations focussing to capture every possible metrics, in my experience we have not been successful in adhering to valuable metrics, gathering the information from metrics and ensure we optimize our software engineering processes accordingly. This article is just an attempt to collate basic definitions of those metrics at one place. Feel free to contact me on sudheer@itwings.co.uk or info@toolsjournal.com if you need more information. Please do provide your valuable feedback and comments so i can keep this article updated at all times with our experiences. Author: Sudheer Raju, IT Project Manager,sudheer@itwings.co.uk

6/6