OHT 7.

Chapter 7.2

Continuing: Factors affecting intensity of SQA activities Verification, validation and qualification Development and quality plans for small and for internal projects A model for SQA defect removal effectiveness and cost
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.2

Factors affecting Intensity of SQA Activities

SQA Activities are linked to the completion of a project phase Requirements, design, etc. The SQA activities need to be integrated into the development plan that implements one or more software development models, such as the waterfall, prototyping, spiral, They need to be activities just like other more traditional activities be entered in plan, scheduled, etc.
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.3

Factors affecting Intensity of SQA Activities

SQA planners need to determine
A list of SQA activities needed for the project And then for each activity, they need to decide on
Timing Type of QA activity to be applied (there are several) Who performs the activity and resources required.
Important to note that many participate in SQA activities Development team Department staff members Independent bodies

Resources required for the removal of defects and introduction of changes. Galin, SQA from theory to implementation

Pearson Education Limited 2004

OHT 7.4

Factors affecting Intensity of SQA Activities

Sad testimony that few want to allocate the necessary time for SQA activities.
This means time for SQA activities and then time for subsequent removal of defects. Often, there is no time for follow-on work!!

Activities are not simply cranked in and absorbed! So, time for SQA activities and defect correction actions needs to be examined.
Galin, SQA from theory to implementation Pearson Education Limited 2004

Factors Affecting the Required Intensity OHT 7.5 of QA Activity

Project Factors
Magnitude of the project how big is it? Technical complexity and difficulty

Discuss

Extent of reusable software components a real factor Severity of failure outcomes if project fails essential!

Team Factors
Professional qualifications of the team members Team acquaintance w/ project and experience in the area Availability of staff members who can professionally support the team, and Percentage of new staff members in the team.

Galin, SQA from theory to implementation

Pearson Education Limited 2004

OHT 7.6

So, what are the activities??

Recognize that these activities take days to undertake, and days to correct. Here are some sample activities: (and dont forget the planning for the reviews, materials, ) Design Review of Requirements Design Review of xxxxx Design Review of . Inspections of . Inspections of .. Unit Testing of software code for each interface module Unit testing of . System Testing of . Design Review of Users Manual..
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.7

Verification, Validation, and Qualification

Three aspects of Quality Assurance for Software Products:

Verification The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase Validation - The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements
Qualification - The process used to determine whether a system or component is suitable for operational use
IEEE Std 610.12-1990 (IEEE 1990)
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.8

More on VV and Q

Verification looks at the consistency of the products being developed with products developed in previous phases. Developers verify as we go. Validation is a customer thing customers validate the outputs, etc. Validate against their original requirements. Necessary for customer satisfaction. Qualification focuses on operational aspects maintenance is the main issue. Has development taken place IAW professional standards and procedures such that follow-on maintenance is easier to undertake? Planners need to determine which of all these need to be examined in each QA activity.
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.9

The model addresses two quantitative aspects of the SQA planning addressing several defect detection activities.
a. Want to study the SQA plans total effectiveness in removing project defects, and b. The total costs of removal of project defects

Note again that SQA activities must be integrated within the projects development plan.
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.10

Model for Defect Removal

The data:
Model based on three types of data: Defect origin distribution in which phase did the defects occur Defect removal effectiveness, and how effective are we at removal of defects?

Cost of defect removal. how much does it cost per defect per phase!!!
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.11

Defect Origin Distribution

Very consistent over many years: Distribution of Defects:
Requirements Specs Design Coding / integration Documentation 15% 35% 40% 10%

Galin, SQA from theory to implementation

Pearson Education Limited 2004

OHT 7.12

Defect Removal Effectiveness

Generally speaking, the percentage of removed defects is lower than the percentage of detected defects, because some corrections are ineffective or inadequate. We simply miss some!! Others are undetected and uncorrected and passed on to successive development phases.
Lingering defects coupled with introduced defects in current development phase add up!!! For discussion purposes, we will assume the filtering effectiveness of accumulated defects of each quality assurance activity is not less than 40%, that is, each activity removes at least 40% of the incoming defects. Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.13

Defect Removal Effectiveness

Removal effectiveness QA Activity

Average defect filtering effectiveness rate requirements specs review 50% design inspection 60% design review 50% code inspections 65% unit test 50% Unit test > code review 30% integration test 50% system tests / acceptance 50% documentation review 50%
Pearson Education Limited 2004

Galin, SQA from theory to implementation

OHT 7.14

Cost Removal

Removal of defects differs very significantly by development phase. Cost are MUCH greater in later development phases.

Note: In general, defect removal data is not commonly available. Most agree with the data based on key studies. (next slide)
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.15

Defect removal phase

Defect Average relative defect removal cost removal {cost unit} effectiveness Defect origination phase Req Des Uni Int Doc

Requirement specification (Req)

Design (Des)

50%
50%

Unit coding (Uni)

Integration (Int) System documentation (Doc) System testing / Acceptance testing (Sys)
Galin, SQA from theory to implementation

50%
50% 50% 50% 100%

1 2.5 6.5 16 16 40
110

--1

-----

-----

-----

2.6
6.4 6.4 16 44

1
2.5 2.5 6.2 17

--1

--1

2.5 6.9

1 2.5

Opertion by customer (after release)

Pearson Education Limited 2004

OHT 7.16

The Model

Our Model is based on following assumptions:

Development process is linear, sequential following waterfall model There is a number of new defects introduced each phase Review and test software quality assurance activities serve as filters, removing a percentage of defects and letting the rest pass to next development phase as we saw three slides back. At each phase, incoming defects are the sum of those not removed plus new defects in current phase Cost of defect removal is calculated for each SQA activity by multiplying the number of defects removed by the relative cost of removing a defect. (see table, previous slide) Remaining defects are passed to the customer. (this is the Galin, SQA from theory to implementation Pearson Education Limited 2004 heaviest cost for defect removal)

OHT 7.17

Model - parameters

The models we show use these parameters:

POD Phase originated defects PD Passed defects (from former phases) %FE - % of filtering effectiveness RD Removed defects CDR Cost of defect removal TRC Total Removal Costs

Galin, SQA from theory to implementation

Pearson Education Limited 2004

OHT 7.18

First Model (standard)

This model applies a standard quality assurance plan (standard defects filtering system) composed of QA activities, that is, filters we have:
QA activity removal effectiveness Cost of removing defect

Requirements Spec review Design Review Unit Test Integration tests Documentation Review System Test Operation Phase

50% 50% 50% 50% 50% 50% 100%

1 2.5 6.5 16 16 40 110

Pearson Education Limited 2004

Galin, SQA from theory to implementation

OHT 7.19

Using the standard quality assurance plans quality assurance activities on previous slide, a process-oriented illustration of the standard QA plan model follows:

Galin, SQA from theory to implementation

Pearson Education Limited 2004

OHT 7.20

POD phase originated defects PD passed defects %FE percent filtering RD removed defects CDR - Cost of removing defects TRC - Total Cost of Defect Removal

Galin, SQA from theory to implementation

Pearson Education Limited 2004

OHT 7.21

Costs of Defect Removal

But we can do better using a comprehensive quality assurance plan with more activities, and hence better filtering. The comprehensive quality assurance plan (comprehensive defects filtering system) accomplishes the following: 1. Adds two quality assurance activities so that the two are performed in the design phase as well as in the coding phase
We have a Design Inspection and a Design Review vice Design, and Code Inspections and unit test vice simple Unit Test

2. Improves the filtering effectiveness of other quality assurance activities.

Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.22 Design Inspect and Design Review

Code inspection precedes unit test Comprehensive Quality Assurance Plan:

POD phase originated defects PD passed defects %FE percent filtering RD removed defects CDR - Cost of removing defects TRC - Total Cost of Defect Removal

Galin, SQA from theory to implementation

Pearson Education Limited 2004

OHT 7.23

Comprehensive Plan

Conclusions:
The standard plan successfully removes 57.6% of the defects in requirements and design compared to 90.2% in the comprehensive plan before coding begins.
Results from more intensive defect-removal efforts.

The comprehensive plan as a whole is much more economical than the standard plan as it saves 41% of the total resources investing in defect removal, compared with the standard plan Compared to the standard plan, the comprehensive plan makes a greater contribution to customer satisfaction by drastically reducing the rate of defects detected during regular operations: 6.9% to 2.6%
Galin, SQA from theory to implementation Pearson Education Limited 2004

OHT 7.24

Conclusion
So, in general, the quantitative results of the comparison comply nicely with the SQA approach.
Additional investments in QA activities yield substantial savings in defect removal costs.

Galin, SQA from theory to implementation

Pearson Education Limited 2004