Documente Academic
Documente Profesional
Documente Cultură
CS577b 3/20/00
Outline
Definitions n Defect analysis review n Sample causal analysis exercises n Defect prevention KPA
n
CS577b 3/20/00
Definitions
Causal analysis: the analysis of defects to determine their underlying root cause n Causal analysis meeting: a meeting, conducted after completing a specific task, to analyze defects uncovered during the performance of that task
n
CS577b 3/20/00
Defect Analysis
Defect: any flaw in the specification, design, or implementation of a product. n Facilitate process improvement through defect analysis
n
defect categorization to identify where work must be done and to predict future defects causal analysis to prevent problems from recurring
CS577b 3/20/00
4
Fault Distributions
Requirements Design Coding
50% 40% 10%
Functional Test
System Test
Field Use
Fault Origin
3%
5%
7% 25%
50% 10%
Fault Detection
CS577b 3/20/00
Requirements
Design
Coding
Functional Test
System Test
Field Use
Phase
Fault Introduction Distribution
10%
5% 3%
4 3 2 1
30% 20%
30%
0% 0% 0% 1
2% 0% 0% 1
8% 17%
3% 2% 1
33% 20
Relative Fault Cost
CS577b 3/20/00
A defect introduction and removal matrix can be generated and used for defect prevention to help answer what are high-leverage opportunities for defect prevention / cost containment?.
CS577b 3/20/00
A defect introduction and removal matrix can be generated and used as a basis for defect analysis and prevention.
Percentage of Defects Phase detected Requirements Preliminary design Detailed design Code/unit test Integration testing System testing Total Phase injected Requirements Preliminary Detailed Code/unit Total design design test 37% 8% 22% 38% 16% 15% 18% 34% 17% 7% 24% 28% 43% 25% 7% 9% 14% 29% 14% 11% 12% 24% 29% 19% 100% 100% 100% 100% 100%
CS577b 3/20/00
Causal Analysis
Data on defects is collected and categorized n Trace each defect to its underlying cause n Isolate the vital few causes
n
Pareto principle: 80% of defects are traceable to 20% of all possible causes
n
CS577b 3/20/00
Post-inspection example:
Moderator, Date Subject, Subject type Item number Assigned to Defect category (interface, requirements, design, code, other) Item description Probable cause Suggestions for eliminating probable cause Action taken Number of hours to take corrective action
CS577b 3/20/00
10
Frequency
1000 100 200 300 400 500 600 700 800 900 0
Correctness
Clarity
CS577b 3/20/00
Completeness Consistency Compliance Maintainability
Defect Category
Functionality Interface Performance Testability Reusability Traceability
11
CS577b 3/20/00
13
Exercise #1 Answer
n n
Determine the defect types and components that contribute the most rework: TYPE user interface 84 hours logic 63 hours hardware interface 53 hours communication 13 hours
> concentrate on the user interface (resolve risk early, allocate resources, user prototyping, inspections, etc.)
COMPONENT
C87 hours B 66 hours A 65 hours
> concentrate on component C
CS577b 3/20/00
14
CS577b 3/20/00
Exercise #2 Answers
Defect Origin # of defects Code 8 Design 4 Specification 3 Documentation 2 Other 2 Environment Support 1 Code Defect Type # of Defects Weight Logic 5 2.5 Computation 3 2.5 Total wt. 12.5 7.5
Design Defect Type # of Defects Weight Total wt. S/W Interface 1 6.2 6.2 Process Comm. 1 6.2 6.2 User Interface 1 6.2 6.2 H/W Interface 1 6.2 6.2 weight 14 6.2 2.5 1 1 1 total weight 42 24.8 20 2 2 1
16
Defect Origin # of defects Specification 3 Design 4 Code 8 Documentation 2 Other 2 Environment Support 1
CS577b 3/20/00
Data analysis from Level 4 activities enables focusing the performance of Defect Prevention (DP), Technology Change Management (TCM), and Process Change Management (PCM)
CS577b 3/20/00
17
Defect Prevention
The purpose of Defect Prevention is to identify the cause of defects and prevent them from recurring. Defect Prevention involves analyzing defects that were encountered in the past and taking specific actions to prevent the occurrence of those types of defects in the future. The defects may have been identified on other projects as well as in earlier stages or tasks of the current project. Defect prevention activities are also one mechanism for spreading lessons learned between projects. Trends are analyzed to track the types of defects that have been encountered and to identify defects that are likely to recur. Based on an understanding of the project's defined software process and how it is implemented (as described in the Integrated Software Management and Software Product Engineering key process areas), the root causes of the defects and the implications of the defects for future activities are determined. Both the project and the organization take specific actions to prevent recurrence of the defects.
CS577b 3/20/00
18
CS577b 3/20/00
19
CS577b 3/20/00
20
Tools:
statistical analysis tools database systems other
Examples of DP training:
defect prevention methods conduct of task kick-off meetings conduct of causal analysis meetings, and statistical methods (e.g., cause/effect diagrams and Pareto analysis).
21
CS577b 3/20/00
DP Project Activities
n
CS577b 3/20/00
22
CS577b 3/20/00
23
CS577b 3/20/00
24
DP Team Activities
n
Each of the teams assigned to coordinate defect prevention activities meets on a periodic basis to review and coordinate implementation of action proposals from the causal analysis meetings. The teams involved may be at the organization or project level. Team activities include: 1.Review the output from the causal analysis meetings and select action proposals that will be addressed. 2.Review action proposals that have been assigned to them by other teams coordinating defect prevention activities in the organization and select action proposals that will be addressed. 3.Review actions taken by the other teams in the organization to assess whether these actions can be applied to their activities and processes. 4.Perform a preliminary analysis of the action proposals and set their priorities. Priority is usually nonrigorous and is based on an understanding of: the causes of defects, the implications of not addressing the defects, the cost to implement process improvements to prevent the defects, and the expected impact on software quality. An example of a technique used to set priorities for the action proposals is Pareto analysis. 25
CS577b 3/20/00
CS577b 3/20/00
26
CS577b 3/20/00
27
CS577b 3/20/00
28
DP Feedback
n
Feedback is needed on the status and results of the organization's and project's defect prevention activities on a periodic basis.
The feedback provides: 1.A summary of the major defect categories. 2.The frequency distribution of defects in the major defect categories. 3.Significant innovations and actions taken to address the major defect categories. 4.A summary status of the action proposals and action items. Examples of means to provide this feedback include: electronic bulletin boards, newsletters, and information flow meetings.
CS577b 3/20/00
29
DP Measurements
n
Examples: the cumulative costs of defect prevention activities (e.g., holding causal analysis meetings and implementing action items) the time and cost for identifying the defects and correcting them, compared to the estimated cost of not correcting the defects profiles measuring the number of action items proposed, open, and completed the number of defects injected in each stage, cumulatively, and over releases of similar products and the total number of defects.
CS577b 3/20/00
30
DP Management Reviews
n
DP reviews cover: 1.A summary of the major defect categories and the frequency distribution of defects in these categories. 2.A summary of the major action categories and the frequency distribution of actions in these categories. 3.Significant actions taken to address the major defect categories. 4.A summary status of the proposed, open, and completed action items. 5.A summary of the effectiveness of and savings attributable to the defect prevention activities. 6.The actual cost of completed defect prevention activities and the projected cost of planned defect prevention activities.
CS577b 3/20/00
31
References
n
Inderpal Bhandari, Michael Halliday, et al., "A Case Study of Software Process Improvement During Development," IEEE Transactions on Software Engineering, Vol. 19, No. 12, December 1993, pp. 11571170. R. Chillarege and I. Bhandari, "Orthogonal Defect Classification -- A Concept for In-Process Measurements," IEEE Software, Vol. 18, No. 11, November 1992, pp. 943-955. Julia L. Gale, Jesus R. Tirso, and C. Art Burchfield, "Implementing the Defect Prevention Process in the MVS Interactive Programming Organization," IBM Systems Journal, Vol. 29, No. 1, 1990, pp. 33-43. C.L. Jones, "A Process-Integrated Approach to Defect Prevention," IBM Systems Journal, Vol. 24, No. 2, 1985, pp. 150-167. Juichirou Kajihara, Goro Amamiya, and Tetsuo Saya, "Learning from Bugs," IEEE Software, Vol. 10, No. 5, September 1993, pp. 46-54. R.G. Mays, C.L. Jones, G.J. Holloway, and D.P. Studinski, "Experiences with Defect Prevention," IBM Systems Journal, Vol. 29, No. 1, 1990, pp. 4-32. Norman Bridge and Corinne Miller, "Orthogonal Defect Classification Using Defect Data to Improve Software Development," Proceedings of the 7th International Conference on Software Quality, Montgomery, Alabama, 6-8 October 1997, pp. 197-213.
CS577b 3/20/00
32