Software Quality

The degree to which a system, component, or process meets specified requirements. The degree to which a system, component or process meets customer or user needs or expectations.

Compiled by Rohitash cse final year

Quality Defined (continued)

Two kinds of quality are sought out
Quality of design
The characteristic that designers specify for an item This encompasses requirements, specifications, and the design of the system

Quality of conformance (i.e., implementation)

The degree to which the design specifications are followed during manufacturing This focuses on how well the implementation follows the design and how well the resulting system meets its requirements

Quality also can be looked at in terms of user satisfaction

User satisfaction = compliant product + good quality + delivery within budget and schedule
2

Software Quality Attributes

http://satc.gsfc.nasa.gov/support/STC_APR96/qualtiy/stc_qual.html

8/12/2012

Compiled by Rohitash Cse final year

The Cost of Quality

Includes all costs incurred in the pursuit of quality or in performing quality-related activities Is studied to
Provide a baseline for the current cost of quality. Identify opportunities for reducing the cost of quality. Provide a normalized basis of comparison .

Involves various kinds of quality costs (See next slide) Increases dramatically as the activities progress from
Prevention Detection Internal failure External failure

"It takes less time to do a thing right than to explain why you did it wrong."
4

Kinds of Quality Costs

Prevention costs
Quality planning, formal technical reviews, test equipment, training

Appraisal costs
Inspections, equipment calibration and maintenance, testing

Failure costs subdivided into internal failure costs and external failure costs
Internal failure costs
Incurred when an error is detected in a product prior to shipment Include rework, repair, and failure mode analysis

External failure costs

Involves defects found after the product has been shipped Include complaint resolution, product return and replacement, help line support, and warranty work

Software Quality Costs

There is a price to pay for quality. (Quality is not Free) Types of software quality costs Cost of Quality Approach:

Prevention: Avoiding defects, planning, preparation, training, reviews, tool costs. Quality Assurance prevents the injection of various types of defects. Appraisal: Finding defects by inspection, audit, developing test cases, test and measurement. Internal Failure: Costs emerging during development. Rework, redesign, modifications, corrective action, down time, concessions and overtime. External Failure: Costs emerging from a failure at the customer: Equipment failure, loss of sales, downtime, administrative cost in dealing with failure, law costs, loss of reputation.

The classic model of cost of software quality

Feigenbaums Model Costs of Control costs Cost of software quality Costs of Failure of control costs Internal failure costs Prevention costs Appraisal costs

External failure costs

7

Software Reviews

Software reviews
4.6.1 Purpose 4.6.2 Minimum requirements
4.6.2.1 Software specifications review (SSR) 4.6.2.2 Architecture design review (ADR) 4.6.2.3 Detailed design review (DDR) 4.6.2.4 Verification and validation plan review 4.6.2.5 Functional audit 4.6.2.6 Physical audit 4.6.2.7 In-process audits
a) Code versus design documentation b) Interface specifications (hardware and software) c) Design implementations versus functional requirements d) Functional requirements versus test descriptions

4.6.2.8 Managerial reviews 4.6.2.9 Software configuration management plan review (SCMPR) 4.6.2.10 Post-implementation review 4.6.3 Other reviews and audits
9 Compiled by rohitash Cse final year

Purpose of Reviews
Serve as a filter for the software process Are applied at various points during the software process Uncover errors that can then be removed Purify the software analysis, design, coding, and testing activities Catch large classes of errors that escape the originator more than other practitioners Include the formal technical review (also called a walkthrough or inspection)
Acts as the most effective SQA filter Conducted by software engineers for software engineers Effectively uncovers errors and improves software quality Has been shown to be up to 75% effective in uncovering design flaws (which constitute 50-65% of all errors in software)

Require the software engineers to expend time and effort, and the organization to cover the costs
10

Defects and failures

Not all software defects are caused by coding errors. Expensive reason may be requirement gaps non-functional requirements testability scalability.
11

Contd
Maintainability Usability performance security

12

defect-fault-failure
Software faults occur through the following processes:

EXECUTED IN CERTAIN SITUATION

13

 Not all defects will necessarily result in failures. E.g:- defects in dead code will never result in failures. A single defect may result in a wide range of failure symptoms.

14

Cost of faults
The cost of software faults that are not detected before a system is put into live operation varies, depending on a number of factors. Example- the European Space Agencys Ariane5 rocket exploded.

15

16

Classification of defects
Severity Wise: Major: A defect, which will cause an observable product failure or departure from requirements. Minor: A defect that will not cause a failure in execution of the product. Fatal: A defect that will cause the system to crash or close abruptly or effect other 17 applications.

Work product wise

SSD: A defect from System Study document FSD: A defect from Functional Specification document ADS: A defect from Architectural Design Document DDS: A defect from Detailed Design document
18

 Source code: A defect from Source code Test Plan/ Test Cases: A defect from Test Plan/ Test Cases User Documentation: A defect from User manuals, Operating manuals

19

Facts about defect rate

In a study conducted at Hewlett Packard, Lim [LIM94] reports that the defect rate for reused code is 0.9 defects per KLOC, while the rate for newly developed software is 4.1 defects per KLOC. For an application that was composed of 68 percent reused code, the defect rate was 2.0 defects per KLOC
20

Software Reliability
Defined as the probability of failure free operation of a computer program in a specified environment for a specified time. It can measured, directed and estimated A measure of software reliability is mean time between failures where MTBF = MTTF + MTTR MTTF = mean time to failure MTTR = mean time to repair
20

Measurement scales
Nominal Scale This is the most primitive form of measurement. A scale is a nominal one if it divides the set of entities into categories, with no particular ordering among them [10]. E.g. IEEE 802.1, 802.2, 802.3802.11. Ordinal Scale A scale is an ordinal one if it divides the set of entities into categories that are ordered according to some order . There is no quantitative comparison. E.g. programmer skill (low, medium, high).
22

Measurement scales
Interval Scale This scale captures information about the size of the intervals that separate classes. In an interval scale, the exact difference between two values is the basis for meaningful statements. E.g. programmer capability between: 60th and 80th percentile of population
23

Measurement scales
Ratio Scale
A measurement mapping that preserves ordering, the size of intervals between entities, and ratios between entities In a ratio scale, contrary to interval scale, there is an absolute and non arbitrary zero point . E.g. project A took twice as long as project B Absolute Scale Absolute scale is the most informative in the measurement scale hierarchy. In absolute scale, measurement is done by counting method. E.g. number of failures observed during integration testing can be measured only by counting the number of failures observed.

24

Inspection process
Inspection refers to peer review of any work product by trained individuals who look for defects using a well defined process. The goal of the inspection is for all of the inspectors to reach consensus on a work product and approve it for use in the project.
25

Inspection process stages

Planning: The inspection is planned by the moderator. Overview meeting: The author describes the background of the work product. Preparation: Each inspector examines the work product to identify possible defects. Inspection meeting: During this meeting the reader reads through the work product, part by part and the inspectors point out the 26 defects for every part.

Inspection process stages

Rework: The author makes changes to the work product according to the action plans from the inspection meeting. Follow-up: The changes by the author are checked to make sure everything is correct.

27

What are Metrics?

Software process and project metrics are quantitative measures They offer insight into the effectiveness of the software process and the projects that are conducted using the process as a framework Basic quality and productivity data are collected These data are analyzed, compared against past averages, and assessed The goal is to determine whether quality and productivity improvements have occurred Remedies can then be developed and the software process can be improved
28

Product Metrics
Help software engineers to better understand the attributes of models and assess the quality of the software Help software engineers to gain insight into the design and construction of the software Focus on specific attributes of software engineering work products resulting from analysis, design, coding, and testing Provide a systematic way to assess quality based on a set of clearly defined rules Provide an on-the-spot rather than after-the-fact insight into the software development
29

 Classes of product metric

Dynamic metrics which are collected by measurements made of a program in execution; Static metrics which are collected by measurements made of the system representations; Dynamic metrics help assess efficiency (response time of a system) and reliability (no of failures); static metrics help assess complexity, understandability and maintainability.

30

Examples of Product Metrics

Number and type of defects found during requirements, design, code, and test inspections Number of pages of documentation delivered Number of new source lines of code created Number of source lines of code delivered Total number or source lines of code delivered Average complexity of all modules delivered

31

 Average size of modules Total number of modules Total number of bugs found as a result of unit testing Total number of bugs found as a result of integration testing Total number of bugs found as a result of validation testing Productivity, as measured by KLOC per personhour
32

Process metrics
A metric used to measure characteristics of the methods, techniques, and tools employed in developing, implementing, and maintaining a software system Private process metrics
are known only to the individual or team concerned.

Public process metrics

enable organizations to make strategic changes to improve the software process.

33

Examples of Process Metrics

Average find-fix cycle time Number of person-hours per inspection Number of person-hours per KLOC Average number of defects found per inspection Number of defects found during inspections in each defect category Average amount of rework time Percentage of modules that were inspected
34

35