Sunteți pe pagina 1din 44

Test vs.

inspection Part 1
Tor Stlhane

What we will cover


Part 1
Introduction Inspection processes Testing processes

Part 2
Tests and inspections some data Inspection as a social process two experiments and some conclusions

Introduction

Adams data - 1
Mean Time to Problem Occurrence years
Product 1 2 1.6 0.7 0.7 5 1.2 1.5 16 2.1 3.2 50 5.0 4.3 160 10.3 9.7 500 17.8 18.2 1600 28.8 28.0 5000 34.2 34.3

3
4 5 6 7 8 9

0.4
0.1 0.7 0.3 0.6 1.1 0.0

1.4
0.3 1.4 0.8 1.4 1.4 0.5

2.8
2.0 2.9 2.1 2.7 2.7 1.9

6.5
4.4 4.4 5.0 4.5 6.5 5.6

8.7
11.9 9.4 11.5 9.9 11.1 12.8

18.0
18.7 18.4 20.1 18.5 18.4 20.4

28.5
28.5 28.5 28.2 28.5 27.1 27.6

33.7
34.2 34.2 32.0 34.0 31.9 31.2

Adams data 2
The main information that you get from the table on the previous slide is that Some defects are important because they will happen quite often. Most defects are not important since they will happen seldom. How can we tell the difference?

Testing alone is not the solution


As can be seen from the next slide, testing is not an acceptable solution alone. It will Take too long time Cost too much We can generate tests automatically, but would never the less have to use large resources to check the result the oracle problem

A limit result
The following relation holds under a rather wide set of conditions:

e MTTF t t N0
The initial number of defects N0 must be estimated e.g. based on experience from earlier projects as number of defects per KLOC.

An example from telecom

Testing and inspection the V model

Testing and inspection 1


The important message here is that testing cannot always be done. In the first, important phases, we have nothing to execute and will thus always have to do some type of inspection. This might be considered one of the weaknesses of traditional software engineering over Agile development.

Testing and inspection 2


In order to understand the main differences between testing and inspection, we should consider Fits list. Based on this, we will give a short discussion of the relative merits of testing and inspection.

Area of competence
Understanding Observe

Man
Good at handling variations in written material General observations, multifunctional

Machine
Bad at handling variations in written material Specialized, good at observing quantitative data, bad at pattern recognition

Reasoning
Memory

Inductive, slow, imprecise but good at error correction


Innovative, several access mechanisms

Deductive, fast, precise but bad error correction


Copying, formal access

Information handling
Consistency Power Speed

Single channel, less than 10 bits per second


Unreliable, get tired, depends on learning Low level, maximum ca. 150 watt Slow seconds

Multi channel, several Megabits per second


Consistent repetition of several actions High level over long periods of time Fast

Man vs. machine 1


Good when we need the ability to Handle variation Be innovative and inductive Recognize and handle patterns Not so good when we need the ability to Do the same things over and over again in a consistent manner Handle large amount of data

Man vs. machine 2


In order to do the best job possible we need processes where we let each part Do what they are best at:
Man is innovative Machine handles large amounts of data

Support the other with their specialties.


Machine supports man by making large amounts of information available Man support machine by providing it with innovative input

General considerations - documents


Architecture, system, sub-system and component design plus pseudo code. Here we can only use inspections. Man will use experience and knowledge to identify possible problems Machine can support by identifying information e.g. find all occurrences of a string.

General considerations code (1)


For executable code, we can use inspection, testing or a combination of both. The size and complexity degree of dynamism of the code will, to a large degree, decide our choice. Other important factors are the degree of experience with The programming language The algorithms used

General considerations code (2)


Simple code Start with inspection all code Design and run tests Complex code Start with inspection focus on algorithm and logic Decide test completeness criteria we cannot test everything Design and run tests

Inspection processes

Inspections 1
The term inspection is often used in a rather imprecise manner. We will look at three types of inspection: Walkthrough Informal inspection also called informal review Formal inspection also called formal review or just inspection The first two types are usually project internal while the last one is used as a final acceptance activity for a document.

Inspections 2
For all types of inspections: The quality of the results depends on the experience and knowledge of the participants. Garbage in Garbage out It might be a good idea to involve customer representatives.

The walkthrough process


Walkthrough is a simple process mostly used for early decisions for an activity. The document owner: 1. Makes a rough sketch of the solution architecture, algorithm etc. 2. Presents explain the sketch to whoever shows up. 3. Registers feedback improvements.

Walkthrough pros and cons


Pros: Easy and inexpensive. Needs no extra preparation. Collect ideas at an early stage of development. Cons: No commitment from the participants May collect many loose or irrelevant ideas

The informal inspection process


Planning

Rules, checklists, procedures

Product document

Individual checking

Logging meeting

Change requests

Informal inspections pros and cons


Pros: Is simple and inexpensive to perform. Can be used at all stages of development Usually has a good cost / benefit ratio Needs a minimum of planning Cons: No participant commitment No process improvement

The formal inspection process


The formal inspection process described below is with small variations the most commonly used. The version shown on the following slides stem from T. Gilb and D. Graham. We recommend this process as the final acceptance process for all important documents

Formal inspection process overview


Change requests Product document

Planning Process improvements Rules, checklists, procedures

Walkthrough

Kick-off

Individual checking

Logging meeting

Edit and followup

Distribution of resources
Typical Activity Range % value % Planning 35 4 Kick-off 47 6 Individual checking 20 30 25 Logging 20 30 25 Editing 15 30 20 Process brainstorming 15 30 16 35 4 Leader overhead, follow up, entry, exit

Initiating the inspection process


The inspection process starts with a request for inspection from the author to the QA responsible. The QA responsible appoints an inspection leader. First step is always to check that the document is fit for inspection.

Planning
Important planning points are: Who should participate in the inspections
Who is interested? Who have time available for preparation and meetings? Who has the necessary knowledge concerning application, language, tools, methods?

Kick-off
Important activities here are: Distribution of necessary documents:
Documents that shall be inspected Requirements Applicable standards and checklists

Assignment of roles and jobs Setting targets for resources, deadlines etc.

Individual checking
This is the main activity of the inspection. Each participant read the document to look for Potential errors - inconsistencies with requirements or common application experience Lack of adherence to company standards or good workmanship

Logging meeting
The logging meeting has three purposes: Log issues already discovered by inspection participants Discover new issues based on discussions and new information that arises during the logging meeting. Identify possible improvement to the inspection or development process.

Improve the product - 1


The author receives the log from the inspection meeting. All items - issues - in the log are categorised as one of the following: Errors in the authors document. Errors in someone elses document. Misunderstandings in the inspection team.

Improve the product - 2


Errors in own document: Make appropriate corrections Errors in someone elses documents: Inform the owner of this document. Misunderstandings in the inspection team: Improve document to avoid further misunderstandings.

Checking the changes


This is the responsibility of the inspection leader. He must assure that all issues raised in the log are disposed of in a satisfactory manner: The documents that have been inspected Related documents - including standards and checklists Suggested process improvements

Formal inspection pros and cons


Pros: Can be used to formally accept documents Includes process improvement Cons: Is time consuming and expensive Needs extensive planning in order to succeed

Testing processes

Testing
We will look at three types of testing: Unit testing does the code behave as intended. Usually done by the developer Function verification testing also called systems test. Does the component or system provide the required functionality? System verification testing also called acceptance test. Does the hardware and software work together to give the user the intended functionality?

The unit testing process


Unit testing is done by the developer one or more times during development. It is a rather informal process which mostly run as follows: 1. Implement (part of) a component. 2. Define one or more tests to activate the code 3. Check the results against expectations and current understanding of the component

Unit testing pros and cons


Pros: Simple way to check that the code works. Can be used together with coding in an iterative manner. Cons: Will only test the developers understanding of the spec. May need stubs or drivers in order to test

The system test process


A systems test has the following steps: 1. Based on the requirements, identify
Test for each requirement, including error handling Initial state, expected result and final state

2. Identify dependencies between tests 3. Identify acceptance criteria for test suite 4. Run tests and check results against
Acceptance criteria for each test Acceptance criteria for the test suite

Systems test pros and cons


Pros: Tests systems behavior against customer requirements. Cons: It is a black box test. If we find an error, the systems test must be followed by extensive debugging

The acceptance test process


The acceptance test usually has three activities all involving the customer or his representatives: Rerun the systems test at the customers site. Use the system to solve a set of real-world tasks. Try to break the system by stressing it or by feeding it large amounts of illegal input

Acceptance test pros and cons


Pros: Creates confidence that the system will be useful for the customer Shows the systems ability to operate in the customers environment Cons: Might force the system to handle input that it was not designed for, thus creating an unfavorable impression.

S-ar putea să vă placă și