Software Testing

Slide 1

Lesson Objectives
Understand a range of different software testing techniques and strategies Be able to apply a specific (automated) unit testing method to your group project

Slide 2

Software Testing: Definition

The execution of software for purposes of validation and verification i? o o
X

Requires test data (inputs) and knowledge of the expected output. A test case is the tuple (i, o),where i is the test data and o=S(i) for S the software under test.
Slide 3

Test Cases Example

Spec of software to test: capitalise a given word. Set of 5 test data:
Spurs, Tottenham Hotspur, keaneR, spurs3, spurs Rule OK

Corresponding test case are the 5 tuples:

(Spurs, SPURS), (Tottenham Hotspur, error), (keaneR , KEANER), (spurs3, SPURS3), (spurs Rule OK, error)

Slide 4

Some really boring observations

It is impossible to completely test any nontrivial module or any system Testing can only show the presence of bugs, not their absence (Dijkstra)

Slide 5

Testing: the standard approach

1. Write a wad of code 2. Get scared enough about some aspect of it to feel the need to try it out 3. Write some kind of driver that invokes the bit of code. Add a few print statements to verify its doing what you think it should 4. Run the test, eyeball the output and then delete (or comment out) the print statements 5. Go back to step 1
Andy Hunt and Dave Thomas, IEEE Software 19(1), 2002

Slide 6

Good testing takes creativity

Testing is done best by independent testers Programmers often stick to the data set that makes the program work A program often does not work when tried by somebody else.

Slide 7

Testing in development
Component testing
Unit or module testing Integration testing

Function testing System testing

Slide 8

Testing after development

Acceptance testing Alpha test Beta test Installation testing During use

Slide 9

Testing Strategies
Bottom-up
Test Driver needed

Top-down
Test stub needed

Slide 10

Testing Methods
Dynamic testing vs Static testing (static analysis) Black Box Testing vs White Box Testing Statistical Testing

Slide 11

Regression testing
Retesting to ensure previously working code does not fail as a result of changes Should be applied irrespective of testing strategy or method

Slide 12

White-box testing
Determining test cases from a knowledge of the internal logic of the software

Slide 13

White-box Testing Example

FindMean(float Mean, FILE ScoreFile) { SumOfScores = 0.0; NumberOfScores = 0; Mean = 0; Read(ScoreFile, Score); /*Read in and sum the scores*/ while (! EOF(ScoreFile) { if ( Score > 0.0 ) { SumOfScores = SumOfScores + Score; NumberOfScores++; } Read(ScoreFile, Score); } /* Compute the mean and print the result */ if (NumberOfScores > 0 ) { Mean = SumOfScores/NumberOfScores; printf("The mean score is %f \n", Mean); } else printf("No scores found in file\n"); }
Slide 14

Determining the Paths

FindMean (FILE ScoreFile) { float SumOfScores = 0.0; int NumberOfScores = 0; 1 float Mean=0.0; float Score; Read(ScoreFile, Score); 2 while (! EOF(ScoreFile) { 3 if (Score > 0.0 ) { SumOfScores = SumOfScores + Score; NumberOfScores++; } 5

Read(ScoreFile, Score); 6 } /* Compute the mean and print the result */ 7 if (NumberOfScores > 0) { Mean = SumOfScores / NumberOfScores; printf( The mean score is %f\n, Mean); } else printf (No scores found in file\n); 9 }

Slide 15

Constructing the Flowgraph

Slide 16

Finding the Test Cases

one value)

Slide 17

Test Cases
Test case 1 : ? (To execute loop exactly once) Test case 2 : ? (To skip loop body) Test case 3: ?,? (to execute loop more than once)

Slide 18

Structural Coverage Criteria: Control Flow

Path testing Visit-each-loop path testing Simple path testing Basis (independent) path testing (McCabe) Branch testing Statement testing

Slide 19

Structural Testing: Examples

1 3 4 5 7 8 6 9 12 14 10 13 11
All paths (infinite number) <2 11>, <2 10 12 14>, <2 10 12 (13 12)n 14>, <1 3 5 6 9>, <1 4 6 9>, <1 3 5 6 (7 8)n 9>, <1 4 6 (7 8)n 9> (any n>0)

Visit-each-loop (7) <2 11>, <2 10 12 14>, <2 10 12 13 12 14>, <1 3 5 6 9>, <1 3 5 6 7 8 9>, <1 4 6 9>, <1 4 6 7 8 9> Simple paths (6) <2 11>, <2 10 12 14>, <1 3 5 6 9>, <1 3 5 6 7 8 9>, <1 4 6 9>, <1 4 6 7 8 9> Basis paths (6) <2 11>, <2 10 12 14>, <2 10 12 13 12 14>, <1 3 5 6 9>, <1 3 5 6 7 8 9>, <1 4 6 9>, <1 4 6 7 8 9> Branch (4) <2 11>, <2 10 12 13 12 14>, <1 3 5 6 9>, <1 4 6 7 8 9> Statement (2) <2 10 12 14>, <1 3 5 6 7 8 9>

Slide 20

Infeasible paths
A C F t t B f

A B C D E F G

Input(score); If score < 45 then print (fail) else print (`pass); If score > 80 then print (with distinction); End

E f G

Can you find an infeasible path here?

Slide 21

Condition coverage testing

a procedure smallprogram(A,B:integer;var X:real); begin if ((A>1) and (B=0)) then X:=X/A; if ((A=2) or (X>1)) then X:=X+1 end; false b A>1 and B=0 true c X=X/A

Condition testing can be done with 2 test cases (A=2, B=0, X=4), (A=1, B=1, X=1)

A=2 or X>1 false d

true e X=X+1

Multiple condition testing can be done with 4 test cases (A=2, B=0, X=4), (A=2, B=1, X=1), (A=1, B=0, X=2), (A=1, B=1, X=1)

Slide 22

Black-box testing
Equivalence partitioning Boundary value analysis Cause-effect graphing Error guessing Random testing Statistical testing Syntax directed testing State transition testing

Slide 23

Equivalence partitioning
0 < x < 1000 x<1 x>999

y=0

Input space

0< y < 6

y>5

Slide 24

Equivalence partitioning example: searching an array A for object e

array A of length n n even n=0 n>1 n odd n=1 n>2

eA

eA

eA

eA

eA eA

e first in A

e last in A

e not first or last

e first in A e last in A e not first or last

Slide 25

Boundary Value Analysis

Choose test cases directly on, above and beneath the edges of the input equivalence class and output equivalence classes Example:

For x valid if 1.0 x 1.0 test x = 1.0,1.0,1.001,1.001

Example: For allowable minimum and maximum number of items in the input/output, test min, min-1, max, and max+1

Slide 26

Statistical Usage Testing

Function Update (U) Delete (D) Query (Q) Print (P) Usage Frequency 32% 14% 46% 8% Distribution Interval 00-31 32-45 46-91 92-99

Test number 1 2 3 4

Random Numbers 29 11 47 52 26 94 62 98 39 78 82 65 83 32 58 41 36 17 36 49 96 82 20 77

Test Cases UUQQUP QPDQQQ QDQDDU DQPQUQ

Slide 27

We need both black-box and white-box testing

Program SQUARE Input x If x = 0 then output 0 else output x
input x x=0? t output 0 f output x destroy files t

Program ECHO Input ch If time = 23:59:59 and date = 31.12.2005 then destroy all files else output ch
input ch time=23:59:59 and date =31.12.2005? f output ch

Easy to do full path testing Without finding obvious fault

Extensive black box testing would not find the obvious fault here
Slide 28

JUnit
Fantastic automated tool support for unit testing in Java For each method you write you build the test case at the same time Fully integrated into Java IDEs Core part of Extreme Programming

Slide 29

JUnit test case

public class MoneyTest extends TestCase { // public void testSimpleAdd() { Money m12CHF= new Money(12, "CHF"); Money m14CHF= new Money(14, "CHF"); Money expected= new Money(26, "CHF"); Money result=m12CHF.add(m14CHF);
test's fixture

code exercises objects in fixture code verifies result

Assert.assertTrue(expected.equals(result)); } }

Slide 30

JUnit: inspecting the results

Slide 31

Lesson Summary
Testing aims to find faults Testing must be properly planned Effective testing necessarily involves a range of methods Testing is still a black art, but many rules and heuristics are available Unit testing is known to be one of the most cost-effective QA procedures. You can do unit testing with full automated support using JUnit.

Slide 32