Documente Academic
Documente Profesional
Documente Cultură
UNIT II
Software testing Fundamentals Test case Design Introduction of Black Box Testing and White Box testing Flow
Graphs and Path testing Path testing Basics - Predicates, Path Predicates and Achievable Paths - Path Sensitizing
Path Instrumentation Implementation and Application of Path Testing.
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
= =
F U N D E M E N T A L O F S O F T W A R E T E S T I N G
Testing is the process of evaluating a system or its component(s) with the intent to find that whether it satisfies the
specified requirements or not.
Testing is executing a system in order to identify any gaps, errors or missing requirements in contrary to the actual
desire or requirements.
W h a t i s t e s t i n g ?
Testing is the process of evaluating a system or its component(s) with the intent to find that whether it satisfies the
specified requirements or not. This activity results in the actual, expected and difference between their results. In
simple words testing is executing a system in order to identify any gaps, errors or missing requirements in contrary to
the actual desire or requirements.
According to ANSI/IEEE 1059 standard, Testing can be defined as A process of analyzing a software item to detect the
differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the
software item.
W h o d o e s t e s t i n g ?
It depends on the process and the associated stakeholders of the project(s). In the IT industry, large companies have a
team with responsibilities to evaluate the developed software in the context of the given requirements. Moreover,
developers also conduct testing which is called Unit Testing. In most cases, following professionals are involved in
testing of a system within their respective capacities:
Software Tester
Software Developer
Project Lead/Manager
End User
Different companies have difference designations for people who test the software on the basis of their experience
and knowledge such as Software Tester, Software Quality Assurance Engineer, and QA Analyst etc.
It is not possible to test the software at any time during its cycle. The next two sections state when testing should be
started and when to end it during the SDLC.
W h e n t o S t a r t T e s t i n g ?
An early start to testing reduces the cost, time to rework and error free software that is delivered to the client.
However in Software Development Life Cycle (SDLC) testing can be started from the Requirements Gathering phase
and lasts till the deployment of the software. However it also depends on the development model that is being used.
For example in Water fall model formal testing is conducted in the Testing phase, but in incremental model, testing is
performed at the end of every increment/iteration and at the end the whole application is tested.
Testing is done in different forms at every phase of SDLC like during Requirement gathering phase, the analysis and
verifications of requirements are also considered testing. Reviewing the design in the design phase with intent to
improve the design is also considered as testing. Testing performed by a developer on completion of the code is also
categorized as Unit type of testing.
Unlike when to start testing it is difficult to determine when to stop testing, as testing is a never ending process and no
one can say that any software is 100% tested. Following are the aspects which should be considered to stop the
testing:
Testing Deadlines.
Bug rate falls below a certain level and no high priority bugs are identified.
Management decision.
These two terms are very confusing for people, who use them interchangeably. Let's discuss about them briefly.
S . N . V e r i f i c a t i o n V a l i d a t i o n
1 Are you building it right? Are you building the right thing?
Ensure that the software system meets all the Ensure that functionalities meet the
2
functionality. intended behavior.
Have static activities as it includes the reviews, Have dynamic activities as it includes
5 walkthroughs, and inspections to verify that executing the software against the
software is correct or not. requirements.
It is an objective process and no subjective It is a subjective process and involves
6 decision should be needed to verify the subjective decisions on how well the
Software. Software works.
T e s t C a s e D e s i g n
Strategy
The strategy is to create one error at a time, while keeping all other components of the input string correct; that is, in
the absence of the single error, the string would have been accepted. Once a complete set of tests has been specified
for single errors, do the same for double errors and then triple errors. However, if there are of the order of N single-
error cases, there will be of the order of N2 double-error and N3 triple-error cases. Once past the single errors, it takes
a lot of judicious pruning to keep the number of tests reasonable. This is almost impossible to do without looking at
the implementation details.
Writing is a skill and that can be achieved by some experience and in-depth study of the
e f f e c t i v e t e s t c a s e s
W h a t i s a t e s t c a s e ?
W h a t i s a t e s t c a s e ?
A test case has components that describes an input, action or event and an expected response, to determine if a
feature of an application is working correctly. Definition by Glossary
There are levels in which each test case will fall in order to avoid duplication efforts.
In this level you will write the and user documentation.
L e v e l 1 : b a s i c t e s t c a s e s f r o m t h e a v a i l a b l e s p e c i f i c a t i o n
This is the in which writing test cases depend on actual functional and system flow of the
L e v e l 2 : p r a c t i c a l s t a g e
application.
This is the stage in which you will group some test cases and . Test procedure is nothing
L e v e l 3 : w r i t e a t e s t p r o c e d u r e
current updated functionalities to test rather than remaining busy with regression testing.
So you can observe a systematic growth from no testable item to a Automation suit.
W h y w e w r i t e t e s t c a s e s ?
The basic objective of writing test cases is If you are working in any
t o v a l i d a t e t h e t e s t i n g c o v e r a g e o f t h e a p p l i c a t i o n .
CMMi company then you will strictly follow test cases standards. So writing test cases brings some sort of
standardization and minimizes the ad-hoc approach in testing.
H o w t o w r i t e t e s t c a s e s ?
T e s t c a s e i d :
What to be verified?
U n i t t o t e s t :
A s s u m p t i o n s :
S t e p s t o b e e x e c u t e d :
E x p e c t e d r e s u l t :
A c t u a l r e s u l t :
P a s s / F a i l :
C o m m e n t s :
V e r i f y
[conditions]
W i t h
To identify what is being tested. You can use entering or selecting here instead of using depending on the
U s i n g :
situation.
F o r m a t o f T e s t C a s e
u n i q u e - t e s t - c a s e - i d : T e s t C a s e T i t l e
Short sentence or two about the aspect of the system is being tested. If this gets too long,
P u r p o s e :
break the test case up or put more information into the feature descriptions.
Assumptions that must be met before the test case can be run. E.g., "logged in", "guest login
P r e r e q :
List of variables and their possible values used in the test case. You can list specific values or
T e s t D a t a :
describe value ranges. The test case should be performed once for each c o m
of values.
b i n a t i o n
These values are written in set notation, one per line. E.g.:
loginID = {Valid loginID, invalid loginID, valid email, invalid email, empty}
Steps to carry out the test. See step formating rules below.
S t e p s :
1. visit LoginPage
2. enter userID
3. enter password
4. click login
5. see the terms of use page
6. click agree radio button at page bottom
7. click submit button
8. see PersonalPage
9. verify that welcome message is correct username
NOTE
N o t e s a n d Q u e s t i o n s :
QUESTION
F o r m a t o f t e s t s t e p s
Each step can be written very tersely using the following keywords:
l o g i n [ a s R O L E - O R - U S E R ]
Log into the system with a given user or a user of the given type. Usually only stated explicitly when the test
case depends on the permissions of a particular role or involves a workflow between different users.
v i s i t L O C A T I O N
Visit a page or screen. For web applications, LOCATION may be a hyperlink. The location should be a well-
known starting point (e.g., the Login screen), drilling down to specific pages should be part of the test.
e n t e r F I E L D - N A M E [ a s V A L U E ] [ i n S C R E E N - L O C A T I O N ]
Fill in a named form field. VALUE can be a literal value or the name of a variable defined in the "Test Data"
section. The FIELD-NAME itself can be a variable name when the UI field for that value is clear from context,
e.g., "enter password".
e n t e r F I E L D S
Fill in all fields in a form when their values are clear from context or when their specific values are not
important in this test case.
c l i c k " L I N K - L A B E L " [ i n S C R E E N - L O C A T I O N ]
Follow a labeled link or press a button. The screen location can be a predefined panel name or English phrase.
Predefined panel names are based on GUI class names, master template names, or titles of boxes on the page.
c l i c k B U T T O N - N A M E [ i n S C R E E N - L O C A T I O N ]
Press a named button. This step should always be followed by a "see" step to check the results.
s e e S C R E E N - O R - P A G E
The tester should see the named GUI screen or web page. The general correctness of the page should be
testable based on the feature description.
v e r i f y C O N D I T I O N
The tester should see that the condition has been satisfied. This type of step usually follows a "see" step at the
end of the test case.
v e r i f y C O N T E N T [ i s V A L U E ]
The tester should see the named content on the current page, the correct values should be clear from the test
data, or given explicitly. This type of step usually follows a "see" step at the end of the test case.
p e r f o r m T E S T - C A S E - N A M E
This is like a subroutine call. The tester should perform all the steps of the named test case and then continue
on to the next step of this test case.
Every test case must include a verify step at the end so that the expected output is very clear. A test case can have
multiple verify steps in the middle or at the end. Having multiple verify steps can be useful if you want a smaller
number of long tests rather than a large number of short tests.
I N T R O D U C T I O N T O B L A C K B O X T E S T I N G
The term 'Black Box' refers to the software, which is treated as a black box. By treating it as a black box, we mean that
the system or source code is not checked at all. It is done from customer's viewpoint. The test engineer engaged in
black box testing only knows the set of inputs and expected outputs and is unaware of how those inputs are
transformed into outputs by the software.
Types of Black Box Testing Techniques: Following techniques are used for performing black box testing
1 ) B o u n d a r y V a l u e A n a l y s i s ( B V A ) :
This testing technique believes and extends the concept that the density of defect is more towards the boundaries.
This is done to the following reasons
a) Usually the programmers are not able to decide whether they have to use <= operator or < operator when trying to
make comparisons.
b) Different terminating conditions of For-loops, While loops and Repeat loops may cause defects to move
c) The requirements themselves may not be clearly understood, especially around the boundaries, thus causing even
the correctly coded program to not perform the correct way.
2 ) E q u i v a l e n c e C l a s s T e s t i n g :
The use of equivalence classes as the basis for functional testing is appropriate in situations like
Decision tables are a precise and compact way to model complicated logic. Out of all the functional testing methods,
the ones based on decision tables are the most rigorous due to the reason that the decision tables enforce logical
rigour.
Decision tables are ideal for describing situations in which a number of combinations of actions are taken under
varying sets of conditions.
4 ) C a u s e - E f f e c t G r a p h i n g T e c h n i q u e :
This is basically a hardware testing technique adapted to software testing. It considers only the desired external
behaviour of a system. This is a testing technique that aids in selecting test cases that logically relate Causes (inputs) to
Effects (outputs) to produce test cases.
A Cause represents a distinct input condition that brings about an internal change in the system. An Effect
represents an output condition, a system transformation or a state resulting from a combination of causes.
(WBT) is also known as Code-Based Testing or Structural Testing. White box testing is the software
W h i t e B o x T e s t i n g
testing method in which internal structure is being known to tester who is going to test the software.
In this method of testing the testcases are calculated based on analysis internal structure of the system based on Code
coverage, branches coverage, paths coverage, condition Coverage etc.
White box testing involves the testing by looking at the internal structure of the code & when you completely aware of
the internal structure of the code then you can run your test cases & check whether the system meet requirements
mentioned in the specification document. Based on derived test cases the user exercised the test cases by giving the
input to the system & checking for expected outputs with actual output. In this is testing method user has to go
beyond the user interface to find the correctness of the system.
Typically such method are used at Unit Testing of the code but this different as Unit testing done by the developer &
White Box Testing done by the testers, this is learning the part of the code & finding out the weakness in the software
program under test.
For tester to test the software application under test is like a white/transparent box where the inside of the box is
clearly seen to the tester (as tester is aware/access of the internal structure of the code), so this method is called as
White Box Testing.
The White-box testing is one of the best method to find out the errors in the software application in early stage of
software development life cycle. In this process the deriving the test cases is most important part. The test case design
strategy include such that all lines of the source code will be executed at least once or all available functions are
executed to complete 100% code coverage of testing.
F L O W G R A P H S & P A T H T E S T I N G
FLOW GRAPHS
N T R O L F L O W G R A P H S :
The control flow graph is a graphical representation of a program's control structure. It uses the elements
named process blocks, decisions, and junctions.
The flow graph is similar to the earlier flowchart, with which it is not to be confused.
A flow graph contains four different types of elements. (1) Process Block (2)
F l o w G r a p h E l e m e n t s :
Decisions (3) Junctions (4) Case Statements
1.
P r o c e s s B l o c k :
A junction is a point in the program where the control flow can merge.
Examples of junctions are: the target of a jump or skip instruction in ALP, a label that
is a target of GOTO.
C O N T R O L F L O W G R A P H S V s F L O W C H A R T S :
F L O W G R A P H - P R O G R A M C O R R E S P O N D E N C E :
A flow graph is a pictorial representation of a program and not the program itself, just as a topographic
map.
You cant always associate the parts of a program in a unique way with flowgraph parts because many
program structures, such as if-then-else constructs, consists of a combination of decisions, junctions, and
processes.
The translation from a flowgraph element to a statement and vice versa is not always unique.
P A T H T E S T I N G :
o Path Testing is the name given to a family of test techniques based on judiciously selecting a set
of test paths through the program.
o If the set of paths are properly chosen then we have achieved some measure of test
thoroughness. For example, pick enough paths to assure that every source statement has been
executed at least once.
o Path testing techniques are the oldest of all structural test techniques.
o Path testing is most applicable to new software for unit testing. It is a structural technique.
o It requires complete knowledge of the program's structure.
o It is most often used by programmers to unit test their own code.
o The effectiveness of path testing rapidly deteriorates as the size of the software aggregate
under test increases.
P A T H T E S T I N G - P A T H S , N O D E S A N D L I N K S :
entry, junction, or decision and ends at another, or possibly the same junction, decision, or exit.
o A path may go through several junctions, processes, or decisions, one or more times.
o Paths consists of segments.
o The segment is a link - a single process that lies between two nodes.
o A path segment is succession of consecutive links that belongs to some path.
o The length of path measured by the number of links in it and not by the number of the
instructions or statements executed along that path.
o The name of a path is the name of the nodes along the path.
F U N D A M E N T A L P A T H S E L E C T I O N C R I T E R I A :
o There are many paths between the entry and exit of a typical routine.
o Every decision doubles the number of potential paths. And every loop multiplies the number of
potential paths by the number of different iteration values possible for the loop.
o Defining complete testing:
Exercise every path from entry to exit
Exercise every statement or instruction at least once
Exercise every branch and case statement, in each direction at least once
o If prescription 1 is followed then 2 and 3 are automatically followed. But it is impractical for
most routines. It can be done for the routines that have no loops, in which it is equivalent to 2
and 3 prescriptions.
o E X A M P L E : H e r e i s t h e c o r r e c t v e r s i o n .
P A T H T E S T I N G C R I T E R I A :
Any testing strategy based on paths must at least both exercise every instruction and take branches in all
directions.
A set of tests that does this is not complete in an absolute sense, but it is complete in the sense that
anything less must leave something untested.
So we have explored three different testing criteria or strategies out of a potentially infinite family of
strategies.
1. : P a t h T e s t i n g ( P i n f )
Execute all possible control flow paths through the program: typically, this is
restricted to all possible entry/exit paths through the program.
If we achieve this prescription, we are said to have achieved 100% path coverage.
This is the strongest criterion in the path testing strategy family: it is generally
impossible to achieve.
2. : S t a t e m e n t T e s t i n g ( P 1 )
Execute all statements in the program at least once under some test. If we do
enough tests to achieve this, we are said to have achieved 100% statement
coverage.
An alternate equivalent characterization is to say that we have achieved 100% node
coverage. We denote this by C1.
This is the weakest criterion in the family: testing less than this for new software is
unconscionable (unprincipled or can not be accepted) and should be criminalized.
3. : B r a n c h T e s t i n g ( P 2 )
Execute enough tests to assure that every branch alternative has been exercised at
least once under some test.
If we do enough tests to achieve this prescription, then we have achieved 100%
branch coverage.
An alternative characterization is to say that we have achieved 100% link coverage.
For structured software, branch testing and therefore branch coverage strictly
includes statement coverage.
We denote branch coverage by C2.
P R E D I C A T E S , P A T H P R E D I C A T E S A N D A C H I E V A B L E P A T H S :
P R The logical function evaluated at a decision is called Predicate. The direction taken at a
E D I C A T E :
decision depends on the value of decision variable. Some examples are: A>0, x+y>=90.......
P A A predicate associated with a path is called a Path Predicate. For example, "x is greater
T H P R E D I C A T E :
than zero", "x+y>=90", "w is either negative or equal to 10 is true" is a sequence of predicates whose truth
values will cause the routine to take a specific path.
M U L T I W A Y B R A N C H E S :
o The path taken through a multiway branch such as a computed GOTO's, case statement, or
jump tables cannot be directly expressed in TRUE/FALSE terms.
o Although, it is possible to describe such alternatives by using multi valued logic, an expedient
(practical approach) is to express multiway branches as an equivalent set of if..then..else
statements.
o For example a three way case statement can be written as: If case=1 DO A1 ELSE (IF Case=2 DO
A2 ELSE DO A3 ENDIF)ENDIF.
I N P U T S :
o In testing, the word input is not restricted to direct inputs, such as variables in a subroutine call,
but includes all data objects referenced by the routine whose values are fixed prior to entering
it.
o For example, inputs in a calling sequence, objects in a data structure, values left in registers, or
any combination of object types.
o The input for a particular test is mapped as a one dimensional array called as an Input Vector.
I N D E P E N D E N C E O F V A R I A B L E S A N D P R E D I C A T E S :
o The path predicates take on truth values based on the values of input variables, either directly
or indirectly.
o If a variable's value does not change as a result of processing, that variable is independent of
the processing.
o If the variable's value can change as a result of the processing, the variable is process
dependent.
o A predicate whose truth value can change as a result of the processing is said to be p r o c e s s
and one whose truth value does not change as a result of the processing is
d d
e p e n e n t p r o c e s s
d d
i n e p e n e n t .
o Process dependence of a predicate does not always follow from dependence of the input
variables on which that predicate is based.
P A T H P R E D I C A T E E X P R E S S I O N S :
o A path predicate expression is a set of boolean expressions, all of which must be satisfied to
achieve the selected path.
o Example:
o X1+3X2+17>=0
o X3=17
o X4-X1>=14X2
o Any set of input values that satisfy all of the conditions of the path predicate expression will
force the routine to the path.
o Some times a predicate can have an OR in it.
o Example:
A: X5 > 0 E: X6 < 0
B: X1 + 3X2 + 17 >= 0 B: X1 + 3X2 + 17 >= 0
C: X3 = 17 C: X3 = 17
D: X4 - X1 >= 14X2 D: X4 - X1 >= 14X2
A B C D + E B C D = ( A + E ) B C D
P R E D I C A T E C O V E R A G E :
o C o m p o u n P r e i c a t e :
o Testing Blindness is a pathological (harmful) situation in which the desired path is achieved for
the wrong reason.
o There are three types of Testing Blindness:
1.
d
A s s i g n m e n t B l i n n e s s :
C o r r e c t B u g g y
X=7 X=7
........ ........
if Y > 0 then ... if X+Y > 0 then ...
If the test case sets Y=1 the desired path is taken in either case, but there
is still a bug.
2.
d
E q u a l i t y B l i n n e s s :
C o r r e c t B u g g y
if Y = 2 then if Y = 2 then
........ ........
if X+Y > 3 then ... if X > 1 then ...
The first predicate if y=2 forces the rest of the path, so that for any
positive value of x. the path taken at the second predicate will be the
same for the correct and buggy version.
3.
d
S e l f B l i n n e s s :
C o r r e c t B u g g y
X=A X=A
........ ........
if X-1 > 0 then ... if X+A-2 > 0 then ...
The assignment (x=a) makes the predicates multiples of each other, so
the direction taken is the same for the correct and buggy version.
P A T H S E N S I T I Z I N G :
R E V I E W : A C H I E V A B L E A N D U N A C H I E V A B L E P A T H S :
o We want to select and test enough paths to achieve a satisfactory notion of test completeness
such as C1+C2.
o Extract the programs control flowgraph and select a set of tentative covering paths.
o For any path in that set, interpret the predicates along the path as needed to express them in
terms of the input vector. In general individual predicates are compound or may become
compound as a result of interpretation.
o Trace the path through, multiplying the individual compound predicates to achieve a boolean
expression such as
( A + B C ) ( D + E ) ( F G H ) ( I J ) ( K ) ( l ) ( L ) .
A D F G H I J K L + A E F G H I J K L + B C D F G H I J K L + B C E F G H I J K L
o Each product term denotes a set of inequalities that if solved will yield an input vector that will
drive the routine along the designated path.
o Solve any one of the inequality sets for the chosen path and you have found a set of input
values for the path.
o If you can find a solution, then the path is achievable.
o If you cant find a solution to any of the sets of inequalities, the path is un achievable.
o The act of finding a set of solutions to the path predicate expression is called P A T H
S . E N S I T I Z A T I O N
H E U R I S T I C P R O C E D U R E S F O R S E N S I T I Z I N G P A T H S :
o This is a workable approach, instead of selecting the paths without considering how to sensitize,
attempt to choose a covering path set that is easy to sensitize and pick hard to sensitize paths
only as you must to achieve coverage.
o Identify all variables that affect the decision.
o Classify the predicates as dependent or independent.
o Start the path selection with un correlated, independent predicates.
o If coverage has not been achieved using independent uncorrelated predicates, extend the path
set using correlated predicates.
o If coverage has not been achieved extend the cases to those that involve dependent predicates.
o Last, use correlated, dependent predicates.
P A T H I N S T R U M E N T A T I O N & A P P L I A T I O N S
P A T H I N S T R U M E N T A T I O N :
o Path instrumentation is what we have to do to confirm that the outcome was achieved by the
intended path.
The coincidental correctness stands for achieving the desired
d
o C o - i n c i e n t a l C o r r e c t n e s s :
F i g u r e 2 . 1 1 : C o i n c i e n t a l C o r r e c t n e s s
The above figure is an example of a routine that, for the (unfortunately) chosen input value (X =
16), yields the same outcome (Y = 2) no matter which case we select. Therefore, the tests chosen
this way will not tell us whether we have achieved coverage. For example, the five cases could be
totally jumbled and still the outcome would be the same. is what we have to P a t h I n s t r u m e n t a t i o n
F i g u r e 2 . 1 2 : S i n g l e L i n k M a r k e r I n s t r u m e n t a t i o n
marker may not do the trick because links can be chewed by open bugs.
.
4
F i g u r e 2 . 1 3 : W h y S i n g l e L i n k M a r k e r s a r e n ' t e n o u g h .
T w o L i n k M a r k e r M e t h o :
based on counters. Instead of a unique link name to be pushed into a string when
the link is traversed, we simply increment a link counter. We now confirm that the
path length is as expected. The same problem that led us to double link markers also
leads us to double link counters.