CTFL 2018 en v3.1k Watermark EU Tender

Axa Prioritară: 3 – Locuri de muncă pentru toți
Obiectivul specific: 3.12. Imbunătățirea nivelului de cunoștințe/competențe/aptitudini

aferente sectoarelor economice/domeniile identificate conform SNC și SNCDI ale
angajaților
Titlul proiectului: Contidigital Nord-Est
Cod proiect: POCU/861/3/12/145669
ny )
r ma
Ge
rl in (
Be
b H,
m
sG
v ic e
ser
© 2019 trendig technology services GmbH, Berlin (Germany)
lo gy
ec hno
t
en d ig
3 tr
0 2 Seminars and Training
©2
ISTQB® Certified Tester
Foundation Level
Training based on ISTQB® syllabus 2018 v3.1.1
Version 3.1k
0. General issues »
1. General information Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Trainers:
o Arjan Brands
ny )
ma
arjan.brands@trendig.com
r
Ge
o Björn Lemke
rl in (
bjoern.lemke@trendig.com Be
b H,
m
o Dominique Mühlbauer
e sG
dominique.muehlbauer@trendig.com v ic
y ser
o Pierre Baum
n o lo g
ch
pierre.baum@trendig.com
te
o Rahul Verma ndig
t re
23
rahul.verma@trendig.com
20
© Lieblang
o Werner
werner.lieblang@trendig.com
trendig technology services GmbH 10787 Berlin I Germany

Kleiststrasse 35 Telefon: +49 30 747628-0
Certified Tester Foundation Level © trendig technology services GmbH 2

1. General information Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Organization
Agenda, brakes, rooms, identification document )
ny
o Do you really need your cellular phone or laptop? r ma
Ge
in (
, B e rl
Brief introduction of the trainer and the participants H
G mb
s
o Your name and background
r v ic e
e
o Fields of work in the company, broad g y s work experience
o
o Experience in software c h nol
engineering and testing
t e
ig
r e nd
t
023 technology services
About trendig
2
©
o Company history and interesting facts
o Training program, experience in training
2. Enclosed material Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Enclosed material
• Complete set of slides
ny )
• Practical exercises
r ma
Ge
rl in (
Abbreviations Be
b H,
• TM = Test Manager m
e sG
• TA = Test Analyst v ic
y ser
g
• TTA o lo
= Technical Test Analyst
e chn
Icons used: d ig t
n
3 t re
2 0 2
definition from glossary
©
not required by syllabus (excursion)
exercise

3. Table of contents Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
This training is designed according to the Certified Tester Foundation Level

syllabus 2018. It is made up of seven chapters:
a ny)day
( G e rm
• Chapter 0 Introduction rl in 1
H , Be
• Chapter I Fundamentals of testing
G mb 1
i c es
• Chapter II Testing throughoutsthe erv software life cycle 1
g y
• Chapter III Static techniquesh n o lo 2
i g te c
• Chapter IV Test
t r endtechniques 2
2
• Chapter 0V23 Test management 2/3
©
• Chapter VI Tool support for testing 3
angajaților
Chapter 0. – Information / introduction
Chapter I. Fundamentals of testing ny )

r ma
• I/1 What is testing? Ge
rl in (
• I/2 Why is testing necessary Be
b H,
• I/3 Seven testing principles m
e sG
• I/4 Test process v ic
y ser
• I/5 The psychology of testingolog
te chn
Chapter II. Testing d igthroughout the software development lifecycle
t r en
023 development lifecycle models
• II/1 Software
2
©
• II/2 Test levels
• II/3 Test types
• II/4 Maintenance testing

angajaților
Chapter III. Static testing

• III/1 Static testing basics
ny )
• III/2 Review process
r ma
Ge
rl in (
Be
Chapter IV. Test techniques b H,
m
e sG
rv ic
• IV/1 Categories of test design techniques
se
• IV/2 Black-box test techniquesogy
h nol
te c
• IV/3 White-box test techniques
ig
d
re n
• IV/4 Experience-based test techniques
3t
02
©2
angajaților
Chapter V. Test management

• V/1 Test organization
ny )
• V/2 Test planning and estimation
r ma
Ge
• V/3 Test monitoring and control
rl in (
Be
• V/4 Configuration management b H,
m
• V/5 Risks and testing e sG
v ic
• V/6 Defect management y ser
n o lo g
Chapter VI. Tool support t ech for testing
ig
r e nd
• VII/1 Test tool t considerations
2 0 23
©
• VII/2 Effective use of tools

4. International organizations Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Qualifying program of the ISTQB®*

• In 1998, a multi level qualifying program was developed in Great Britain
)
m any
• The basics of software testing are laid down in the syllabus G r
for the
i n ( e
Foundation Level (current release: 2018) rl
, Be
H
• Since 2004 there are also certificates for theGmb
s
Advanced Level (Test Manager, Technical
r v ice Tester, Functional Tester)
e
• Expert Levels as well as further g ys
specializations are under preparation or ready
h n o lo
for download (see slideeon ISTQB certification scheme)
i g t c
• The local Testing d
enBoards of each country make up the umbrella organization
3 t r
0 2
International Software Testing Qualifications Board (URL: ISTQB.org), e.g.
©2
o in Spain: The Spanish Testing Board (SSTQB ) ®
o in Germany: The German Testing Board (GTB®)

o In India: The Indian Testing Board (ITB®)
* ISTQBâ = International Software Testing Qualifications Board
4. International organizations Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
ISTQB Agile Core Specialist
Test Management
Expert Level
ny )
ma
(CTEL) Improving the Test
r
Process
Ge
rl in (
Be Security Tester
Agile Test Leadership at H,
Test Manager
Advanced Level Scale (Beta)
G mbAnalyst
Test
Test Automation Engineer
(CTAL) s
eTechnical Test Analyst Model-Based Tester
v ic
ser
Agile Technical Tester
Usability Testing
y
n lo g
AgileoTester Automotive Software
ech
Tester
ig t Gambling Industry Tester
r e nd
Extension23
t
Foundation Level Mobile App Tester
2 0 Performance Testing
© Extension)
(CTFL
Acceptance Testing
Foundation Level
Certified Tester
(CTFL)

5. Syllabus and examination Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
This set of slides is based on the ISTQB® syllabus for the „CTFL“,
version 2018
o At the end of the course, there is an option to take the exam a ny)
to acquire the certificate CTFL ( G e rm
n
rliexam
o A representative of an examination institute will conduct , B e
the
o The exam consists of a 60 min multiple choice m bH
test
G
(non-native time extension of 15 min.) ices
v
o You are asked to answer 40 questions. y ser 26 questions (65%) must be answered
g
correctly in order to pass. The
h n olonumber of correct choices to be made for each
i g te c
question is explicitly stated
t r end
Learning 2 0 23
objectives / cognitive levels of knowledge
©
Cognitive levels are assigned to each chapter of the syllabus
o K1: remember
o K2: understand
o K3: apply
6. Goals and audience Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Main goals for the Certified Tester training

• Learning basic skills for planning tests )
ny
• Applying software testing skills to your projects r ma
Ge
in (
e rl
• Understanding the value that testing brings to stakeholders
,B
H
mb
• Selecting appropriate testing techniques and objectives
G
• Learning a common terminology ic es
e rv
Audience g ys
hn o lo
• The workshop addresses
i g tec (software) testers, developers and project managers
in the (software) d
enproduction as well as in industrial business that wish to give
3 t r
20 2
their knowledge a more solid ground
©
Workshop documents
• Set of the presentation slides
• Exercises and solutions
• References and glossary
trendig technology services Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
our services
ny )
r ma
Ge
rl in (
Be
b H,
m
sG
v ic e
innovation engineering y ser training events
g
h n o lo
ec
Design Thinking d ig tRequirements ISTQB® Agile Testing Days
n
3
Design Sprintst re Engineering Agile Testing - ATF® Agile Testing Days US
02
©2
Gamification Software Development IREB® Agile Testing Days
Open Air
User Centered Design Testing Services Design Sprint Master
Mobile App TestLab X-United
We prefer to accompany you already at the start of your project trip and are still at your side when
you reach your goal.
trendig technology services – our training portfolio Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Requirements IREB® Certified Professional for Requirements Engineering

Engineering - Foundation Level
ny )
ma
r Foundation Level
ISTQB® Certified Tester Foundation Level (GeExtension – Automotive
(3 and 4 days) & Boot Camp erlin
H ,B Software Tester
mb
G Advanced Level
Advanced Level
- Test Manager ice
s - Test Analyst
Advanced Level -
Technical Test Analyst
e rv
g ys
o
nol Level
Advanced
Software Testing chAutomation
- Test Mobile App Tester (MAT)
e Engineer - Foundation Level
d ig t
n
3 t re SeU - Certified
2 0 2 X-United
Selenium Engineer
©
Efficient software testing for the business departments
- Basics, methods and best practices for acceptance tests

trendig technology services – our training portfolio Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Innovation & Design Sprint

Design Master 2.0
ny )
er ma
n (G
Test-Driven Development
rl i
Software
H , Be
Development mb
s GBlockchain Ethereum Professional
BcU Certified
v ic e
ser
lo gy
ec hno Certified Scrum Certified Scrum
ig t
Master® plus Product Owner®
e nd
3 tr
02Practices
©2
Agile ISTQB® Certified Tester Foundation Level Extension, Agile Tester
Holistic Testing: strategies for agile teams
trendig technology services – our training in numbers Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
ny )
r ma
Ge
in (
, B e rl
17.000 3.000 Seminars bH
26 Countries
m
4 Languages
Participants sG
v ic e
er
ys
ol og
e chn
g tworldwide pass rates in percent %
Our
di
en
02 3 tr
© 2
100% 94% 91% 90% 98% 86%
ISTQB ISTQB IREB Design Sprint ISTQB Test

Holistic Testing
Foundation Agile Tester Foundation Master Automation

trendig technology services – our conferences Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
ny )
r ma
Ge
rl in (
Be
b H,
m
e sG
13 - 16 Nov. 2023
e r vic 2023
13 - 15 June 22 - 24 May 2023
Potsdam, Germany s Germany
Cologne, Chicago, IL, USA
gy
hn o lo
ig te c
nd agiletestingdays.us
t re
agiletestingdays.com openair.
0 23
©2 agiletestingdays.com
6. trendig technology services » copyright Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
trendig technology services – instructions for use

The present training material was created by trendig technology services GmbH.
)
It is based on the ISTQB Syllabus for the CTFL, Version 2018. any
G e rm
in ( of each label,
For all mentioned registered trademarks, all rights remain by the lowner
r
even if this is not explicitly specified. , Be
m bH
The use of this material, in the whole or parts of it,s isGnot allowed to any third party.
Clients of trendig technology services GmbH r v iceare allowed to use the material for
e
internal purposes and not for commercial g y suse.
l o
If you are not sure of whether c h
the
nointended use is permitted, please directly contact
i g te
end
trendig technology services GmbH.
www.trendig.com t r
2 0 23
©
Berlin, 27.01.23

angajaților
ny )
r ma
Ge
rl in (
Be
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
I. Fundamentals of testing »
Agenda Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Chapter I. Fundamentals of testing

• I/1 What is testing? )
ny
• I/2 Why is testing necessary r ma
Ge
• I/3 Seven testing principles rl in (
Be
• I/4 Test process b H,
m
e sG
• I/5 The psychology of testing v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

Learning objectives Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Learning objectives for fundamentals of testing: 1 of 2
1.1 What is testing? )

ny
FL-1.1.1 (K1) Identify typical objectives of testing r ma
Ge
FL-1.1.2 (K2) Differentiate testing from debugging
rl in (
Be
1.2 Why is testing necessary? H,
G mb
FL-1.2.1 (K2) s
Give examples of why testing isenecessary
r v ic
FL-1.2.2 (K2) Describe the relationshipsebetween testing and quality assurance and give
y
l og contributes to higher quality
examples of howotesting
FL-1.2.3 (K2) Distinguish t chn
ebetween error, defect, and failure
d ig
FL-1.2.4 (K2) en
Distinguish between the root cause of a defect and its effects
3 tr
1.3 202 testing principles
Seven
©
FL-1.3.1 (K2) Explain the seven testing principles
angajaților
Learning objectives for fundamentals of testing: 2 of 2
1.4 Test process )

ny
FL-1.4.1 (K2) Explain the impact of context on the test process r ma
FL-1.4.2 (K2) n ( the test process
Describe the test activities and respective tasksliwithin
Ge
B er
FL-1.4.3 (K2) H, the test process
Differentiate the work products that support
FL-1.4.4 (K2) G
Explain the value of maintainingestraceability
mb
between the test basis and
v i c
test work products er
g ys
1.5 o lo
The psychology of testing
hn
FL-1.5.1 (K1) Identify the c
tepsychological factors that influence the success of testing
ig
FL-1.5.2 (K2) end the difference between the mindset required for test activities
Explain
t r
023 and the mindset required for development activities
©2

angajaților

ny
Ge
Be
m
sG
• I/5 The psychology of testing v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
1. What is testing? Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
The economic importance of software

• The functioning of machines and equipment depends largely on software
)
a ny
rm or traffic
• We cannot imagine large systems in telecommunication, finance,
e
(G
control running without software rl in
Be
• More and more, the quality of software has become b H, the determining factor
m
for the success of technical or commercial
c e s Gsystems and products
i
s e rv
Testing and quality y
n o lo g
ech
• Software faults may cause many problems, e.g., lack of functionality, loss of
money, or even injury
t
ig or death
r e nd
t to assess the quality of the software
• Testing is a23way
2 0
© (including reviewing) insures the improvement of the quality of
• Testing
software products as well as the quality of the software development process
itself

angajaților
Dynamic and static testing

• Dynamic testing involves the execution of the test object
n y)
• Static testing is testing without the execution of a component ormaasystem.
e r
This includes reviews of documents and code n (G rl i
Verification and validation H , Be
G mb
• Testing often focuses on the verification s
r v iceof requirements, such as
specifications, user stories, or use scase. e This is mostly done when testing
g y
components or features olo
n
• But testing also involves t ech
validation, i.e., check that the system fulfills the
d ig
n
3 t re
needs and expectations of the customer and is fit for use. This is done mainly
0 2
during systems and acceptance testing
©2
Software quality
• the sum of attributes that refer to the capability of the software to meet given
requirements
angajaților
Testing means more than running tests

• Running tests is only one part of testing )
ny
• The test process includes: r ma
Ge
rl in (
o planning and control
Be
o choosing test conditions b H,
m
o designing and executing test cases e sG
v ic
o checking results y ser
ol og
o evaluating exit criteria
te chn
dig process and system under test
o reporting on the testing
en
o finalizing2or r
3 tcompleting closure activities
© 20
• Reviewing documents, source codes and conducting static analysis also help
to prevent defects appearing in the code

angajaților
Testing objectives 1 of 2
• Prevent defects by evaluating work products )

ny
o requirements, design, user stories, test cases, code r ma
Ge
o using reviews and / or static analysis
rl in (
Be
• Ensure software fulfils the requirements H,
G mb
o all specified functionality needs to be implemented s
r v ic e
e
• Check for completeness, validate
g y s the system
o
o meet customers expectations
c h nol
o meet requirementsigofte all other stakeholders
n d
3 t re
• Gaining confidence about the level of quality
2 0 2
©
o measure the level of quality to know the level of quality
angajaților
Testing objectives 2 of 2
• Find defects and failures, thereby reducing risks

ny)
ma
o testing reduces the probability of software causing damages at thercustomers site
Ge
r l in (
o regression testing shows that no new defects have been introduced during
development of changes ,B e
m bH
• Providing information for decision-making G
i c es
o give information to stakeholders about v
erthe risk of releasing the system at a given
time g ys
h n o lo
• Ensure (the test objects)
i g tec compliance with
t r end or external
o standards, internal
023regulations
o rules 2and
©
o laws and other legal requirements

angajaților
Testing objectives – more

Objectives of testing can vary depending on the test level, the context) and the
ny
software development model. Examples include: ma r
Ge
in (
e rl
during component testing / developer testing bH, B
s Gm
• finding as many defects as early as possible
c e
e rv i
ys
• ensure a high degree of code coverage
no lo g
t ech
during acceptancend ig
testing
t re
023the software is of use to the customer
• proving that
2
©
• gain sufficient knowledge about the quality of the system (and its risk)
to release the software
angajaților
Testing and debugging 1 of 2
While test execution may show failures, debugging is the process of

a n y)
finding, analysing and removing the causes of failures in software
( G e rm
rl in
, Be
bH
correcting
m
test debugging G re-test
i c es defects
er v
g ys
o
olactivities
• Test and re-test are testing c h n
Testing shows system i g te
failures
Re-testing proves,
t r endthat the defect has been corrected
2 0 23
©
• Debugging and correcting defects are developer activities
Through debugging, developers can reproduce failures, investigate the state
of programs and find the corresponding defect in order to correct it

angajaților
Testing and debugging 2 of 2
Debugging activities (done by developers), targeting component ) and

a n y
component integration testing (continuous integration), e.g. rm
( G e
• inspect the code by review to find causes of faulty e rlinbehaviour
B
b H,
• use a debugger tool to single step through s Gm the object instructions of a
ic e
test object s e rv
y
o l og
• use a source level debugger hn to execute high level language
e c
g t C#) one at a time
instructions (e.g.,diC,
n
3 t re
• use dynamic
20 2 analysis tools to track difficult to find errors, e.g., wild
©
pointers, memory leaks
In agile development testers can be part of component testing and
debugging (cross functionality)
angajaților
Summary
• The test process comprises )
ny
o test planning and control r ma
Ge
o test analysis, test design
rl in (
Be
o test implementation, test execution
b H,
m
o evaluating on exit criteria and reporting e sG
v ic
o test completion activities
y ser
n o lo g
• Testing objectives maycbe: finding defects, the level of quality, information for
t e h defects
ig
decision-making, preventing
r e nd
t means execution of the software, static testing means
23
• Dynamic0testing
2
among © others reviews of test artifacts (documents)
Dynamic testing shows failures that are caused by defects, debugging finds,
analyses and removes the cause of the failure, the defect

angajaților

ny
Ge
Be
m
sG
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
2. Why is testing necessary Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Why is testing necessary

Intensive testing of components, interfaces and systems can help to reduce the
risk of software failing at the customer’s site. a ny)
m er
G
r l in (
This is important, because software failure might cause enormous damage –
B e
H, of people
this could be financial damage or even injury or death
b
m
Examples for financial damage: sG
r v ic e
• stock is ordered or sold at a wrongsprice e
g y
• customer invoices contain n o
ol amounts
wrong
e c h
• payment data gets lost
d ig t or is destroyed
n
t re
Examples for23damage to people:
2 0
• a train©signal goes green before the level crossing gates are closed
• malfunction of a cardiac pacemaker
• meltdown of a nuclear power plant reactor

angajaților
Testers contribution to success - examples

• Testers detect defects during review of requirements or user stories.) This
helps avoiding development of incorrect or untestable software m any
/ features
r
Ge
o specifications are unambiguous, correct and complete in (
B e rl
b H,
• Testers participate in design and help in understanding the system, this way
m
s Gfinding defects early
reducing the risk of design defects and help
v ic e
ser
o design for testability, review design
g y
• Testers working with developers
h n olo support a better understanding of the code
and how to test it – this c
tereduces the risk of code defects and / or test defects
ig
end correctly and can easily be tested
o components rwork
3t
02 and validate software prior to release, so more errors can be
©2
• Testers verify
removed by the developers and less errors occur at the customer’s side.
Software better meets the stakeholder’s expectations
o systems test ensures that delivered software meets requirements
angajaților
Failure example 1: Ariane 5 launch

Flight 501, which took place on June 4, 1996, was the first test flight of) the
Ariane 5 expendable launch system. It was not successful; the rocketm anytore itself
e r
apart 37 seconds after launch because of a malfunction in nthe
i (Gcontrol software,
r l
e in history.
making the fault one of the most expensive computer, B
bugs
b H
Gm
es
ic from the Ariane 4, but the Ariane 5’s
e rv
The Ariane 5 software reused the specifications
s
flight path was considerably different yand beyond the range for which the reused code
had been designed. Specifically,n o logAriane 5’s greater acceleration caused the back-up
the
t
and primary inertial guidanceechcomputers to crash, after which the launcher’s nozzles
ig data. Pre-flight tests had never been performed on the re-
were directed by e
r nd
spurious
t
023 under simulated Ariane 5 flight conditions, so the error was not
alignment code
2
© before launch.
discovered
Wikipedia.com

angajaților
Failure example 2: lethal X-rays

Because of a software failure several patients received a lethal dose of)
ny
gamma rays: ma r
Ge
in (
B e rl
Therac-25 was a radiation therapy machine produced by
b H,Atomic Energy of Canada
Gm between 1985 and 1987, in
Limited. It was involved with at least six known accidents
s
e
which patients were given massive overdose
e r v icof radiation, which were in some cases on
s patients died of the overdoses. These
the order of hundreds of gray. At leasty five
l o g software
accidents highlighted the dangerso of
hn
control of safety-critical systems.
ec
ig t
Wikipedia.com
e nd
02 3 tr
©2
angajaților
Quality Assurance and Testing

• Quality assurance and testing are not the same. They are both part of) quality
ny
management (QM) ma er
n (G of their
• QM covers all activities of an organization regarding theliquality
r
Be
products and processes H, b
m
s G control
• QM includes both, quality assurance and quality
r v ic e
• Quality assurance ensures good processes, e trained personnel, compliance
g ys
with rules and regulations. This ol o mainly helps to avoid errors in the first place.
h n
Any error avoided is one
i g tecless that needs fixing. It provides confidence, that
t r end
the appropriate quality levels will be achieved. This is sometimes called
023 quality assurance”
“constructive
2
©
• Quality control consists of activities to find and fix errors. This includes
testing throughout the complete software development and maintenance
lifecycle. Reviews, test case execution and debugging (developer task). This
is sometimes called “analytical quality assurance”

angajaților
Quality control and testing
Quality Management (QM)

ny )
r ma
Ge
in (
B Controle rl
Quality Assurance
bH,
Quality
m
sG
v ic e
organisatorical
y ser dynamic
lo g
hno
i g te c
regulations black box
ndstandards white box
3 tre processes experience based
02
©2
technical static
methods reviews
tools Static analysis
languages
angajaților
Quality Assurance (constructive QA)

• Quality of process – quality management )
ny
er ma
Motto G
organisational
Guidelines (
rl in
Be
Standards
Quality Assurance
• Defects not made need not ,

(constructive QA)
be fixed m bH lists
Check
G Process rules and regulations
es
• Defects that were made v ic
need not be repeated y ser Legal requirements
ol og
• Prevent defects e chn Methods
ig t Tools
technical
e nd
02 3 tr Languages
© 2 Lists/templates
IDE*
* IDE = integrated development environment

angajaților
Quality Control (analytical QA)

• Quality of product – verification and test procedure )
ny
Equivalence G er ma
Motto ( partitioning
rlinvalue analysis
Black-box
Boundary
e
Quality Control (analytical QA)

• Defects should be B
, transition testing
detected as early as m bH
State
G Decision tables
es
possible in the process v ic Use case based testing
ser
dynamic
y
• Static testing
o l og experience-based techniques
examination without echn
ig t
executing the program Statement coverage
nd
White-box
e
3 tr
Decision coverage
• Dynamic 2
20testing includes Condition coverage
©
executing the program Path coverage
Reviews / walkthroughs
Control flow analysis
static
Data flow analysis

Compiler metrics/analyzer
angajaților
Errors, defects, and failures

• People can make errors which might lead to a defect in a test object
)
any
e rm
• Defects may propagate from one test object to another one,Ge.g.
(
in
o wrong requirement may result in wrong design
,B e rl
H
o wrong design may result in wrong code G mb
i c es
• Executed defects may cause a failure, v
er but not all failures are easily
g ys
observable
h n o lo
i g tec location
o data stored in the wrong
• Failures may t r end as false positives, caused by wrong data, wrong
occur
2 23
0results
expected
© (e.g., caused by out-dated specifications), test environment
problems or for other reasons
• False negatives – the opposite - are tests that do not detect defects they
should detect

angajaților

Errors may have different causes )
ny
• Human error er ma
( G
r n or the
A person introduced a defect into the software code, thelidata
e
B
configuration parameters H, b
o time pressure, distractions, new technologies Gm
s
r v ic e
e
g ys
o excessive demands because of complexity of code or design,
o
c nol
o misunderstandings abouthinterfaces, especially when there are a lot, and they are
e
ig t companies
developed by different
d
n
• Environmental3 treconditions may also cause failures
0 2
© 2of negative environmental conditions
Causes
o radiation, magnetism, electronic fields, sunspots
o pollution, hard disk crashes, power fluctuations
angajaților

• Error )
ny
ma
a human action that produces an incorrect result,
r
e.g., a programming error Ge
rl in (
Be
• Defect H,
An imperfection or deficiency in a work product where G mbit does not meet its requirements
s
r v ice that can cause the component or
or specifications. A flaw in a component or system
e
g ys
system to fail to perform its required function, e.g., an incorrect statement or data
definition ol o
ec hn
• Failure d ig t
n
An event in which
3 trea component or system does not perform a required function within
2
0 The physical or functional manifestation of a defect. A defect, if
specified 2limits.
©
encountered during execution, may cause a failure

angajaților
Defects, root causes and effects

• The root cause of a defect is the underlying cause why the defect
was introduced in the first place. a ny)
( G e rm
• Root cause analysis (RCA) is the process of discovering e rl in the root
, B
causes of problems so that appropriate solutions H can be found.
G mb
• It is more effective to systematically es
icprevent and solve underlying
e r v
issues rather than just treating s
l o gy ad hoc symptoms (putting out fires).
• RCA is done using differente c hno techniques, and methodologies to
d ig t
identify the rootncauses (chain of “why”s).
t r e
23
• Looking20beyond obvious cause and effect, RCA can show where
©
problems occur (processes, systems, tests) or issues arise.
• RCA can reduce the probability that, in a similar situation in the
future, the same defect is introduced again.
angajaților
Defects, root causes and effects

Goals and benefits
a ny)
• The first goal of root cause analysis is to discover theerroot m cause of
( G
a problem or event. rl in
, Be
• The second goal is to fully understand how m bH to fix, compensate, or
G
learn from any underlying issues within i c es the root cause.
er v
• The third goal is to apply lwhat g y swe learn from this analysis to
h n oo
systematically prevent ec future issues or to repeat success.
ig t
r e nd
t
2 0 23
©

angajaților
Defects, root causes and effects – example

Problem: )
ny
• a wrong invoice amount is computed r ma
Ge
rl in (
Root cause analysis: Be
b H,
• a wrong invoice amount is computed due to G am
defect that had its cause in an
e s
ambiguous specification of a requirement. ic The requirement problem was
s e rv
caused by an inexperienced author,y
o l og whose work should have been reviewed,
hn missed because developers thought testers to
but was not. The review was
be responsible for ig tecand vice versa
this
e nd
Activity:202 3 tr
©
• a change in process, that assigned review responsibility to the test
department solved this problem by eliminating the root cause
angajaților
Test case, test basis

• Test case (as per ISO 29119) )
ny
test case definitions include at least the following information
r ma
Ge
o pre-conditions
rl in (
Be
o set of input values
b H,
m
o set of expected results
e sG
v ic
ser
o expected post-conditions
y
o unique identifier
n o lo g
o dependencies on other h cases
ectest
ig t
o reference to thend requirement that will be tested
t r e
2 23
o how to 0execute the test and check results (optional)
©
o priority (optional)
• Test basis
The body of knowledge used as the basis for test analysis and design
(often a set of documents including requirements of a component or system).

angajaților
Summary
• Software failures may cause enormous damage )
ny
• Software quality (SQ) is the sum of attributes that refer to the ercapability of ma
( G
in
the software to meet given requirements e rl B
b preventing defects H,
• Quality Assurance (constructive QA) deals
Gm with
i ces
• Quality Control (analytical QA) deals
s erv with finding defects and correcting
y
them lo g
e c hno
• Errors made by people
d ig t cause defects in the software, which in turn might
cause failurestre n
when executed
2 3
• Root © 20 analysis helps to ensure that in future the same error is not
cause
made again in a similar situation
• Testers look for failures in the system and report them (testing). Developers
look for defects and correct them (debugging)
angajaților

ny
Ge
Be
m
e sG
• I/5 The psychology of testing v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

3. Seven testing principles Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Principle 1: Testing shows the presence of defects

Testing can prove the presence of defects )
ny
• Deviations discovered while testing show a failure r ma
Ge
rl in (
• The cause of the failure might not be obvious Be
H,
G
Testing cannot prove the absence of defects
mb
es
se rv ic
• Testing reduces the probability ofg ydefects remaining undiscovered.
l o
hno not prove the correctness of the software
The absence of failures does
ec
t
Examples:
en d ig
3 tr
0 2
• The test2procedure itself might contain errors
©
• The test conditions might be unsuitable for finding errors
angajaților
Principle 2: Exhaustive testing is impossible

Exhaustive testing
a ny)
• A test approach in which the test suite comprises all combinations
e rm of input
(G
values and preconditions l in r
, Be
Test case explosion bH
s Gm
r v ice and costs when testing
• defines the exponential increase of efforts
e
exhaustively ys
ol og
Sample test e chn
gt
di a (systematically or randomly derived) subset of all
• The test includes
r e nonly
t
023 values, usually based on a risk analysis
possible input
©2
Examples:
• Under real life conditions, sample tests are generally used
• Only in trivial cases testing all combinations of inputs/preconditions is feasible

angajaților
Principle 3: Early testing saves time and money

The earlier a defect is discovered, the less costly is its correction )
m any
• Highest cost effectiveness when errors are corrected before r
Geimplementation (
e r l in
Concepts and specifications may already be tested, B
m bH
• Defects discovered at the conception phase G
i c es are corrected with the least
v
effort and costs
y ser
g
Preparing a test is time-consumingh n o lo as well
e c
• Testing involves n d ig t than just test execution
more
3 t re
2
0 can be prepared before software development is completed
©2
• Test activities
Example: Testing activities (including reviews) could run in parallel to
software specification and design
angajaților
Principle 4: Defect clustering

Find a defect and you will find more defects nearby! )
ny
• Defects often appear clustered like mushrooms or cockroachesr ma
Ge
r l in (
• It is worth screening the same module where one defect Be was found
b H,
m
Testers must be flexible
c e sG
i
• The location of a defect might be s e rv
screened at higher detail level,
l o gy
e.g., starting additional tests noor modifying existing tests
t ech
ig
Example:
r e nd
t
023testing time is available, do more tests of the elements where
• If additional
2
lots of©defects have been found already. Probability is highest
to find further defects in those elements.

angajaților
Principle 5: Pesticide paradox

Repeating tests under the same conditions is ineffective
a ny)
• Each test case should contain a unique combination of input m
erparameters for
( G
a single test object, otherwise no additional information lcan
e r in be gained
, B
• If the same tests are repeated over and over again, m bH it is very unlikely that new
G
bugs can be found
i c es
rv
sedifferent
Tests must be revised regularly y
for code modules
o l og
n
• It is necessary to repeat
t ecahtest after changes have been made in the code
ig
(bug fixing, new functionality)
r e nd
t
• Automating
2 023tests can be an advantage if a group of tests cases is used
©
regularly
Example:
• Regression tests uses of this principle to ensure that changes to software
have no ill effect on the old functionality of a program
angajaților
Principle 6: Testing is context dependent

Testing is done differently in different contexts )
ny
Different test objects are tested differently r ma
Ge
r l in (
, Bethan those of an
• The engine controller of a car requires tests different
H
eCommerce application b
Gm
vi ces
ser
Test environment (test bed) vs. production environment
y
og
c h nol
• Tests take place in an environment other than the production environment.
The test environment t e
ig should be very similar to the production environment
t r end
0 23
• There will always be deviations between the test environment and the
© 2
production environment. These deviations impeach the conclusions drawn
after testing
Example: a test environment may be the production environment during a
production free time slot (e.g., at night, weekend)

angajaților
Principle 7: Absence-of-errors fallacy

Successful testing finds the most serious failures )
ny
• In most cases, testing will not find all defects of the system G er principle 2),
(see ma
in (
but the most serious defects should be found e rl B
b H,
This alone does not prove the quality of the
Gmsoftware
ces
vi meet the needs and expectations of
• The functionality of the software mays ernot
y
the users lo g
e c hno
ig t into the product, it must be built in from the very
• You cannot test quality
d
beginning! tre n
2 0 23
© the customer was expecting the weight of goods to be calculated
Example:
from available data base information but forgot to specify this. So, the
implementation assumed the weight to be entered manually.
angajaților
Summary
• Tests can help finding defects in the software, however, they cannot give
proof of the absence of defects a ny)
m r
Ge
in (
, B erl sample testing is
• For non-trivial systems, exhaustive testing is impossible,
necessary bH
s Gm
• Early testing helps reduce costs because
r v ice defects discovered early on are
e
fixed with less effort ys
n o lo g
• Defects show up clustered.
t ech Finding a defect at one place will mean you
probably find another ig
r e nd defect nearby
• Repeating 3t
02identical tests produces no new information
© 2
• Each particular environment determines the way in which tests will run
• “Error-free” software does not imply suitability for use

angajaților

ny
Ge
Be
m
sG
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
4. Test process Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Testing process in context

Testing requires a certain set of activities to be performed. The activities) are
called the test process. The order of execution, the roles, the work m any
products and
e r
G test process
i (
other details might differ from organization to organization. nThe
r l
can be influenced by: , Be
m bH
• the software development life cycle (e.g., GV-model, Scrum, RAD)
i c es
er v
• the test levels and types of testing
g y s that are relevant
h n o lo
• the product and projecterisks
i g t c
• standards, rules d regulations (internal, external)
enand
t r
2 0 23
• ©
operational constraints like budget, time, complexity, capacity
• organizational influences like a quality policy

angajaților
Testing process in context

Aspects of organizational test processes considering the following: )
ny
• Test activities and tasks r ma
Ge
• Test work products rl in (
Be
H,
mb products
• Traceability between the test basis and test work
e sG
rv ic
are described on the following slides.
s e
coverage criteria for the test basis
l o gy can be used as metrics to monitor the
o
progress in testing chn te
Example: ig
tr end
023 application, the test basis may include a list of requirements
for a warehouse
2
© of supported data bases. The coverage criteria may require at least
and a list
one test case for each element (requirement, data base) of the test basis.
Test results will inform stakeholders whether specified requirements are fulfilled
angajaților
Testing activities and tasks

• Depending on the approach chosen, testing will take place at different) points
ny
within the development process ma r
Ge
• Testing is a process itself rl in (
Be
b H,
• The testing process is determined by the following
Gm phases:
es
o Test planning v ic
y ser
o Test monitoring and control
ol og
o Test analysis
te chn
ig
end
o Test design
tr
0 23
o Test implementation
2
© execution
o Test
o Test completion
• Test phases may overlap, activities may be executed iteratively, so an

appropriate tailoring to the context of the project and system is necessary

angajaților
Testing throughout the software development process

• Testing is more than test execution!
ny)
Test Planning
a
• Includes overlapping and e rm
Test Monitoring & Control

backtracking rl i n (G
Test Analysis
H , Be
• Each phase of the testing process G mb Test Design
s
takes place concurrent to the
r v ic e
e
phases of the software
g ys Test Implementation
development process nolo
ech
d ig t Test Execution
3t re n
02
©2 Test Completion
angajaților
Test planning – main tasks

• Determining the scope and risks
ny)
Test Planning
a
• Identifying the objectives of testing and exit rm
(G e
criteria rlin Test Analysis

Be
H,
• Determining the approach: test techniques,Gmb
Test Design
es
v ic
test coverage, test levels, testing teams
y ser
• Implement testing method / test gstrategy, Test Implementation
h n o lo
plan time for further activities
i g te c Test Execution
end
• Acquiring andtrscheduling test resources:
people, 2 2 3
0 infrastructure, test budget
test
© Test Completion
• The work product of this phase is the test
plan or the master test plan and agreed upon
by all important stakeholders

angajaților
• Master test plan (German: Testkonzept)

A test plan this is used to coordinate multiple test levels or test types
(a document describing the scope, approach, resources and schedule of intended ) test
activities. It includes, but is not limited to, the test items, the features tom n y
beatested,
e r
resources and contingency planning) ( G
rl in
• Test strategy , Be
bH
Documentation that expresses the generic requirements
s Gm for testing one or more
e
projects run within an organization, providing
e r v icdetail on how testing is to be performed,
and is aligned with the test policy. y s
g
h n ololevels to be performed and the testing within those
(a high-level description of the test
orc program)
levels for an organizationte
en d ig
3 tr
• Test approach
© 20 2
The implementation of the test strategy for a specific project
(it typically includes the decisions made that follow the (test) project’s goal and the risk
analysis, starting points regarding the test process, the test design techniques to be
applied, exit criteria and test types to be performed)
• Exit criteria (after Gilb and Graham)

The set of conditions for officially completing a defined task
angajaților
Test monitoring and control – main tasks

Monitoring the status of the test process
by comparing the progress achieved against ny)
Test Planning
a
rm
(G e
the test plan.

rlin Test Analysis
Be
Test control is an on-going activity influencing b H,
test planning. The master test plan may be s Gm Test Design
e
r v ic
modified according to the information eacquired
from test controlling. g ys Test Implementation
l o o
e chn
d i t be
• Necessary activitiesgwill
Test Execution
n
started accordingly
t re 3
2 analyze results
20and
• Measure
© Test Completion
• The test progress, test coverage and the
exit criteria (agile: definition of done) are
monitored and documented
• Start corrective measures, prepare and take
decisions
angajaților
Test monitoring and control - evaluating exit criteria

• Assessing test execution against the
defined objectives (e.g., test end criteria) ny)
Test Planning
a
e rm

• Evaluating test logs (summary of test rl i n (G
Test Analysis
, Be
activities, test results, communicate
m bH
exit criteria)
e sG Test Design
e rv ic
g ys
• Provide information to allow the decision,
Test Implementation
whether more tests shouldno lo place
take
ech
d ig t Test Execution
3t re n
02
©2 Test Completion
angajaților
Test analysis – main tasks 1 of 2
• Reviewing the test basis (e.g., requirements,

system architecture, design, user stories, ny)
Test Planning
a
rm
(G e
epics, interfaces, code, database queries,

rlin Test Analysis
risk analysis reports, models) Be
H,
• Evaluating testability of test basis, e.g. G mb Test Design
s
ambiguities, omissions, inconsistencies,
r v ic e
se
superfluous gy Test Implementation
• Identifying and prioritizing h n oloconditions
test
i
(what to test) – definitiong tec of Test Execution
nd
t re
measurable3coverage criteria
2 2
0the
related
© to considered test level Test Completion
• Test conditions related to risk, functional,
non-functional, structural and other
business and technical factors
• Creating bi-directional traceability between test basis and test conditions

angajaților
Test analysis – further considerations 2 of 2
• Using test design techniques can help to

avoid forgetting important test conditions ny)
Test Planning
a
e rm

and support the process of making test
rl i n (G
Test Analysis
conditions more precise
H , Be
• Test conditions may be used for formulation Gmb Test Design
s
of test charters for experienced based r v ic e
e
g ys
testing, these may be used to determine Test Implementation
lo
coverage achieved hno
• Finding defects during
te c
ig the test analysis is not Test Execution
r e nd
t (finding defects early), it
only beneficiary
2 0 23
also helps
© to check, whether requirements Test Completion
appropriately address stakeholder needs
• Creating test cases is one possibility of test analysis, e.g., creating test cases
from user stories and acceptance criteria using ATDD or BDD
• “What to test”
angajaților
Test design – main tasks 2 of 13 of 2
• Designing the tests / test cases

ny)
Test Planning
a
o Create and prioritize logical / high level rm
(G e
test cases (test cases without specific values

rlin Test Analysis
for test data) Be
b H,
o Positive tests give proof of the functionality,
s Gm Test Design
negative tests check the handling of error
r v ic e
situations se y
n o lo g Test Implementation
• Identify specific test conditions and required
t ech
test data ig Test Execution
r e nd
t availability of test data and / or the
023of generating test data
o evaluate the
© 2
feasibility Test Completion
• ”How to test”

angajaților
Test design – main tasks 3 of 23 of 2
• Designing the test environment (test bed)

ny)
Test Planning
a
o (Exclusive) availability of the test environment,
e rm

time windows, etc.
rl i n (G
Test Analysis
o Define the operation of the test environment, , Be
including user administration m bH
e sG Test Design
rv ic
o Loading data sets and system parameters
se
y
o Connecting the test environment
o l og to adjacent Test Implementation
systems hn
• Test infrastructure
t r endand test tools, if needed
2 023 procedures and responsibilities
o Processes,
Test Completion
©
o choosing, provisioning, installation and
operations of test tools
• Capturing bi-directional traceability

o between test basis, test conditions, test cases and test procedures
angajaților
• Test data
Data created or selected to satisfy the execution preconditions and inputs to
execute one or more test cases y) an
• Input value G e rm
r l in (
Be
An instance to an input (a variable that is read by a component or system)
,
H
• Test oracle G mb
i c escompare with the actual result of
A source to determine expected results er v to
g ys
the system under test. (e.g., benchmarks, the results of earlier tests, user’s
n o lo
manual or specialized knowledge.
ch It should not be the code.)
e
d i gt
n in glossary)
• Test coveragere(not
3 t
The degree
2 0 2 to which a specified item has been exercised by a test suite
©
(expressed as a percentage). Used mostly on white-box tests to determine
code coverage

angajaților
Test implementation 1 of 2
• finalizing, implementing and prioritizing

test cases ny)
Test Planning
a
e rm

o preparation of test data
rl i n (G
Test Analysis
, Be
• developing and prioritizing test procedures
m bH
o creating test data e sG Test Design
e rv ic
ys
o preparing test harnesses (optional)
l og Test Implementation
hno if necessary
o writing automated test scripts,
end
• creating test suites from the procedures
t r
2 023 test execution
o for efficient
Test Completion
©
• building and verifying the test environment
• verifying and updating traceability (test basis – test cases)
• In experience-based testing, test design and test implementation are typically
done in parallel
angajaților
Test execution 2 of 2
• executing tests (manually or automatically)

ny)
Test Planning
a
o follow test sequence stated in the test execution rm
(G e
schedule (test suites, order of test cases)

rlin Test Analysis
Be
• test results logging and analysis
b H,
o recording identities and versions of the Gm Test Design
es
software / test tools / testware rv ic
e
ol
• comparing actual results – nexpected o results
e c h
• reporting and analyzing
d ig t defects (anomalies) Test Execution
n
tre cause
to establish 3their
0 2
© 2/ test data / document / execution
o code Test Completion
• repeating test activities to confirm a fix (retest or confirmation test after

defect correction) or an unimpacted quality level (regression test)
o Ensure that changes (after installing a new release, or defect fixing) did not uncover
other or introduce new defects
angajaților
• Test suite (test sequence):

a set of test cases or test procedures to be executed in a specific test cycle.
(often constructed, so that the post condition of one test case is used as the )
ny
precondition for the next one) ma r
Ge
• Test procedure specification (test scenario):
rl in (
, Be order, and any
Documentation specifying a sequence of test cases in execution
H
associated actions that may be required to set up the b
initial preconditions and any wrap
s Gm test script [after ISO 29119])
ce
up activities post execution. (also known as i(manual)
rv
• Test log (German: Testprotokoll): se
gy o
c h nol details about the execution of tests (e.g., when the
A chronological record of relevant
te were produced)
tests were done, what results
en d ig
3 tr
• Regression tests:
02previously tested program following modification to ensure that defects
Testing of2a
©
have not been introduced or uncovered in unchanged areas of the software, as a result
of the changes made. (It is performed when the software or its environment is changed)
• Confirmation testing (re-testing):
Dynamic testing conducted after fixing defects with the objective to confirm that failures
caused by those defects do not occur anymore.
angajaților
Test completion – main tasks

• Collect data from completed test activities to
consolidate experience, testware, facts and ny)
Test Planning
a
rm
(G e
numbers
rlin Test Analysis
Be
• Closure of defect reports or raising change b H,
m
requests or new backlog items for any
c e sG Test Design
i
remaining open points e rv
o
• Documenting the acceptance
c h nolof the system
e
d ig t testware, the test
• Finalizing and archiving Test Execution
n
environment 3 tre the test infrastructure for
and
0 2
© 2 hand over to operations
later reuse, Test Completion
• Analyzing “lessons learned” for future projects

• Using the information gathered to improve test (process) maturity

angajaților
Test work products 1 of 4
• Test work products are created in the different phases of the testing process,
a ny)
they differ by nature like organizations, people, projects and technology
rm
(G e
differs. Even names for work products and their contents ndiffer
rl i
Be
H, ISO 29119 for the
• Suggested is to follow the syllabus, the glossary and
b
relevant work products Gm
es
v ic
ser
Test planning work products
gy
o from the planning phase is the test plan.
c h nol
• The most important work product
e
g t one, if multiple test levels or test types are covered
There can be more than
di
n
tre that are described in the test plan:
• Some elements
20 23
o Test
©basis
o Traceability to testware
o Entry and / or exit criteria (like definition of done)
angajaților
Test monitoring and control work products )

ny
• Test reports, including test progress and test summary reports r ma
Ge n(
rl i
Beand test results
• Contain audience-relevant information about progress
b H,
• Other topics covered are project management G mrelated like tasks, resources
e s
and effort v ic
y ser
Test analysis work products lo g
e c hno
• Defined and prioritized t
ig test conditions, may be in the form of test charters
n d
• Bi-directional re
3 ttraceability test basis – test conditions
2 0 2
© of defects in the test basis (testability review report)
• Reporting

angajaților
Test design work products )

ny
• Test cases, covering the identified test conditions r ma
Ge
rl in (
• Re-usable high level (logical) test cases Be
H,
G mb
• Definition of the required test data and infrastructure (environments, tools,
e s
office environment, laboratory) v ic er
• The extend of documentation o s as the time needed to realize the
asgywell
o l
chndiffer significantly
required infrastructure can
e
d i gt
Test implementation n work products
3 t re
02
©2
• Test procedures and their sequencing
• Test suites
• Test execution schedule
• Realized test data and test environments, automated test scripts
• A test oracle may be used to identify expected results
angajaților
Test execution work products )

ny
• Status documentation of test cases and test procedures (e.g., erready to run, ma
( G
in
pass, fail, blocked, skipped) e rl ,B
H
• Defect reports
G mb
ces infrastructure, testware)
• Documentation of items in testing (testviobject,
er
y s coverage and other elements can be
• Based on the status and traceability,
g
l o
analyzed and reported chno
e
i gt
nd products
Test completionework
0 2 3 tr
©2
• Test summary reports
• Improvement suggestions or actions for future iterations (agile retrospective)
or projects (lessons learned, continuous improvement)
• Change request or new product backlog items
• Finalized and archived testware

angajaților
Traceability (bi-directional) between test basis / test work products

• Upstream traceability: which test case was included in the test portfolio,
) based
n y
on which part of the test basis (requirement)? ma er
• Downstream traceability: the consequences of changes (Gthe requirements
in in
r l
on the tests to be made can be identified directly H, Be
m coverage b
sG
• Traceability also helps to determine requirement
e test scenarios
v ic
ser
lo gy
hno test case 6
te c test case 5
test object n d ig defining
test case 6 test case 4
and 23 t re test test case 5
0
©2
require- test case 4
requirements
ments test case 3
on the
and test test case 2
test object test case 6
conditions test case 1 test case 5
test case 1
angajaților
Traceability
Good traceability supports: )
ny
• Impact analysis for changes r ma
Ge
rl in (
• Auditability of testing Be
b H,
m
• Adhering to IT governance e sG
v ic
• Improving the understandabilitygof ser
y test progress reports and test summary
l o
hnoof elements of the test basis
reports to include the status
e their tests c
gt
o requirements that ipassed
e nd
3 tr that failed their tests
o requirements
02
©2
o requirements that have pending tests
• Relating the technical aspects of testing to stakeholders in terms that they can
understand
• Providing information to assess product quality, process capability, and project
progress against business goals

angajaților
Definitions
• Test object:
an y)
The component or system to be tested (the subject to be examined:
rm e.g., a
G e
in ( process)
document or a piece of software in the software development
rl
Be
• Test condition: H,
G mb
An aspect of the test basis that is relevant sin order to achieve specific test
r v ic e
objectives e
g ys
o
• Test execution: c h nol
e
The process of running
d ig t a test on the component or system under test,
n
tre results
producing actual
3
© 20 2
• Test execution schedule (non glossary):
A scheme for the execution of test procedures. The test procedures are
included in the test execution schedule in their context and in the order in
which they are to be executed
angajaților
Summary
The testing process can be divided into different phases: )
ny
• Test planning covers the activities defining test approach for r test phases
eall ma
( G
in
e rl
as well as planning resources (time, personnel, machines)
B
b H,
• Test control controlling activities covering all G phases
m of the testing process,
e s
rv ic
including the evaluation and reporting covering exit criteria and recording of
test results in written form s e
y
n o lo g
ech covers designing the test cases and their
• Test design (specification)
t
expected results ndig
e
0 2 3 tr covers defining test data, performing test execution and
• Test execution
© 2 results
comparing
• Test completion covers closure of incident report and lessons learned
• The respective work products should be made traceable (bi-directional), to
fully satisfy the need for information of stakeholders

angajaților

ny
Ge
Be
m
sG
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
5. The psychology of testing Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Roles and responsibilities

Developer role Tester role
a ny)
• implements requirements • plans testing activities
( G e rm
• develops structures • designs test casesrl in
, Be
• designs and programs the software bH only with finding defects
• is concerned
m
G
• creating a product is his success •i c es
finding a defect made by a developer
v
er is his success
y s
n o lo g
t ech
ig
r e nd Perception:
t
2 023 are constructive!
Developers Testers are destructive!
©
Wrong!
Testing is a constructive activity as well,
it aims at finding (and eliminating) defects from a product!

angajaților
Communication and interpersonal skills

• Building positive relationships is essential to bring about the message
)
a ny
• Start with collaboration by reminding everyone of the common e rmgoal of
( G
better-quality systems l in r
Be
• Emphasize the benefits of testing H,
G mb
o defect information can help authors improve s their work products and their skills
r v ic e
o defects found and fixed during testing se will save the organization time and money
yquality
o g
nol
and reduce overall risk to product
e c h
ig t
• Communicate test results and other findings in a neutral, fact-focused way
without criticizing d
nthe person who created the defective item. Write objective
3 t re
02defect reports and review findings
and factual
©2
• Try to understand how the other person feels and the reasons they may react
negatively to the information
• Confirm that the other person has understood what has been said and vice
versa
angajaților
Personal attributes of a good tester 1 of 2
• Curious, perceptive attentive to detail – not all error show up great manor
)
any
e rm
o to comprehend the practical scenarios of the customer
o to be able to analyze the structure of the test in (G
,B e rl
o to discover details, where failures might show H
G mb
ces has a critical eye
• Skepticism (professional pessimism) iand
s e rv
gyjust have to find them
o test objects contain defects – you
lo
no are told by the developers
o do not believe everythinghyou
ig
o one must not getdfrightened
te c
by the fact that serious defects may often be found
r e nan impact on the course of the project
t
which will 3have
2 02
©

angajaților
Personal attributes of a good tester 2 of 2
• Good communication skills )

ny
o to bring bad news to the developers r ma
Ge
o to overbear frustration states of minds in (
B system must be e rl
of, the
o both technical as well as issues of the practical use H
b
understood and communicated Gm
ic es
erv or to ease difficult situations
o positive communication can help to avoid
s
y
og
o to quickly establish a workinglrelationship with the developers
h no
• Experience te c
e n d ig
t r
o personal 3factors influencing error occurrence
2 0 2
o experience
© helps identifying where errors might accumulate (error guessing)
angajaților
Differences: to design – to develop – to test

• Testing requires a different mindset from designing or developing new)
ny
computer systems ma r
Ge
o common goal: to provide good software in (
B e rl
H, requirements
o design mission: help the customer to supply the right
b
m
s G into functions, create a product
o developer’s mission: convert the requirements
v ic e
o tester’s mission: examine the correct
s erimplementation of the customer’s
requirements g y
o
c h nol
e
• In principle, one person
d ig t can be given all three roles to work at
n
o differences in
3 tregoals and role models must be taken into account
0 2
o this is2difficult but possible
©
o other solutions (independent testers) are often easier and produce better results,
they are important and / or mandatory for complex or safety-relevant systems
o Be aware of confirmation bias, bringing independent testers would mean bringing
other types of cognitive biases

angajaților
Summary
• People make mistakes, every implementation has defects )
ny
• Human nature makes it difficult to stand in front of one’s own r
edefects (error ma
n ( G
i
blindness – confirmation bias) e rl B
b H,
• Developer and tester means two different worlds
Gm meet each other.
e s
rvicis created that was not there before
o developing is constructive – something
se
y
og glance – defects will be found!
o testing seems destructive at first
ol
n
e h test are constructive in their objective to ensure
o Together, developmentcand
t
ig defects possible
software with thedleast
n
3 t re
202 testing enhances quality of testing:
• Independent
©
instead of developer teams, use tester teams or teams with external
personnel for testing
Keywords Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Keywords
• test design
• coverage
ny)
• test execution
• debugging a
• test implementation
( G e rm
• defect in
• error , e rl
• test monitoring
B
H
• testbobject
• failure G m
i c es• test objective
• quality er v • test oracle
• quality assurance g ys
h n o lo • test planning
• root cause e c • test procedure
• test analysis d ig t
• test basis 23 t
re n • test process
0 • test suite
• ©2
test case • testing
• test completion • testware
• test condition • traceability
• test control • validation
• test data • verification
angajaților
ny )
r ma
Ge
rl in (
Be
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
II. Testing throughout the software development lifecycle »

angajaților
Chapter II. Testing throughout the software development lifecycle

• II/1 Software development lifecycle models )
ny
• II/2 Test levels r ma
Ge
• II/3 Test types rl in (
Be
• II/4 Maintenance testing b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

angajaților
Learning objectives for testing throughout the sdlc: 1 of 2
2.1 Software development lifecycle models

a ny)
FL-2.1.1 (K2) Explain the relationships between software development
e rmactivities and
n(
test activities in the software development lifecycle G
rl i
Be
FL-2.1.2 (K1) Identify reasons why software development
b H, lifecycle models must be
adapted to the context of project andmproduct characteristics
e sG
2.2 Test levels
se rv ic
y
FL-2.2.1 (K2) Compare the different
o l og test levels from the perspective of objectives, test
n typical defects and failures, and approaches and
ech
basis, test objects,
ig t
responsibilities
e nd
02 3 tr
©2

angajaților
Learning objectives for testing throughout the sdlc: 2 of 2
2.3 Test types

a ny)
FL-2.3.1 (K2) rm
Compare functional, non-functional, and white-box testing
Ge
FL-2.3.2 (K1)
r l in (
Recognize that functional, non-functional, and white-box tests occur at
any test level ,B e
bH
FL-2.3.3 (K2) Gm testing and regression testing
Compare the purposes of confirmation
es
v ic
2.4 Maintenance testing
y ser
g
FL-2.4.1 (K2) olo for maintenance testing
Summarize triggers
FL-2.4.2 (K2) Describe the chn
te role of impact analysis in maintenance testing
en d ig
02 3 tr
© 2

angajaților

ny
Ge
Be
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

1. Software development lifecycle models Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Software Development and Software Testing

Principles of all models )
ny
• Every development activity has a corresponding test activity r ma
Ge (
in
o no software goes into operation without being tested
,B e rl
• Each test level should be tested specifically mbH
G
es
v ic
o each test level has its own test objectives
er
ys
• Testing begins long before test
og execution ol
o as soon as development n
chbegins, the preparation of the corresponding tests can start
e
d ig t
o this is also the n for document reviews starting with concepts, specification and
ecase
2 3 tr
overall 0design
©2
• Tester participate in requirements discussion and clarification
o as soon as development begins, the preparation of the corresponding tests can start
o this is also the case for document reviews starting with concepts, specification and
overall design

angajaților
Software Development and Testing

Testing along sequential development models )
ny
• The general V-model is the most commonly used software r
edevelopment ma
( G
in
e rl
model. Development and test are two equal branches
,B
H
mb
o Each development level has a corresponding test level
e sG
rv ic
§ Tests (right hand side) are
designed in parallel with
y seRequirement Acceptance testing
software development
o l og Definition
(left hand side) hn ec
ig t place
Functional System
§ Testing activities take System testing
throughouttrthe n d
e complete
Design
0 2
software 3life cycle
©2
Technical System
Integration testing
Design
Component
Component testing
Specification
Programming

angajaților
Testing along the general V-model

Software development branch )
ny
• Requirement definition r ma
Ge
Requirement
o specification documents definition
rl in (
Be
• Functional system design b H,
msystem
Functional
G
o design functional program flow esdesign
er v ic
ys
• Technical system design og Technical system
o define architecture/interfacesc h nol design
e
d ig t
n
• Component specification
3 t re Component
2
0of component
o structure specification
©2
• Programming
Programming
o create executable code

angajaților
Testing along the general V-model

Software test branch )
ny
• Acceptance testing ma
erAcceptance testing
( G
o formal test of customer requirements
rl in
Be
• System testing b H,
m
sG System testing
o integrated system, specifications
ic e
s e rv
• Integration testing gy
hn o lo Integration testing
te c
o component interfaces
ig
t r end
• Component testing Component testing
3
202 functionality
o component’s
©
Programming

angajaților
Types of iterative models

• Iterative software development
a divided into ny)
o The activities: requirements definition, design, development, testing
e rm
are
small steps and run continuously n (G rl i
Be
H, must be reached with the
o In order to redirect the progress, if necessary, a consent
b
customer after each iteration m G
i c es
• Iterative development where groups v
r features are specified, designed, built,
y seof
and tested together in a series g
of cycles, often of a fixed duration
h n o lo
ec
o Changes to features tdeveloped in earlier iterations
i g
t r end scope
o Changes in project
023 delivers working software (until all software is delivered or
o Each 2iteration
©
development is stopped)

angajaților
Types of iterative models

• Iterative models are, for example:
a ny)
o Rational Unified Process (RUP): object oriented model and a product
e rm of the
G
company Rational (IBM). It mainly provides the modeling language
r l in ( UML and support
Be
for the Unified Process, long iterations, bigger feature groups
H,
o Rapid Application Development (RAD): the user
G mbinterface is implemented using
s
ic e
out-of-the-box functionality faking the functionality which will be later developed
rv
e
y s few weeks), each covering a small increment
o Scrum: small iterations (hours, days,
g
o
nol
of functionality, like some enhancements
ch
or a few user stories or features
e
ig t with or without fixed length iterations, delivers features on a
o Kanban: implemented
d
ren or grouped to be released at once
continuous basis
3t
02 models like Extreme Programming (XP)
o other2Agile
©
o Prototyping: building quickly a usable representation of the system, followed by
successive modification until the system is ready

angajaților
SCRUM – process overview
ny )
r ma
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

angajaților
Iterative models: characteristics

• each iteration contributes with additional characteristics of the system
) under
n y
development ma r
Ge
• each iteration can be tested separately rl in (
Be
b relevance, specifically H,
• regression tests and test automation are G ofm
high
where concepts like continuous delivery escontinuous deployment are used
icor
e r v
(delivery pipelines) ys
no lo g
ech
d ig t
3t re n
02
©2

angajaților
Iterative models: Test Driven Development (TDD)

• Based on: test case suites )
ny
o Prepare test cycles r ma
Ge
o Automated testing using test tools
rl in (
Be
• Development according to test cases b H,
m
sG
icefor testing
o Prepare early versions of the component
v
er
o Automatic execution of tests ys
o lo g
hnversions
o Correct defects on further
te c
ig no errors are found
o Repeat test suiteduntil
n
3 t re
• First the 2
0tests are designed, then the software is programmed
©2

angajaților
Alignment of test activities and other lifecycle activities

• Testing should be an integral part of development projects, regardless
) of
n y
the development model used ma er
G
• The model used must be understood by the Test Manager
r l in ( to effectively and
Be
efficiently align testing
b H,
m
• Model alignment: sG
v ic e
ser – Test process phases overlap the
o Sequential models (Waterfall, V-Model)
y
corresponding project phaseslog
ch
o Iterative, incrementalemodels
no
(RAD, RUP) – High level planning at project
i g t
initiation, detaileddplanning and the other phases of the test process are executed
en
3 tr
for each iteration
02
© 2Models (SCRUM, XP) – Like iterative models but with bigger overlap to
o Agile
development activities. Role of the test manager usually is more like an technical
advisor than a manager
o Spiral Models – Early testing of the prototypes to discover unsolved technical
problems with the implementation. Once the problems are solved the project
continues iterative or sequential, so the test process is aligned accordingly

angajaților
Influences of iterative / incremental lifecycle models

• Timing and extend of tasks and activities vary, e.g., the test environment is
only planned and implemented as far as needed for this iteration a ny)
m er
G
• Each iteration can be a very small test project in its own,
r l in ( including post
Be
H,
iteration review to improve the upcoming iterations
b
Gm
• Corrective measures must be included tosprevent recurring of known
i c e
problems (retrospective) er v
g ys
olo must be considered
• Regression for previous phases
hn
ec
g t monitoring can help to identify problems early
• Overall test progress
di
n
tre how alignment with the model is achieved should be
• Information about
3
captured20 2
in the test strategy
©

angajaților
Additional test levels

• Depending on: )
ny
o Project r ma
Ge
o Organization
rl in (
Be
o Product H,
G mb
additional test levels besides those of the s fundamental test process
r v ic e
might be required e
g ys
• Examples for additional test
lo
nolevels:
c h
o Hardware-software te
igintegration testing
e n d
t r
0 23
o System integration testing
2
© interaction testing
o Feature
o Customer product integration testing

angajaților
Elements that must be defined for each test level

• Test objectives, incl. achievable goals )
ny
• Test scope and test items r ma
Ge
• Test basis and means to measure its coverage rl in (
Be
• Entry and exit criteria b H,
m
• Test deliverables including reporting vice sG
r
sedesired
• Applied test techniques and how g ythe coverage is achieved with them
o ol
n
ech used
• Measurements and metrics
t
ig
• Tools
tr end
3
20 2
• Resources
©
• Responsibilities
• Compliance with standards (if applicable)
• Definition of elements must be coherent across all test levels

angajaților
Combination and reorganizing the SDLC

• Software development lifecycle models themselves may be combined.) Some
n y
examples: ma er
G
o V-Model used for the development and testing of the backend
r l in (systems and their
integrations, while an Agile development model is used
H , Btoe develop and test the
front-end user interface (UI) and functionalitym b
s G
o Prototyping used early in a project, with an
r v iceincremental development model adopted
se
once the experimental phase is ycomplete
o g
c h nol which consist of many different objects, such as
o Internet of Things (IoT) systems,
devices, products, andteservices, may apply separate SDLC’s for each object
ig
nddevelopment of Internet of Things system versions
rethe
§ challenge for
3t
© 20 2
§ Additionally, the SDLC of such objects places stronger emphasis on the later phases of the
SDLC after they have been introduced to operational use (e.g., operate, update, and
decommission phases)

angajaților
Adaption of SDLC models

A different context of project and product characteristics might need the)
ny
adaption of existing software development models. Reasons for this ma can be: er
(G
lin simple project)
• Systems might have different product risks (complex or
r
H , Be
• Multiple organizational units (e.g., business units,
G mb departments, locations)
can work on the same project or program s
r v ice (combination of sequential and
agile development) e
g ys
ol o
• Time pressure to deliverchanproduct to the market fast (merge of test levels
te types in test levels)
and/or integration ofigtest
t r end
2 0 23
©

angajaților
Summary
• Software development models are used for software development including
test activities a ny)
m r
Ge
• The best-known model is the V-model, which describes rl development levels in (
, B e
and test levels as two related branches bH
Gm
es
vicRUP, XP and SCRUM
• The most relevant iterative modes are
y ser
• Test activities are recommended g at all development levels
h n o lo
• Test activities are toigbe
c
teintegrated in the SDLC, where even the SLDC may be
n d
adjusted for the
3 trepurpose of the product being developed (incl. COTS, IoT)
0 2
©2

angajaților

ny
Ge
Be
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

2. Test levels Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Test levels
• Test levels are groupings of test activities which can be related to specific:
)
any
e rm
o goals
G
o test basis
r l in (
,B eAcceptance testing
o test object H
mb
o addressed defect (types) e sG System testing
o responsibilities se rv ic
y
lo g
Integration testing
hno
o approaches
t ec Component testing
• Objectives: n d ig
3 t re Programming
02
o Reducing risk
© 2 defects
o Finding
o Building confidence
o Showing the component shows desired functional
and non-functional behavior
o Preventing defects from escaping to higher test levels or production

angajaților
Component testing: definition

• Component (unit) testing )
ny
o test of each individual software component after its realization r ma
Ge
rl
• Because of the naming of components in different programming languages, in (
, Be
the component test may be referred to as: b H
Gm
o module test (e.g., in C) es
e rv ic
o class test (e.g., in Java or C++)
g ys
o
nol
o unit test (e.g., in Java orhPascal)
c
e
• Because of the possibled ig t involvement of developers in Acceptance testing
n
3 tre they are also called developer’s test
the test execution,
0 2
©2
System testing
Integration testing
Component testing
Programming

angajaților
Component testing
• Test basis (test cases may be derived from): )
ny
o Component specification
r ma
Ge
o (Detailed) software design
rl in (
Be
o Code
b H,
m
o Data models sG
v ic e
• Typical test objects: y ser
g
ol/omodules
o Components / classes / units
hn
ec
ig t
o Programs
d Acceptance testing
ren / migration programs
o Data conversion
3t
202 modules
o Database
©
System testing
• Typical defects: Integration testing

o Incorrect functionality (e.g., wrong or missing functions)
o Data flow problems (e.g., variables not used properly) Component testing
o Incorrect code and logic (e.g., wrong operators) Programming

angajaților
Component testing: scope

• Only single components are tested )
ny
o components may consist of several smaller units r ma
Ge
o test object often cannot be tested stand alone
rl in (
Be
• Every component is tested on its own b H,
m
sG
ic e
o finding failures caused by internal defects
s e rv
gy are not within the scope of this test
o cross effects between components
lo
no
t ech
ig
tr end
3
© 20 2

angajaților
Component testing: functional / non-functional testing

• Testing functionality )
ny
o Every function must be tested with at least one test case r ma
Ge
§ are the functions working correctly, are all specifications met?
rl in (
Be
o Defects found commonly are:
b H,
m
§ defects in processing data, often near boundaryGvalues
es
v ic
ser
§ missing functions
o gy
nol to invalid input data)
• Testing robustness (resistance
ch
e
ig t invalid inputs are called negative tests
o Test cases representing
nd
tre provides an appropriate handling of wrong inputs
o A robust system
23
20 inputs accepted in the system may produce failure in further processing (wrong output,
§ Wrong
©
system crash)
• Other non-functional attributes may be tested

o e.g., performance and stress testing, reliability

angajaților
Component testing: test harness

• Test execution of components often requires drivers and stubs )
ny
o Drivers handle the interface to the component r ma
(Ge test driver
l in
§ drivers simulate inputs, record outputs and provide a test harness
r
§ drivers use programming tools , Be
bH
o Stubs replace or simulate components not Gm
es
rv ic
yet available or not part of the test object
e
g ys
• To program drivers and / or ol o
c h nstubs you:
o must have programming t e component
d ig skills
n
e source code available
3 r
o need to have tthe
0 2
©2
o may need special tools
stub

angajaților
Component testing: methods

• The program code is available to the tester )
ny
o In case “tester = developer”: r ma
Ge
testing takes place with a strong development focus
rl in (
Be
H, and variables may be
o knowledge about functionality, component structure
b
applied to design test cases m
e sG
o often functional testing will apply
se rv ic
y
o l og and other development tools (e.g., unit test
o additionally, the use of debuggers
hn
frameworks) will allow direct access to program variables
i g te c
t r end
• Source code knowledge allows to use white-box methods for component
test 02 3
©2

angajaților
Component testing: summary

• A component is the smallest system unit specified )
ny
r
• Module, unit, class and developer’s test are used as synonyms ma
Ge
r l in (
• Drivers will execute the component functions and adjacent Be functions that are
b H,
replaced by stubs m
c e sG
i
• Component tests may check functional s erv and non-functional system
y
properties
n o lo g
t ech
ig
r e nd
t
2 0 23
©

angajaților
Integration testing (also: interface testing)

• Test basis: )
ny
o Software and system design r ma
Ge
o Architectural design
rl in (
Be
o Workflows
b H,
m
o Use cases sG
v ic e
o Interface specifications ser
lo gy
hno
o Data models
te c
d ig
• Typical test objects:
n Acceptance testing
3
o Sub-systems t re
0 2
©2
System testing
o Database
o Infrastructure Integration testing
o Interfaces / APIs
Component testing
o Microservices
Programming

angajaților
Integration testing (also: interface testing)

• Each component has already been tested for its internal functionality )
(component test). Integration tests examine the external functions a nyafter
component testing ( G e rm
n i
, B e rl
• Examines the interaction of software elements b(components) H between
different systems or hardware and software G m
(often after system testing)
i c es
er v
• Integration is the activity of combining
g y s individual software components into a
larger subsystem or in a series
h n olo of systems
i g te c
• Further integration nd subsystems is also part of
of Acceptance testing
t r e
the system2integration process
2 0 3 System testing
• ©
May be performed by developers, testers
Integration testing
or both
Component testing
Programming

angajaților
Integration testing: component / system

Component integration testing
• interactions and interfaces between integrated components man
y)
( G er
l in
• performed after component testing, and is often automated
er
B
In iterative and incremental development, component
b H, integration tests are
m
sG
usually part of the continuous integration process.
v ic e
System integration testing ser
gy
o lo
• interactions and interfaces nbetween systems, packages, and microservices
t e ch
ig
• can also cover interactions with external organizations (e.g., web services).
r e nd
• may be done t
2 0 23 after system testing or in parallel with system tests(in both
sequential
© development and iterative and incremental development).
If external interfaces are outside our scope of control various challenges for
testing arise (e.g., resolving test-blocking defects in the external software,
arranging for test environments, etc.).

angajaților
Integration testing: typical defects and failures

Examples of typical defects and failures for component integration testing:
)
any
• Incorrect data, missing data, or incorrect data encoding n (G e rm
rl i
Be
H,
• Incorrect sequencing or timing of interface calls
b
m
• Interface mismatch sG
r v ic e
• Failures in communication between e components
g ys
o
• Unhandled or improperly handled
c h nol communication failures between
e
components
d ig t
n
3 t re
• Incorrect assumptions about the meaning, units, or boundaries of the data
0 2
©2
being passed between components

angajaților
Integration testing: typical defects and failures

Examples of typical defects and failures for system integration testing:
)
any
• Inconsistent message structures between systems (G e rm
in
B e rl
H,
• Incorrect data, missing data, or incorrect data encoding
b
m
• Interface mismatch sG
v ic e
ser systems
• Failures in communication between
gy
o communication failures between
c h nol
• Unhandled or improperly handled
e
systems gt di
n
3 t re
• Incorrect assumptions about the meaning, units, or boundaries of the data
2
20 between systems
being passed
©
• Failure to comply with mandatory security regulations

angajaților
Integration testing: scope 1 of 3
• Integration tests assume that the components have already been tested
)
any
• Integration tests examine the interaction of software components
( G e rm
in
(subsystems) with each other: e rl ,B
H
mb
o interfaces with other components
G
o interfaces among GUIs / HMIs*
ic es
e rv
ys
• Integration tests examine the interfaces with the system environment
og ol
n
ech tested is that of the component and simulated
o In most cases, the interaction
t
d ig
environment behavior
en
o Under real tr
3 conditions, additional environmental factors may influence the
2 0 2
components
© behavior
* Graphical User Interfaces / Human Machine Interfaces

angajaților
• A (sub-) system, composed of individual components, will be tested )

o Each component has an interface either external and/or interactingrm a another
with
ny
G e
component within the (sub-) system n( rl i
, B e
• Test drivers (which provide the process environment
m bH of the system or sub-
system) are required G
i c es
erv of the (sub-) system
o to allow for or to produce input and soutput
g y
o to log data
h n o lo
ec
ig t
• Test drivers of thedcomponent tests may be re-used here
r e n
t
2 0 23
©

angajaților
• Monitoring tools logging data and controlling tests can support testing
activities a ny)
m r
Ge
• Stubs replace missing components rl in (
, Be
o data or functionality of a component that have not b H been integrated will be
yet
replaced by programmed stubs G m
i c es
o stubs take over the elementary tasks v
r the missing components
eof
g ys
h n o lo
i g te c
t r end
2 0 23
©

angajaților
Integration testing: approach

• Integration tests aim at finding defects in the interfaces. They check the
correct interaction of components a ny)
m er
G
r l in ( aspects, requiring
o among other reasons, in order to check performance and security
additional (non-functional) tests , Be
bH
Gm
• Replacing test drivers and stubs with realescomponents may produce new
i c
defects, such as: e rv
ys
o loosing data, wrong handlingloofgdata or wrong inputs
h no
tec interpret the input data in a different manner
o the components involved
ig
endwhere data is handed over is not correct: too early, too late, at a
o the point in rtime
t
wrong 0 23
frequency
©2

angajaților
Integration testing: strategies 1 of 3
• There are different strategies for integration testing

a ny)
rmfor example
o Common to most strategies is the incremental approach (exception
e
“Big Bang” strategy) n (G rl i
, Bestrategies
o Bottom-up and top-down are the most commonly used
H
b
s
• Choosing a strategy must also consider easpects of test efficiency Gm
v ic
er amount of test effort needed (e.g., using
o The integration strategy determinessthe
g y
olo and stubs etc.)
tools, programming test drivers
hn
o Component completion
i g tec determines for all types of integration strategies, at what
time frame the n d
e component is available. Therefore development strategy influences
t r
23 strategy
the integration
0
©2
• For each individual project, the trade-off between reducing time and
reducing test efforts must be considered:
o testing what is ready: more costs for testing but less idle time
o follow a strict integration test plan: lower costs but more idle time

angajaților
ny )
Test Test Test(Ger
ma
rl in
, Be
m bH
sG
v ic e
ser
lo gy
Bottom-Up ec hno Top-Down Big Bang
t
en d ig
3 tr
2
Integration 0 2 Integration Integration
©

angajaților
• Ad-hoc integration
ny)
e ma
o components will be integrated and tested, if possible, directly after rprogramming and
component tests have been completed n ( G
rl i
Be
• Characteristics of ad-hoc integration H,
G mb
o early start of testing activities, possibly allowing s for a shorter software development
r v ic e
process as a whole e
g ys
o depending on the type of componentol o completed, stubs as well as test drivers will be
h n
needed
i g te c
end
• Use of ad-hoctrintegration
023 that can be followed at any stage in the project
o It is a2strategy
©
o It is often used combined with other integration strategies

angajaților
Integration testing: summary

• Integration means building up groups of components )
ny
• Integration tests examine component interactions against G er specification of
the ma
in (
interfaces e rl B
b H,
• Integration takes place either bottom-up, top-down,
Gm or in a big-bang
i ces
erv of integrated components) is also a
• Integrating sub-systems (they consist
s
y
form of integration lo g
e c hno
t
• A further integrationigstrategy is ad-hoc integration
n d
3 t re
0 2
©2

angajaților
System testing: test basis and test objects

• Test basis )
ny
o System and software requirement specification r ma
Ge
o Use cases, business process
rl in (
Be
o Functional specification
b H,
m
o Risk analysis reports sG
v ic e
ser
o System, user and operation manuals
y
ol og
• Typical test objects
te chn
o Applications ig
end
Acceptance testing
o Hardware tr
3/ software systems
2 0 2 System testing
©
o Infrastructure / operating systems
o System configuration and configuration data Integration testing
o System under test (SUT)

Component testing
o End-to-end
Programming

angajaților
System testing: typical defects and failures

Examples of typical defects and failures for system testing include: )
ny
r ma
• Incorrect calculations (e.g., caused by wrong work-flow) n (Ge
e rl i
B
H,
• Incorrect or unexpected system functional or non-functional behavior
b
m
s Gsystem
• Incorrect control and/or data flows within the
ic e
erv out end-to-end functional tasks
• Failure to properly and completely scarry
o gy
nolproperly in the production environment(s)
• Failure of the system to work
ch
t e
• Failure of the system ig to work as described in system and user manuals
r e nd
t
2 0 23
©

angajaților
System testing
• The process of testing an integrated system to verify that it meets specified
requirements a ny)
m r
Ge
in (
e rl
• The system testing means the behavior of the whole system
B
b H,
• The scope is defined in the Master Test Plan
Gm(or Level Test Plan)
vi ces
seruser’s point of view
• Software quality is looked at from the
y
l og
hnoISO 25010):
• System tests refer to (as per
i g te c
end
• functional and non-functional requirements Acceptance testing
t r
0 23
(functionality, reliability, usability, efficiency,
© 2
maintainability, portability, security, compatibility)
System testing
Integration testing
Component testing
Programming

angajaților
System testing: scope

• Test of the integrated system from the user’s point of view )
ny
o Complete and correct implementation of requirements
er ma
( G
o Deployment in the real system environment with real life data
e rl in
, B
• The test environment should match the truem bH
environment
G
o No test drivers or stubs are needed i c es
e rv
y s true conditions
o All external interfaces are testedgunder
o Close representation of the no lo
later true environment
e c h
d i gt
n
• No tests in theereal-life environment!
3 tr
202defects could damage the real-life environment
o Induced
©
o Software under deployment is constantly changing. Most tests will not be
reproducible
o Test execution could impact non-functional system behavior for productive users
(e.g., execution of test cases could lead to a reduced performance appreciation)

angajaților
System testing: functional requirements 1 of 2
• Goal: to prove that the implemented functionality exposes the required

characteristics a ny)
m r
Ge
in (
e rl
• Characteristics to be tested include (as per ISO 25010):
,B
H
mb
o Suitability
G
esexpected use
§ Are the implemented functions suitable for their
er v ic
o Accuracy
g ys
olo (agreed upon) results?
§ Do the functions produce correct
e chn
o Compliance
d ig t
n
§
3 tre comply with applicable norms and regulations?
Does the system
0 2
©2
o Security
§ Are data / programs protected against unwanted access or loss?
o Interoperability
§ Does interaction with the system environment show any problems?

angajaților
System testing: functional requirements 2 of 2
Three approaches for testing functional requirements: )

ny
• Business process-based test r ma
Ge
in (
e rl
o each business process serves as basis for deriving tests
,B
o the ranking order of the business process can be applied forHprioritizing test cases
m b
• Use case-based test e sG
e r v ic
o test cases are derived from sequences ofs expected or reasonable use
y
o sequences used more frequently o l og a higher priority
receive
n
t ech
• Requirements based ig
r e nd test (building blocks)
t derived from the requirement specification
0 23
o test cases are
o the© 2
number of test cases will vary according to the type / depth of specification Requirements
based test

angajaților
System testing: non-functional requirements

• Compliance with non-functional requirements is difficult to achieve:)
a user ny
rm
o Their definition is often very vague (e.g., easy to operate, well structured
e
interface, etc.) n (G rl i
Be
H, of system description, still
o They are not stated explicitly. They are an implicit part
b
they are expected to be fulfilled m
sG
r v ic e
o Quantifying them is difficult, often non-objective metrics must be used, e.g., looks
pretty, quite safe, easy to learn y s e
n o lo g
ech documentation
• Example: testing / inspecting
t
ig
o Is documentation
r e nd of programs in line with the actual system, is it concise,
t
023and easy to understand?
complete
2
©
• Example: testing maintainability
o All programmers comply to the respective coding-standards?
o Is the system designed in a structured, modular fashion?

angajaților
System testing: summary

• System testing is performed using functional and non-functional test
) cases
m any
• Functional system testing confirms that the requirements for r
a specific
i n (G e
intended use have been fulfilled (validation) rl
, Be
H
• Non-functional system testing verifies non-functional
G mb quality attributes, e.g.,
usability, efficiency, portability etc. vice s
y ser
• Non-functional quality attributes gare often an implicit part of the requirements,
h n o lo
this makes it difficult toevalidate them
i g t c
• end
The quality of tthe
r data characteristics is very important
0 2 3
©2

angajaților
Acceptance testing
• Test basis )
ny
o User, business or system requirements r ma
Ge
o Regulations, legal contracts and standards
rl in (
Be
o Use cases and / or user stories
b H,
m
o Business processes sG
v ic e
ser procedures, security standards
o Installation manuals, disaster recovery
y
o Risk analysis reports ol og
te chn
d ig
• Typical test objects
n Acceptance testing
o Business23 t re
processes on fully integrated systems
2 0 System testing
©
o Operational and maintenance processes
o User procedures Integration testing
o Recovery systems
Component testing
o Forms and reports
o System configuration and configuration data Programming

angajaților
Acceptance testing: typical defects and failures

• System workflows do not meet business or user requirements
ny )
• Business rules are not implemented correctly or not at all r ma
Ge
in (
e rl
• System does not satisfy contractual or regulatory requirements
,B
H
G mb
• Non-functional failures such as security vulnerabilities, inadequate
performance efficiency under high loads, s
eor improper operation on a
e r v ic
supported platform ys
no lo g
ech
d ig t
3t re n
02
©2

angajaților
Acceptance testing: operational acceptance testing (OAT)

• Requires that software is fit for use in a productive environment
o Integration of software into the customer IT-infrastructure (backup-r/m a
restore-
ny)
e
n (Getc.)
systems, restart-, install and de-install-ability, disaster recovery,
rl i
, Be in use
o User management, interfacing to file and directory structures
bH
Gm data base servers, etc.)
o Compatibility with other systems (other computers,
es
v ic
ser
o Maintenance tasks
o Data load and migration taskslogy
h no
tec vulnerabilities
o Periodic checks of security
ig
o Performancere nd
testing
t
2 0 23
• Operational
© acceptance testing is often done by the customer’s system
administrators (operators)
• Also called operational readiness testing (ORT) or production acceptance
testing (PAT)

angajaților
Acceptance testing: contract and regulation acceptance

• Does the software fulfill all contractual requirements (acceptance criteria)?
)
anypayment
o With formal acceptance legal milestones are reached: begin of warranty,
m
r
milestones, maintenance agreements, etc. (G e
er contract is agreed, this
o Verifiable acceptance criteria should be defined whenBthe
l in
,
serves as a baseline for both parties bH
Gm
es
o Governmental, legal, industrial and othericregulations have to be taken into account
for acceptance testing (e.g., safety s e r v
regulation FMVSS 208: Federal Motor Vehicle
y
Safety Standards) lo g
e c hno
Acceptance testing: d ig t user acceptance testing (UAT)
n
3 t re
• Customer
0 2
selects test cases, verifies fitness for use by intended users
©2
• Create confidence that system fulfills requirements, execute business
processes with minimum difficulty, risk and / or cost
• Testing is done using the customer environment (real or simulated)
o Customer environment may cause new failures

angajaților
Acceptance testing: alpha- and beta- (or field) testing

• A stable preliminary version of the software is needed )
ny
r software)
• Mostly done for market or standard software (also called COTS* ma
Ge
r l in (
• Customers use the software to process their daily business Be processes using
b H,
the supplier’s location and environment (alpha Gmtesting) or using their own
e s
location and environment (beta testing) ic
s e rv
y
• Feed back is given on problems
n o logfound, usability, etc.
• Advantages of alpha t echbeta tests
and
ig
r endof acceptance testing
o reduce the tcost
3
20 2
o use©different user environments
o involve a high number of users
* COTS (commercial off the shelf) = software that is build as a standard product with the intention of selling it in high
numbers to a wide customer base (e.g., MS office products, games, etc.)

angajaților
Acceptance testing: summary

• Acceptance testing is the customer’s systems test )
ny
• Acceptance testing is a contractual activity, the software will r verified to
ebe ma
( G
in
comply with customers requirements e rl B
H,
mb
• Alpha and beta tests are tests performed byGpotential or existing customers
s
e (alpha) or the customers
e r v ic
either using the developer’s site/environment
site/environment (beta) ys g
h n o lo
• In iterative development, different forms of acceptance testing may be used
i g te c
at different pointsnd in time during or after an iteration or set of iterations
t r e
2 0 23
©

angajaților

ny
Ge
Be
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

3. Test types Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Test types and test levels

• Test levels )
Acceptance testing
ny
o The previous section explained the various testing levels, ma
r System testing
Ge
i.e., component test, integration test etc.
rl in (
Be
o At every test level the test objectives
H, Integration testing
have a different focus! mb
e sG
rv ic
Component testing
o Therefore, different test types are
applied during different test levelsy se
no lo g Programming
• Test types ech
i gt
nd
o Functional testing
e
3 tofr function)
(Goal: testing
© 20 2
o Non-functional testing
(Goal: testing product characteristics)
o Structural testing
(Goal: testing of SW structure / architecture)
o Change related testing
(Goal: testing changes – confirmation / regression testing)

angajaților
Testing of functional quality characteristics (functional testing)

• Goal: the functioning of the test object
a ny)
rm
o Functionality can be linked to input and output data of the test object
( Ge
lin cases
o Black-box techniques are applied to design the relevant test
er
B
H, in specifications, concepts,
o Testing is to verify functional requirements (as stated
b
Gm documents)
case studies, business rules, user stories or related
es
v ic
• Area of use ser
o gy
nol at all test levels, mainly during systems test
o Functional testing is performed
h
ec
• Execution ig t
t r end
2 0 3
o The test 2object is executed using test data derived from test cases
©results of the test execution are compared to the expected results
o The
• Comments
o May need additional knowledge and skills, depending on the domain or intent of the
system
o Coverage can be calculated to identify potential gaps (e.g., requirements coverage)
angajaților
Testing of non-functional software characteristics

• Goal: software product characteristics )
ny
o How well does the software perform its functions?
er ma
( G
e rl in
o The non-functional quality characteristics are often vague, incomplete or missing all
together, making testing difficult , B
m bH
• Area of use G
i c es
o Non-functional testing is performed sat v test levels, mainly during systems test
erall
g y
h n olo areas, please refer to following slides
o For typical non-functional testing
• Execution i g te c
r
o Compliancetwithendthe non-functional requirements is measured using selected
023test cases
functional
2
©
• Comments
o Start non-functional testing as early as possible. Project success may be
jeopardized if not (e.g., performance errors are very expensive to fix).
o Non-functional coverage can be calculated and may identify coverage gaps
(e.g., number of browsers / OS / mobile devices for compatibility coverage)

angajaților
Non-functional testing (examples for system test level)

ISO 25010: reliability, usability, efficiency, maintainability, portability, security,
compatibility (successor of ISO 9126) a ny)
m r
Ge
• Load test in (
, B e rl
o System under load (minimum load, more users / H
m b transactions)
G
• Stress test
i c es
v
o Reaction to overload / recoveryyafter ser return to “normal”
g
• Performance test h n o lo
o How fast does the i g tec perform a certain function (response time)
system
t r end
0 23
• Volume test
© 2
o Processing huge volumes of data / files
• Reliability test
o Performance while in “continuous operation mode”

angajaților
Non-functional testing (examples for system test level)

• Test for robustness )
ny
o Reaction to input of wrong or unspecified data r ma
Ge
o Reaction to hardware failures/disaster recovery rl in (
Be
• Test for usability H,
G mb
s
o Structured, understandable, easy to learncfor
r v i e user
e
• Test for security g ys
o
o Inhibit unauthorized access c h notol programs and data
e
d ig t
n quality aspects:
t re
• Other non-functional
3
02 replaceability, install-ability, conformance / compliance, adaptability
©2
o portability:
o maintainability: verifiability, stability, analyzability, changeability
o reliability: maturity, robustness, recoverability

angajaților
White box testing (structure / architecture testing)

• Goal: coverage )
ny
o Analyses the structure of the test object (white-box approach) r ma
o Testing aims at measuring how well the structure of the testlin ( is covered by the
object
Ge
r
test cases , Be H
b
• Area of use s Gm
r v ic e
o Structural testing possible on all testselevels, code coverage testing using tools
y
mainly done during component
n o logand integration testing
o Structural test designte h
iscfinalized after functional tests have been designed, aiming
i g
end
at producing a high degree of coverage
t r
• Execution2 0 23
©
o Will test the internal structure of a test object (e.g., control flow within components,
flow through a menu structure)
o Goal: all identified structural elements should be covered by test cases

angajaților
Testing related to changes 1 of 3
• Goal: test object after changes )

ny
o After a test object or its system environment r ma
Ge
has been changed, results related to the
in (
change have become invalid: tests have B e rl
to be repeated b H, correction
defect functional extension
s Gm
e
rv ic
o Two main reasons for changing software
se
y confirmation testing new test cases
lo g
§ Defect correction
§ Functional extension h no
te c
ig side effects of
r e nd
o Because of undesired
extended23ortnew functionality, it is
0 to also retest adjacent areas
©2
necessary
using regression tests regression testing
release software

angajaților
• Areas of use
a ny)
rm a
o Repeating a test of functionality that has already been verified is called
e
regression test n (G rl i
o The scope of the regression test depends on the risk,,thatBe the newly implemented
b H
m
functionality (extension, defect fix, retirement) imposes to the system
sG
ice analysis (see Ch.II.4)
o Analyzing this risk can be done with anrvimpact
se
gy
o Change related testing may beoperformed at all test levels
ol
chnare:
o Typical tests after changes
te
dig after correction of errors, also called confirmation testing)
§ re-testing (= testing
t re n
23 testing (= testing to uncover newly introduced defects)
§ regression
20
© new features, changes and refactoring result in frequent changes to
• In agile,
the code. Thus, the importance of confirmation and regression testing grows
• Particularly relevant for Internet of Things systems (IoT) where individual
objects (e.g., devices) are frequently updated or replaced

angajaților
• Execution
a ny)
rm
o Basically, execution takes place as in previously executed test iterations
( Ge
rlinbecause it is too
o In most cases, a complete regression test is not feasible,
e
expensive and takes too much time ,B
bH
o A high degree of modularity in the software and
s Gmin the regression test set allows for
e
v ic
more appropriate reduced regression rtests
y test cases: se
og
o Criteria for the selection of regression
ol
chn
§ Test cases with high priority
te
digfunctionality, skip special cases and variations
§ Only test standard
en
tr
§ Only test3 configuration that is used most often
§ ©
Only
02
2 test subsystems / selected areas of the test object
o If during early project phases, it becomes obvious that certain tests are suitable for
regression testing, test automation should be considered

angajaților
Summary
• On different test levels different types of tests are used )
ny
• Test types are functional, non-functional, structural and G er
change related ma
in (
testing e rl B
b H,
• Functional testing examines the input / output
Gm behavior of a test object
vi ces
ser characteristics
• Non-functional testing checks product
y
og
c h nol but is not limited to, load testing, stress
• Non-functional testing includes,
e
testing, performanceig t testing, robustness testing
t r end
0 23
• Common white box (structural) tests are tests that check data and control
© 2
flow within the test object, measuring the degree of coverage
• Important tests after changes are re-tests (confirmation tests) and
regression tests

angajaților

ny
Ge
Be
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

4. Maintenance testing Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Testing after product acceptance 1 of 4
• Customer has approved the product and sets it into production

ny)
ma
o The initial development cycle, including its related tests, has been rcompleted
i n (G e
• The software itself is at the beginning of its life cycle:er l
,B
o It will be used for many years to come, it will be m bH
extended
s Gbe further modified and corrected
o It will most likely still have defects, henceicitewill
er v
o It needs to adapt to new conditions g y sand to be integrated into new environments
o
olthe
o It needs to change or extend h n configuration data
t ec
o It needs to preserve ig or even improve its non-functional behavior
r e nd
t be retired, put out of operation
o It will one23day
2 0
• Any new© version of the product, any new update and any other change
in the software requires additional testing!

angajaților
• Configuration
a ny)
rm nature, and
o The composition of a component or system as defined by the number,
e
interconnections of its constituent parts n( G
rl i
• Impact analysis H , Be
G mb
s
o The assessment of change to the layers ofedeployment documentation, test
r v ic
y se
documentation and components, to implement a given change to specified
requirements g
h n o lo
• Maintenance testingg tec
i
r e nd to an operational system or the impact of a changed
o Testing the changes
t
023 to an operational system
environment
2
©

angajaților
• Software maintenance covers two different fields:

a ny)
e rm
o maintenance as such: correction of defects or implementation of hot-fixes, that
already were part of the initial version of the software n ( G
rl i
Be
H, characteristics of the
o Maintenance to preserve / improve non-functional quality
b
s Gm
component or system, especially performance efficiency, compatibility, reliability,
security, compatibility, and portability ic e
e rv
ys
o planned software releases: adaptations
g
defect correction functional extension
o
c h nol
as a result of a changed environment or
te
new customer requirements
ig confirmation testing new test cases
end
• Test scope 3oftrmaintenance testing
02
2 and defect correction require retests
o Hot©fixes
o Extended functionality requires new test cases
regression testing
o Migration to another platform requires operational tests
o In addition, intensive regression testing release software
is needed

angajaților
• Scope of testing is affected by the impact of the change

ny)
maamount of
o Impact analysis is used to determine the affected areas to deciderthe
e
regression testing, difficult if: n( G
rl i
Betraceability is not in place
§ Specifications and / or test cases are missing or outdated,,and
§ People lacking domain and / or system knowledge m bH
sG
ce a lack of tool support
viand
§ Maintainability of the software is neglected
er
s
l o gy
o Problems might occur, if documentation of the old software is missing or
incomplete, either manual hn o
or tool based. Furthermore, if other system, domain or
t e c
other knowledge isglacking or if system maintenance or traceability are outdated
i
r e nd
t
2 23
o IoT maintenance
0integration
specific
may be triggered by completely new hardware devices, requiring
testing and security related testing
©
• Software retirement, tests may include (examples):
o Data migration tests (e.g., copy customer data to new data base)
o Verifying archiving and retrieving / restoring data and programs (archive old system)
o Parallel testing of old and new systems (old sys used as test oracle for new system)

angajaților
Summary
• Ready developed software needs to be adapted to new conditions; )
ny
errors must be corrected ma r
Ge
in (
erl risks
• An impact analysis can help to judge the change related
,B
H
• Maintenance tests make sure, that mb
e sG
rvic(new test cases)
o New functions are implemented correctly
se
y
og (old test cases)
o Errors have been fixed successfully
ol
chn been verified, is not affected (regression test)
o Functionality, that has already
e
d ig t
n
• If software gets
3 treretired, migration tests or parallel tests may be necessary
0 2
©2

angajaților
Keywords
• non-functional testing
• coverage
n )
• operational acceptance ytesting
• acceptance testing • regression testingerm
a
G
• alpha testing l in (
• regulatory racceptance testing
• beta testing , B e
m bH
• sequential development model
• change-related testing G
•
es• system integration testing
commercial off-the-shelf (COTS) ervic • system testing
s
• component integration testinglogy • test basis
• component testing c h no
e • test case
• confirmation testing d ig t
n • test environment
• 3 t re
contractual2acceptance testing
0 • test level
• © 2 testing
functional • test object
• impact analysis • test objective
• integration testing • test type
• maintenance testing • user acceptance testing
• non-functional testing • white-box testing

angajaților
ny )
r ma
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

III. Static testing »
angajaților

• III/1 Static testing basics )
ny
• III/2 Review process r ma
Ge
rl in (
Be
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

angajaților
Learning objectives for static testing:

3.1 Static testing basics
a ny)
FL-3.1.1 (K1) rm
Recognize types of software work product that can be examined
e
by the
different static testing techniques n ( G
e rl i
FL-3.1.2 (K2) , Btesting
Use examples to describe the value of static
m bH
FL-3.1.3 (K2) Explain the difference between static
c e s G and dynamic techniques,
considering objectives, types
s e rv i of defects to be identified, and the role of
y the software lifecycle
these techniques within
n o lo g
3.2 Review Process ch
e
FL-3.2.1 (K2) Summarize
d ig t the activities of the work product review process
n
t re
FL-3.2.2 (K1) 23 Recognize the different roles and responsibilities in a formal review
2 0
FL-3.2.3©(K2) Explain the differences between different review types: informal review,
walkthrough, technical review, and inspection
FL-3.2.4 (K3) Apply a review technique to a work product to find defects
FL-3.2.5 (K2) Explain the factors that contribute to a successful review

angajaților

ny
Ge
rl in (
Be
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

1. Static testing basics Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Basics
• Static testing summarizes various methods, that do not execute the)
ny
component or the system that is being tested ma r
Ge
• Static testing includes: rl in (
Be
H,
mb
o reviews (manual activity)
s G
o static analysis (mostly a tool-based activity)
r v ic e
e
• Static testing complement dynamic g y smethods
l o
o static testing finds causesc h nofofailures (defects) rather than failures
t e
ig as well, not only executable code
r e nd
o artifacts are inspected
3t
o Defects0/2anomalies are found in an early phase, before they are implemented in
©
code
2
o Static testing might find defects not found in dynamic testing
• Assumption: high quality documents lead to high quality products

o even if the reviewed specifications do not contain any errors, interpreting the
specification and creating the design could be faulty
angajaților
Review objectives
• Reviews are done in order to improve product quality )
ny
o Reviews are used to verify the correct transition r ma
Ge
from one phase to the next phase, as defined
in (
rlRequirements
in the left half of the V-model Be
b H, Definition
m
• Detecting defects early saves costs
e sG
e r v ic Functional System
• During reviews, the following defects s might Design
l o gy
hno
be detected:
e c Technical System
d ig t
o defects in the specifications Design
n
3 tredesign and architecture
o defects in the Component
0 2
© 2 in the interface specifications
o defects Specification
o insufficient maintainability (e.g., comments)

Coding
o deviations from agreed standards
(e.g., programming guidelines)
o Defects in test artefacts

angajaților
Static analysis
Static analysis is becoming more important: )
ny
• essential for safety-relevant (safety-critical) systems r ma
Ge
o e.g., aviation, medical or nuclear software rl in (
Be
• as an important part of security testing b H,
m
e delivery system
• as an integrated part of automated buildicand sG
e rv
y s delivery and continuous deployment
o e.g., in agile development, continuous
n o lo g
Comments for staticteanalysis ch
e d ig
nbe
• Artefacts should
3t r understandable and readable for the participating
0 2
©2
reviewers
• Static analysis needs a work product with a formal structure (typically code
or models) for which the tool is suitable
• Static analysis with tools can even be applied on work products in natural
language (e.g., checking for spelling, grammar, and readability)

angajaților
Work products for evaluation by static testing

• Development artefacts:
a ny)
rm
o Specifications, including business, functional and security requirements
( Ge
o Epics, features, user stories and acceptance criteria
rl in
o Models (e.g., activity diagrams) H , Be
mb
o Architecture and design specifications e sG
o Code se rv ic
y
lo g
hno
• Test artefacts:
e c
t plans, test cases, test procedures, automated test scripts
o Testware, includingigtest
n d
treand infrastructure setup
o Configuration
3
20 2
• Other©artefacts:
o User guides
o Web pages
o Contracts, project plans, schedules and budgets

angajaților
Advantages of static testing

• Lower costs and a relatively high savings potential
a ny)
rmdynamic test
• Detecting and correcting defects more efficiently, and prioreto
(G
execution l in er
B
H,
• Identifying defects which are not easily found bybdynamic testing
G m
• Preventing defects in design or codingcby
i es uncovering inconsistencies,
ambiguities, contradictions, omissions, v
er inaccuracies, and redundancies in
g ys
requirements o lo hn
• Increasing development
i g tecproductivity (e.g., due to improved design, more
maintainable code)nd
t re 3
• Reducing 2
20development and testing cost and time
©
• Reducing total cost of quality over the software’s lifetime, due to fewer
failures later in the lifecycle or after delivery into operation
• Improving communication between team members in the course of
participating in reviews

angajaților
Distinction between static and dynamic testing

• static testing finds defects in work products directly rather than identifying
failures caused by defects when the software is run. Static testing a ny) be
may
able to find the defect with much less effort ( G e rm
n rl i
Be and internal quality of
H,
• static testing can be used to improve the consistency
b
artefacts; dynamic testing typically focuses G
onmexternally visible behaviors
ic es
ervfind / fix through static testing include:
• typical defects, easier and cheaper to
ys
o requirement defects og
h nol
o design defects ec
d ig t
re n
o coding defects
3t
202 from standards
o deviations
©
o incorrect interface specifications
o security vulnerabilities
o gaps or inaccuracies in test basis traceability or coverage
o most types of maintainability defects can only be found by static testing

angajaților
Advantages and drawbacks of reviews

• Advantages )
ny
o Lower costs and a relatively high savings potential r ma
(G e
l in
o Defects in documentation are detected and corrected early
er
o High quality documents improve the development H ,B
process
G mb
es of know-how
o Increased rate of communication / exchange
er v ic
• Drawbacks ys
lo g
hno is confronted directly
o Stress may arise if the author
o Experts involveddin
te c
igreviews need to attain specific product knowledge, good
r e n
0 3t
preparation is
2
necessary
©2
o Considerable time investment (10%–15% of the overall budget)
o Moderator / participants influence review quality directly

angajaților
Advantages and drawbacks of static analysis

• Advantages
a ny)
rm
o Control and data flow anomalies can be detected and corrected early
( Ge
l in
o Compliance with programming rules can be checked by tools
er
o Detect problematic design by calculation of metrics H , Bcyclomatic complexity
like
b
o Detect inconsistencies and dependencies es Gm
rv ic
o Check the maintainability of code orse
design
lo gy
hno
• Drawbacks
e c
ig t documents can be inspected (e.g., source code, scripts)
o Only formal language
e nd
3 tr work is needed to prepare code for analysis
o Often additional
2
0 in tools is necessary (purchase, training, analysis of results)
©2
o Investment
Tools can perform some tasks faster and more reliable than people
(e.g., detect non-compliance with programming guidelines)

angajaților

ny
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

2. Review process Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Review process 1 of 3
• Process, roles review techniques are described in ISO 20246 )

ny
• Degree of formality er ma
n ( G
o Formal reviews: team participation, documentation output,
e rlifollowing a defined
process, adhere to standards, rules and regulations,Hoften, B mandatory reviews
b
o Informal reviews: not following a defined process, s Gm not having a formal
ic e
e rv
documentation output, often optional reviews
y s
o l og by the software lifecycle model, maturity of
o The degree of formality is influenced
organization, complexitych ofnproduct, applicable rules and regulations
t e
ig
• Focus
r e nd
t
2 0 3
o Possible2agreed objectives of reviews: finding defects, gaining understanding,
© participants, discussing technical alternatives, deciding by consensus
training
o Focus can have an influence on the type of review to choose
o One document can be inspected with different types of reviews

angajaților
Review
Planning Management
Review Leader
ny )
r ma
Ge
l in (
Initiate review All er
, BInvolved
b H
Gm
es
v ic
ser
Reviewers (1–3)
g y
o lo
evt. repeating (Review Leader)
Individual review
e chn Fixing and reporting
ig t
e nd
02 3 tr Author
© 2
Facilitator
Scribe
Communication
Author and analysis
Reviewer (review meeting)

angajaților
Activities of a formal review 1 of 3
Planning
a ny)
• Defining review criteria (checklists, type of review, scope, criteria
( G erm to check)
• Selecting the personnel (reviewers, moderator, …) erlin
,B
• Allocating roles and time in project schedules m bH is doing what)
(who
G
• Estimating effort and time i c es
v
y setor review (depending on the importance
• Selecting which parts of documents g
or complexity) h n o lo
i g te c
end
• Defining the entry and exit criteria for formal review types, check that entry
t r
0 3
criteria are2met
2
Initiate©review
• Distributing documents (to the reviewers), including checklists, log forms
• Explaining the objectives, roles, process and documents (checklists)
• Answer questions that might have come up

angajaților
Individual review (i.e., individual preparation)

a ny)
• Reviewers inspect all or part of the review item (document)
( G e rm
• Reviewers note potential defects and items in need rl in
, Be of clarification
H
mb green
• Findings have different severity, e.g., red, Gyellow,
s
Issue communication and analysis r v ic e
e
ys
og
• Communication of potentialolproblems (e.g., during review meeting)
h n
• Analyze potential defects
i g tec and make decision on repair, ownership and
status t r end
2 23
0quality
• Evaluate
© of the document
• Check that exit criteria have been reached
• Make review decision (reject, accept, accept with minor changes, etc.)

angajaților
Fixing and reporting

a ny)
• Create defect reports for major findings resp. where changeseare rm needed
( G
• Author fixes defects, communicates potential problems e lin during review
r(e.g.,
, B
meeting) to author
m bH
G
• Analyze potential defects and make decision i c es on repair, ownership and status
rv
• Record the status of the defect gy se
• Check that exit criteria are h o lo
nfulfilled
e c
• Gather metrics for d ig t reviews
formal
en
trper
02 3
o e.g., ,errors page, pages per hour
© 2
o defect density (e.g., errors per hour)
o influence of review type, number of reviewers

angajaților
Roles and responsibilities 1 of 2
• Management )
ny
o initiates and plans the review r ma
Ge
o decides on participants
rl in (
Be
o allocates time and budget in project schedules
b H,
m
o monitors ongoing cost effectiveness sG
ic e
e rv
o executes control activities if review soutcome insufficient
y
ol og
• Review Leader
te chn
d ig
o has the overall responsibility
en
for the review
o decides 2 3 trtakes part in the review, organizes when and where it will take place
who
© 20
• Facilitator (also: moderator)
o leads the meeting/the discussion, guards the process, summarizes
o mediates between different viewpoint
o has the biggest influence on the success of the review

angajaților
Roles and responsibilities 2 of 2
• Author
a ny)
rm
o person with chief responsibility for the review-object, often the writer
(G e
l in
o performs changes, fixes defects in the product under review
r
Be
• Reviewers (also: inspectors or checkers) H,
G mb
o stakeholders, experts, individuals with a specific s technical or business background
r v ic e
e
y s problem areas, etc.
o discover defects, deviations, potential
g
o
c h nol (tester, user, programmer, analyst, etc.)
o represent different perspectives
e
• Scribe (also: recorder)
d ig t
n
o documents 3 treissues, problems and open points identified during the meeting
all
o for© 202 reviews a formal protocol might be prepared
important
o scribe might not be needed of documentation tools are used
Based on the type of review, one person may take on multiple roles and
activities. Different types of reviews are looking at software products or related
work products from different perspectives. More: ISO 20246

angajaților
Checklist for reviews 3 of 3
• Checklists can make reviews more efficient. A checklist with the typical
problems may help to uncover previously undetected issues man
y)
(G er
l in
• Checklist – example items (depending on the review-object)
er
B
o Identify target audience, use change history
b H,
m
o Create website to promote the software
c e sG
i
o Define minimum system requirements
s e rv
y
o Optimize website for search engines
n o log (SEO)
ech
o Set up order pages on your site
o Create key benefits
t
iglist
r e nd
o The terms3aret correct and unique
2 0 2
o The©requirements are complete and unique
o The comments are “speaking” and in the expected language
o All variables in the coding begins with a “V”
o All constants in the coding will be named in capitals
o The interfaces are consistent and in the same metric
o…
angajaților
Types of reviews (ISO 20246) 1 of 10
• The outlined basic review process covers the following variants of reviews:
)
any
e rm
o Inspection, walkthrough, technical review, informal review
( G
lin base practice
o These variants differ in a few aspects from the general outlined
er
B risk, business domain,
H,
o Selected review type depends on project needs, resources,
b
Gm among others
company culture, applicable rules and regulations,
s
r v ic e
• A further distinction of reviews is ymade se depending on the nature of the
g
reviewed object: product ornprocess
h o lo
te c
o SW-developmentig process or project process
e n d
§ CMMi, ISOtr12207, TPI are terms relating to process improvement
2 3
20called management reviews, assessments or audits these reviews do not directly
§ Also
©
interfere with the testing process, they are not part of this lecture
o Documents / products of the development process
§ These reviews are addressed here

angajaților
• Inspection: key characteristics

a (e.g., ny)
rm
o Reviewers inspect the object under review using checklists and metrics
e
problems per page) n (G rl i
, Be
o A trained, independent moderator is leading the review
bH
Gm
o Review-ability of the object is assessed prior to the review
ic es
e rv
o Specified entry and exit criteria for acceptance
s
of the software product
y
o Formal process based on rules
n o logand checklists for preparation, execution,
chup activities
documentation and follow
te
ig
nd as a peer examination
o Usually conducted
t re
023 preparation
o Pre-meeting
2
©
o Inspection report including list of findings

angajaților
• Inspection: advantages and drawbacks )

ny
o Well organized formal session with clear roles r ma
Ge
o Needs intensive preparation and follow up activities
rl in (
Be
o Facilitator (moderator) and scribe are necessary H,
G mb
es method
o Main purpose: finding defects using a structured
er v ic
ys
no lo g
ech
d ig t
3t re n
02
©2

angajaților
• Walkthrough: key characteristics )

ny
o There is an optional pre-meeting preparation of the reviewers r ma
Ge
o Open-ended sessions in (
, B e rl
o Make take the form of scenarios, dry runs, peer group
m bH participation
G
o Meeting led by author
i c es
v
o During presentation by the author, the
y serreviewers try to locate deviations and/or
g
problem areas
h n o lo
i g te c
o Examples for using walkthroughs:
§ Walkthrougheofnd
documents
t r
023 of drafts for user interfaces
§ Walkthrough
2
§ ©
Walkthrough of business process modeling data

angajaților
• Walkthrough: advantages and drawbacks

a ny)
rm
o Little effort in preparing a review session, but it is an open-end session
( Ge
o Session can be initiated on short term notice
e rl in
o Author has a great influence on the outcome: since H , B moderates the review,
she/he
b
m points are not addressed in
there is a danger of domination by the author G (critical
e s
depth) rv ic se
y
og is also in charge of any follow-up activities
o Little control possible, since author
ol
hn
te c
en d ig
02 3 tr
©2

angajaților
• Technical review: key characteristics

a ny)
rm
o Target of examination is a technical aspect of the object under review:
e
n (GNew ideas?
Is it fit for use? Are there alternative implementations possible?
rl i
, Beexternal) experts
o Reviewers either technical peers of author or (preferably
bH
Gm
o May be performed as a peer review without management participation
ic es
ervauthor
o Ideally led by trained facilitator, not the
s
y
lo g
o Pre-meeting preparation by reviewers, usage of checklists (optional)
no
ech
o A unanimous vote of trecommendation is given by the expert panel
ig
o Preparation of
t r eandreview report which includes the list of findings, the verdict whether
023 product meets its requirements
the software
2
©technical review may be very formal or informal depending if the importance
o The

angajaților
• Technical review: main purposes )

ny
o Discussion during the review r ma
Ge
o Making decisions, gaining consensus
rl in (
Be
o Evaluating alternatives
b H,
m
o Finding defects sG
v ic e
o Solving technical problems ser
gy
o lo
o Checking conformance (standards, plans, specifications, regulations)
hn
ec
ig t
e nd
02 3 tr
©2

angajaților
• Informal review: key characteristics )

ny
o Simplest form of reviews r ma
Ge
o Often initiated by the author
rl in (
Be
o Only reviewers (one or more) will be involved
b H,
m
o No separate meeting necessary sG
ic e
ervaction list
o Results may be recorded in form ofsan
gy
olothat a colleague is asked to review a document
o Often performed in such a way
e chn
g t buddy check, pair review, pairing
o Also called: peer review,
di
en
02 3 tr
© 2

angajaților
• Informal review: advantages and drawbacks )

ny
o Easy to perform, even on short term notice r ma
Ge
o Cost effective
rl in (
Be
o No protocol needed
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

angajaților
• Select type of review to use

o One document may be reviewed with different types of reviews,rm a an
e.g.,
ny)
e
n (G by other people
inspection is performed by some experts, but also informal reviews
rl i
Be before an inspection
H,
o The order may vary: an informal review might be performed
b
s Gm
(to ensure there are no big issues in the formal review) or after an inspection (to
incorporate different viewpoints from further
ic e people)
se rv
o All reviews are best done as peeryreviews, i.e., author and reviewer come from the
o g
same level of hierarchy nol
e ch
g t are found depend on the type of document: e.g., design or
o Types of defects ithat
d
en
specification rproblems, coding defects, non-compliance with programming rules
02 3t
© 2

angajaților
Applying review techniques 1 of 2
Different techniques can be applied by the reviewers when they inspect ) the
documents. These techniques can be applied to all types of reviews, n y
a although
e rm
( G
some might be more effective than others for certain reviewn types.
e rl i
B
• Ad hoc: no or little guidance for the reviewers,bthey H, work based on their
experience. They just read through the document G m and report their findings as
i c es
v needed
ser
they are encountered. Little preparation
g y
• Checklist based: Systematic
h n olomethod, where inspection is done with help of a
c beginning of the review. Allows for a systematic
checklist distributed gattethe
i
t r end defect types, de-personalizes the review and lists prior
coverage of specific
problems2 23 potential defects. But: also look outside the checklist
0and
©
• Scenarios and dry runs: Uses guidelines for reading the documents,
suggest dry runs on scenarios (e.g., use cases) based on the expected use of
the product. Better guidance than checklists. Also look outside the scenarios

angajaților
Applying review techniques 2 of 2
ny)
• Perspective based: Similar to a role based review approach. If multiple
a
reviewers inspect the same document, they might take on different
( G erm roles. This
way the variety of stakeholder viewpoints lead to more depth
e rlin of the review
B
and the risk of duplicate findings is reduced. Further
b H, to reading the document,
reviewers would also attempt to use the product m to perform their role
c e sG
activities. A tester might try to derive test
s e rv i cases from a specification, or a
programmer might try to develop
l o gycode from an interface design document.
Special perspective-based o
hnchecklists might be used
i g te c
t r end
• Role based: Evaluation is done from the perspective of a certain stakeholder
023roles can be: user administrator, system administrator,
role. Typical
2
© tester, performance tester, usability tester, marketing, designer.
functional
Specifically defined personas might also serve as a role
Perspective based reading has been shown to be the most effective method for
reviewing requirements and other work products.

angajaților
Success factors of reviews - organizational 1 of 2
• Reviews are to be performed goal oriented, i.e., define objectives in )planning

ny
and use metrics to measure their achievement ma er
G
r l in (
• Systematic usage of the introduced techniques and templates - all review
B e
techniques are suitable for finding defects H, b
m and the level of reviewers
sG
• Pick a review type appropriate to the document
v ic e
ser be stated in an unbiased manner. Use
• Deviation in the reviewed object should
y
g
o lo
positive communication (I-statements)
hn
to not demotivate the author
tec will improve the efficiency of a review
• Using check lists orgroles
di
en budget is needed to perform proper reviews (10% to 15%
• Sufficient timetrand
3
202 development cost)
of the overall
©
• Make use of the lessons learned effect, use feed back to implement a
continuous improvement process
• Training is provided where needed, a culture of learning is promoted
• Management supports a good review process (e.g., allow enough time)

angajaților
Success factors of reviews – people-relatedf 2 2 of 2
• Reviews are planned, meetings are well-managed and seen as valuable

)
any
• The review is conducted in an atmosphere of trust. The outcome
( G erm will not be
in
used for the evaluation of the participants e rl B
b H,
• The right people for the review objectives are
Gminvolved (different roles)
vi ces
ser necessary care and intensity
• Reviewers use enough time and apply
y
l og
hntoo allow better concentration and early feedback
• Break-down review work
i g te c
end
• Testers are valued reviewers who contribute to the review and learn about
t r
0 23
the product, which enables them to prepare tests better and earlier
© 2
• Give feedback in a positive manner (e.g., “your document will be better still”
instead of “your document is of poor quality”)
• There is an emphasis on learning and process improvement, defects found
are acknowledged and appreciated

angajaților
Overview of the review types
review type advantages disadvantages a ny)

( G e rm
inspection structured meeting long in
rlpreparation
B e
protocol b H,moderator, scribe
m
e sG
walkthrough short preparation
e r v ic no protocol
s
gy
long duration
o author determined
c h nol
technical review
ig te
unanimous vote long preparation
d
t re n
0 23
informal review short preparation no protocol
©2 low cost
no meeting

angajaților
Summary
• During static testing, the test object is not executed )
ny
er
• Reviews can take place during the early phases of the development process, ma
( G
in
e rl
they complement/extend the methods of dynamic testing
,B
H
• Phases of a review: mb
e sG
rv ic
o Plan – initiate – individual review – communicate and analyze – fix and report
e
g ys
o lo
• Roles and tasks for the review:
hn
te–c facilitator – leader – reviewer – scribe
o Author – management
ig
r e nd
t
• Types of 2reviews:
2 0 3
©
o Inspection – walkthrough – technical review – informal review
• Techniques:
o Ad hoc – checklists – scenarios and dry runs – role - perspective

angajaților
Keywords
• technical review
• ad hoc review )
• walkthrough ny
• checklist-based review r ma
Ge
• dynamic testing rl in (
Be
• formal review b H,
m
sG
• informal review v ic e
ser
• inspection
lo gy
• perspective-based reading
ec hno
d ig t
• review
3 t re n
202review
• role-based
©
• scenario-based review
• static analysis
• static testing

angajaților
ny )
r ma
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

IV. Test techniques »
angajaților
Chapter IV. Test techniques

• IV/1 Categories of test techniques )
ny
• IV/2 Black-box test techniques r ma
Ge
• IV/3 White-box test techniques rl in (
Be
• IV/4 Experience-based test techniques b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

angajaților
Learning objectives for test techniques: 1 of 2
4.1 Categories of test techniques

a ny)
• FL-4.1.1 (K2) rm between
Explain the characteristics, commonalities, and differences
e
G and experience-
r l in (
black-box test techniques, white-box test techniques,
based test techniques Be ,
H
4.2 Black-box test techniques mb
sG
• FL-4.2.1 (K3) v ice to derive test cases from given
Apply equivalence partitioning
r
se
requirements gy lo
o
• FL-4.2.2 (K3) chn value analysis to derive test cases from given
Apply boundary
t e
ig
requirements
nd
• FL-4.2.3 (K3)3 treApply decision table testing to derive test cases from given
2 02 requirements
©
• FL-4.2.4 (K3) Apply state transition testing to derive test cases from given
requirements
• FL-4.2.5 (K2) Explain how to derive test cases from a use case

angajaților
Learning objectives for test techniques: 2 of 2
4.3 White-box test techniques )

ny
• FL-4.3.1 (K2) Explain statement coverage r ma
Ge
• FL-4.3.2 (K2) Explain decision coverage in (
B coverage e rl
• FL-4.3.3 (K2) H,
Explain the value of statement and decision
b
4.4 Experience-based test techniques s Gm
ic e
• FL-4.4.1 (K2) Explain error guessing s e rv
y
• FL-4.4.2 (K2) o
Explain exploratoryl og testing
t chn
echecklist-based
• FL-4.4.3 (K2) Explain
ig testing
n d
3 t re
0 2
©2

angajaților

ny
Ge
Be
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

1. Categories of test techniques Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Criteria for choosing the appropriate test techniques 1 of 3
• State of information about the test object )

ny
o Can white-box tests be made at all (source code available)?
er ma
( G
o Is there sufficient specification material to define black-box
e rlintests, or are
explorative tests needed to start with? , B
m bH
G
• Predominant test goals
i c es
v
ser
o Are functional tests explicitly requested?
y
o Which non-functional test are og
olneeded?
h n
tec to attain the test goals?
o Are structural testsigneeded
t r end
0 23
• Risk aspects
2
©
o Is serious damage expected from hidden defects?
o How high is the sequence of usage for the test object?
o Are there contractual or legal standards on test execution and test coverage that
have to be met?

angajaților
• Project preconditions )
ny
o How much time is planned for testing? r ma
Ge
o Who is performing the tests? in (
, B e rl
o How high is the risk, that testing might not be completed
m bH as planed?
G
o Which software development methods are
i c esused?
o What are the weak points of the project v
er process?
g ys
• Characteristics of the test h n o lo
object
c
tetesting
o What possibilities i gfor does the test object offer?
r e nd
tavailability of the test object?
0 23
o What is the
© 2
• Contractual and client requirements
o Were there any specific agreements made with the client/originator of the project
about the test procedures?
o What documents are to be handed over at the time of deployment of the system?

angajaților
• Best practice
a ny)
rm
o Which approaches have proven to be appropriate on similar structures?
o What experience was gained with which approaches in thelin (
past?
Ge
r
Be
o Always use a combination of test techniques b H,
m
sG
• Test levels v ic e
y ser
ogbe done?
o At which test levels should tests
• Further criteria shouldte
be
nol
chapplied depending on the specific situation!
ig
nd
• The use of testretechniques in other testing activities (analysis, design,
3 t
© 20 2
implementation activities) can range from very informal to very formal. The
appropriate level depends on the context of testing, e.g.:
o the maturity of test and development processes
o development lifecycle model / time constraints
o safety or regulatory requirements
o the knowledge and skills of the people involved, and the software

angajaților
Different interests result in different test design approaches

• Interest of the project manager:
a ny)
rmrestrictions
o To create software of ordered quality while meeting time and budget
in (G e
e rl
• Interests of the client/initiator of the project
B
b H,
o To receive software of best quality (functionality, reliability, usability, efficiency,
m
G and budget restrictions
portability and maintainability) while meetingstime
e
v ic
• Interests of the Test Manager:
y ser
n o lo g
o Sufficient and intensive testing/adequate deployment of the required techniques,
e
from the testing pointtofc h
view
ig
nd level that the project has reached
o To assess theequality
3 tr
202 and use the resources planed for tests in an optimal way
o To allocate
©
Example: If the client has ordered the new system because the old system was
too slow, what would be the most important test from a client’s point of view?
Most likely the performance test!

angajaților
Black-box, white-box and experience-based testing

• Dynamic testing is divided in three categories / groups )
ny
• The grouping is done on the erpartitioning ma
Equivalence
( G
bases of the method to derive in value analysis
e rl
Black-Box
Boundary
Quality Control (Analytical QA)

BState
test cases
b H, transition testing
Gm Decision tables
es
v ic Use case based testing
ser
dynamic
Black-Box White-Box
lo gy Experience-based techniques
ec hno
t
d ig Statement coverage
White-Box
n
tre its own set of
• Every group3has Decision coverage
methods20 2 Condition coverage
© for designing Path coverage
test cases
Reviews/walkthroughs
P
static
Data flow analysis

angajaților
Black-box, behavioral or specification-based techniques

• The tester looks at the test object as a black box )
ny
o Internal structure of the test object is irrelevant or unknown r ma
Ge
• Test cases are derived / selected based on specificationrl analysis (functional in (
, Be
and non-functional) of a component or system bH
Gm
o Testing of input / output behavior es
e rv ic
ys
• Functionality is the focus of attention!
og ol
n
ech called functional testing, behavior-based or
o Black box technique is also
t
dig testing
specification-based
n
3 t re
2
0 case design
2test test object
©test cases on the basis
of specifications
internal program structure black-box
irrelevant
test of all selected resp. chosen
combinations of input / output data

angajaților
White-box or structure-based techniques

• The tester knows the internal structure of the program/code )
ny
o i.e., component hierarchy, control flow, data flow, etc. r ma
Ge
• Test cases are selected on the basis of internal program rl code/program in (
, Be
structure bH
Gm
ces
o During testing, interference with the test execution
i is possible
s e rv
• Structure of the program is theofocus
l gy of attention!
n o
o White-box techniques are
t echalso called structural testing or
control flow-based ig testing
r e nd
t
2 023case design
test
©test cases on the basis
of program structure
test process controlled
externally
analyze control flow within the test
object during test execution

angajaților
Experience-based techniques
• The experience and knowledge of the tester is the primary test basis,) which
ny
can be extended by the following items: ma r
Ge
o intuition and skills
rl in (
Be
o defect taxonomies and defect listings b H,
m
o checklists e sG
v ic
y ser
o competing or old systems
ol og
te
o any other means of information chn
ig
r e nd
t
• Black- and white-box techniques may be used during experience-based
testing, 2 023testing input fields with boundary values
e.g.,
©
test case design
test cases on the basis
of experience

angajaților
Categories of test techniques – overview

• Black-box techniques
a model ny)
rm
o Test items have been selected in accordance with the functional software
( Ge
lin identified
o Gaps or deviations between test basis and test object can be
er
B
H, percentage of specification
o The coverage of test basis can be measured (e.g., which
b
covered by test cases) Gm
es
v ic
• White-box techniques ser
gy
lo object is used to design test cases (code /
o The internal structure of the
c h notest
e etc.)
ig t
statements, menus, calls,
o The coverage n d
percentage is measured and used as a bases for creating additional
3 t re
02
test cases
©2
• Experience-based techniques
o Knowledge and experience about the test object and its environment are the bases
for designing test cases
o Knowledge and experience about possible weak spots, probable errors and former
errors are used to determine test cases

angajaților
Summary
Test cases can be designed using different techniques )
ny
• If specified functionality is the focus of testing, the methods r
eused are called ma
( G
in
behavior-based or black-box techniques e rl B
H,
G mb
• If the internal structure of an object is investigated, the methods used are
e s
v ic
called structure-based or white-box techniques
y ser
• Experience based techniques g
use knowledge and skills of the personnel
h n o lo
involved in test design ec
d ig t
n
tre use a combination of test techniques including process,
• Testers generally
3
2
business20rule and data-driven techniques to ensure adequate coverage of the
object©under test
• Criteria for choosing the appropriate test case design approach:
o Test basis, testing goals, risk aspects, project framework / preconditions,
contractual / client requirements

angajaților

ny
Ge
Be
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

2. Black-box test techniques Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Overview
• The following black-box techniques will be explained in detail: )
ny
o Equivalence partitioning r ma
Ge
o Boundary value analysis
rl in (
Be
o Decision table testing & cause effect graphing
b H,
m
o State transition testing e sG
v ic
o Use case testing
y ser
n o lo g
• This accounts for some c important and popular methods
t e h
• Other black-boxen d ig
methods are for instance:
t r
023testing
o Statistical
2
©
o Pairwise testing
o Smoke testing

angajaților
General
• Functional testing is targeted at verifying the correctness and the )
ny
completeness of a function ma r
Ge
o Do executed functions give correct results? in (
B e rl
H,
o Are all specified functions available within the module?
G mb
• The execution of the test cases should ibe
r v cesdone without high redundancy, but
nevertheless comprehensive e
g ys
o test as little as possible butnolo
o test as much as necessary t ech
ig
r e nd
t
0 23
Source: Wikipedia
© 2

angajaților
Equivalence partitioning
• Equivalence class partitioning is what most testers do intuitively: they) divide
ny
the possible values into classes. Hereby they look at: ma r
Ge
o input values of a program (usual use of EC-method)
rl in (
Be
H,
o output values of a program (rarely used EC-method)
G mb
• The range of defined values is groupedicinto es equivalence classes, for which
the following rules apply: e r v
g ys
o
o All values, for which a common
c h nol behavior of the program is expected, are grouped
i g te
together in one equivalence class
end may not overlap and may not contain any gaps
o Equivalence rclasses
t
023 classes may contain a range of values
o Equivalence
2
© 0 < x < 10) or a single value (e.g., x = “Yes”)
(e.g.,

angajaților
Equivalence partitioning – valid and invalid

• The equivalence classes of each variable (element) may be divided further
)
o valid EC: all values within the defined range are combined into onerm a
equivalence
ny
G e
class, if they are handled identically by the test object n( rl i
§
, Be or system
Valid values are values that should be accepted by the component
bH
Gm
o invalid EC: we distinguish two cases for values outside of the defined range:
s
§ Values with a correct format but outside of
r icevalue range can be combined into one or more
vthe
equivalence classes, they typically y e
s to a rejection by the component or system
lead
ol
Values with a wrong formatngenerally
og
build a separate EC
ech
§
ig t
nd using a single representative from each EC
• Tests are performed
re
t
023other value from the EC the same behavior as for the chosen value is
o For every
2
©
expected

angajaților
Equivalence partitioning – example

• Equivalence classes are chosen for valid and invalid inputs
o If a value x is defined as 0 ≤ x ≤ 100, then we can initially identify rma
ny)
e
three equivalence classes: n (G rl i
Be
1. x<0 (invalid input values) b H,
Gm
2. 0 ≤ x ≤ 100 es
(valid inputicvalues)
s rv
einput
3. x > 100 (invalid
gy values)
l o
hno
te c
d ig
o Further invalid EC can be defined, containing, but not limited to:
n
t re
0 23
§ non-numerical
2 inputs,
§©
numbers too big or too small,
§ non-supported format for numbers
<0 0..100 > 100

angajaților
Equivalence partitioning – input variables

• All input variables (elements) of the test object were identified, e.g.)
any
e rm
o fields of a GUI (e.g., system test)
o parameters of a function (e.g., component test) in (G
,B e rl
H
• A range for each input value is defined
G mb
s
ic e
o this range defines the sum of all valid equivalence class values (vEC)
s e rv
gyresult from the values outside of this range
o invalid equivalence classes (iEC)
lo
e c hno differently (known or suspected) are assigned to a
o values expected to be handled
ig t class
separate equivalence
e nd
02 3 tr
©2
<0 0..100 > 100


angajaților

• A program expects a percentage value according to the following )
ny
requirements: ma r
Ge
o only integer values are allowed
rl in (
Be
o 0 is the valid lower boundary of the range
b H,
o 100 is the valid upper boundary of the range m
e sG
s rv ic
• Valid are all numbers from 0 to 100,einvalid are all negative numbers, all
g y
numbers greater than 100, alllodecimal numbers and all non-numerical values
no
(e.g., “w”) ech t
ig
o one
e nd
valid equivalence class: 0 ≤ x ≤ 100
o 1st
0 2 3 tr equivalence class:
invalid x<0
© 2 invalid equivalence class:
o 2nd x > 100
o 3rd invalid equivalence class: x = no integer
o 4th invalid equivalence class: x = not numeric (n.n.)
<0 0..100 > 100

angajaților

• The percentage value will now be displayed in a bar chart.
: an
The following additional requirements apply (both values included)m
y)
r
Ge
o values between 0 and 15: grey bar
rl in (
Be
o values between 16 and 50: green bar
b H,
m
o values between 51 and 85: sG
yellow bar
ce
o values between 86 and 100: e r
red vibar
y s
• Now there are four instead
og
ol one valid equivalence classes:
hnof
o 1st i g tec equivalence class:
valid 0 ≤ x ≤ 15
d
o 2nd 3t re n valid equivalence class: 16 ≤ x ≤ 50
o 3rd© 2
02 valid equivalence class: 51 ≤ x ≤ 85
o 4th valid equivalence class: 86 ≤ x ≤ 100
<0 0..15 16..50 51..85 86..100 > 100


angajaților
Equivalence partitioning – picking representatives

• On a last step, one representative of each EC is determined as well as the
result to be expected for it a ny)
m r
Ge
rl in (
Be
b H,
variable equivalence class Gm
representatives
es
v ic
ser
percentage EC1: 0 ≤ x ≤ 15 10
y
og
value EC2: 16 ≤ x ≤ 50 20
l
hno
(valid)
c
EC3: 51 ≤ x ≤e85 80
d
EC4:n86 i≤gxt≤ 100 90
t re
percentage023EC5: x < 0 -10
value ©
2
EC6: x > 100 200
(invalid)
EC7: x no integer 1,5
EC8: x non numeric w

angajaților
Exercise IV.1: Equivalence partitioning

(10 min. individual work / 15 min. discussion of results)
ny )
From a given specification please extract: r ma
Ge
rl in (
• all input values Be
b H,
m
sG
• equivalence classes for each of the input values
v ic e
ser
o valid equivalence classes
o invalid equivalence classes lo gy
ec hno
t
en d ig
3 tr
0 2
©2

angajaților
Equivalence partitioning – example 2 1 of 4
• Analyzing the specification

a adiscount in %
o A piece of code computes the price of a product based on its value,rm
ny)
e
n (G
and shipping costs (6, 9 or 12 EUR, depending on shipping mode)
rl i
, Be
variable equivalence class status representative
b H Assumptions:
m
EC11: x ≥ 0 valid
sG
1000.00
e-1000.00 • Value of goods is
v ic
value of
ser
EC12: x < 0 invalid
goods given as a positive
EC13: x non-numerical value
l o gyinvalid fred
number with 2
EC21: 0% £ x £ 100%
h no valid 10%
decimal places
ec
discount
EC22: x < 0%
d ig t invalid -10%
• Discount is a
en
3 tr
EC23: x > 100% invalid 200%
percentage value
2 02 : x non-numerical value
EC 24 invalid fred
without decimal
© EC31: x = 6 valid 6
places between 0%
EC32: x = 9 valid 9 and 100%
shipping
EC33: x = 12 valid 12
costs • Shipping costs can
EC34: x ¹ {6, 9, 12} invalid 4
only be 6, 9 or 12
EC35: x non-numerical value invalid w

angajaților
• Test cases for valid EC:

a ny)
rmcases: T01,
o Valid equivalence classes provide the following combinations or test
e
T02 and T03 n( G
e rl i
B
, T01 T02
variable equivalence class status representative
b H T03
EC11: x ≥ 0 valid
s Gm
1000.00 * * *
value of
v ice-1000.00
ser
EC12: x < 0 invalid
goods
g y
invalid fred
lo
EC21: 0% £ x £ 100%
hno valid 10% * * *
EC22: x < 0%
ig te c invalid -10%
discount d
t re n
EC23: x > 100% invalid 200%
2 023: x non-numerical value

EC 24 invalid fred
© EC31: x = 6 valid 6 *
EC32: x = 9 valid 9 *
shipping
EC33: x = 12 valid 12 *
costs
EC34: x ¹ {6, 9, 12} invalid 4
EC35: x non-numerical value invalid w

angajaților
• Test cases for invalid EC: )

ny
o The following test cases were created using the invalid EC, r ma
Ge
each in combination with valid ECs of other elements:
rl in (
, Be T05
variable equivalence class status representative H T04 T06 T07 T08 T09 T10
EC11: x ≥ 0 valid G mb
c e s1000.00 * * * * *
invalid vi
value of
ser
EC12: x < 0 -1000.00 *
goods
o gyinvalid w *
EC21: 0% £ x £ 100%
h nol valid 10% * * * *
ec
EC22: x < 0%
ig t invalid -10% *
discount
e nd
3 tr
EC23: x > 100% invalid 200% *
02 : x non-numerical value
2EC 24 invalid w *
© EC31: x = 6 valid 6 * * * * *
EC32: x = 9 valid 9
shipping
EC33: x = 12 valid 12
costs
EC34: x ¹ {6, 9, 12} invalid 4 *
EC35: x non-numerical value invalid w *

angajaților
• All test cases:

a values) ny)
rm
o 10 test cases are derived: 3 positive (valid values) and 7 negative (invalid
e
test cases: n( G
e rl i
B
, T07 T08
bH
variable status representative T01 T02 T03 T04 T05 T06 T09 T10
valid 1000.00 * * * G m * * * * *
e s
rv ic *
value of
invalid -1000.00
goods
s e
y
og * * * *
invalid w *
n o l
ech
valid 10% * * *
t
dig200%
invalid -10% *
invalid en
discount
3 tr *
2
20invalid w *
© valid 6 * * * * * *
valid 9 *
shipping
valid 12 *
costs
invalid 4 *
invalid w *

angajaților
Equivalence partitioning – output based

• Equivalence classes can also be built, based on the expected output) values
any
e rm
o the method is used in analogy, applied to the output values
( G
lin value on the GUI)
o The variable (element) is thus the output (for example, a field
er
B
H, defined
o Equivalence classes are built for all possible outputs
G mb
es
o A representative is determined for each equivalence class of output values
ic
v
ser value is then acquired
o The input leading to the representative
y
n o lo g
• A higher costs and effort because the input values have to be found for the
t ech
given output recursivelyig
r e nd
t
2 0 23
©
Example:
A report creates one separate line for every type of stock (ECs) that is owned by
a customer. Pick your input values in such a way that every output line (i.e.,
every EC) in the report is created at least once.

angajaților
Equivalence partitioning – in general 1 of 2
• Partitioning
ny)
e ma
o The quality of the test depends on precisely segmented variables /relements in
equivalence classes n( G
e rl i
o EC that were not identified hold the risk of overlooking, B
b H possible defects, since the
m
representatives used did not cover all possibilities
e sG
• Test cases e rv ic
g ys
o
n l
o Equivalence class techniqueoprovides test cases for which a representative still has
to be chosen e c h
d ig t are selected by defining the representative or
n
o Test data combinations
3 tre of each equivalence class
representatives
02
©2

angajaților
Equivalence partitioning – in general 2 of 2
• choosing representatives )
ny
o any value within the EC can be a representative. Optimal are: r ma
Ge
§ typical values (used often)
rl in (
Be
§ problem values (suspected failures)
b H,
m
sG
§ boundary values (on the edge of the EC)
r v ic e
1. Representatives of valid EC maysbe e combined
g y
l o
2. Representatives of invalid c h noEC may not be combined
t e
ig
3. Representativesr e ndof invalid EC may only be combined with valid
t
2 0 23
representatives of other EC
©
4. For test cases, representatives of invalid EC should be combined with
always the same values of other valid EC (standard combinations)

angajaților
Equivalence partitioning – coverage

• Equivalence class coverage can be used as exit criteria to end testing)
ny
activities ma r
Ge
rl in (
number of EC tested Be
EC - coverage = × 100%
b H,
number of EC identified Gm
es
v ic
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

angajaților
Equivalence partitioning – transition

• The transition from the specification or definition of functionality to the
creation of equivalence classes a ny)
m er
(G
o Often a difficult task due to the lack of precise and completeindocumentation
B e rl
H, for boundary value
o Boundaries that are not defined (especially problematic
b
Gm
analysis!) or missing descriptions make it difficult to define equivalence classes
s
e
rvic to complete information
o Often, contact with the customer is needed
e
g ys
hn o lo
ec
ig t
e nd
02 3 tr
© 2

angajaților
Equivalence partitioning – benefits

• Benefits
ny)
ma a minimum
o Systematic test case design method to achieve defined coverage rwith
e
amount of test cases n( G
rl i
Be
o Values with equivalent behavior can be combined in, equivalence classes, based
b H
m
on their expected (i.e. specified) outcomes according to the specifications
sG
o Prioritizing equivalence classes canrvbe ic e
used to prioritize test cases (inputs that
s e
y
are used rarely are to be tested last)
lo g
o Tests of known exceptions
e c hnois covered by test cases on the basis of negative
ig t
equivalence classes
e
o Equivalencetrpartitioning
nd
is applicable at all levels of testing
23
20used to achieve input and output coverage goals
o Can
©be
o Can be applied to human input or via interfaces to a system or interface parameters
in integration testing

angajaților
Boundary value analysis 1 of 2
• Boundary analysis extends equivalence class partitioning by introducing a

rule for the choice of representatives a ny)
m r
Ge
rl to be tested
• The edge (boundary) values of the equivalence class eare in (
, B
intensively bH
Gm
• Why put more attention to the edges?vices
er
y s are not well defined or lead to different
o Often, the boundaries of value ranges
g
o
interpretations nol ch
e were coded correctly
ig t
o Checking if the boundaries
e nd
• Please note:
0 2 3 tr
©2
o Experience shows that errors occur very frequently on the boundaries of value
ranges!
Boundary value analysis can be applied at all test levels. It is easy to apply,
and its defect-finding capability is high on base of detailed specifications.

angajaților
Boundary value analysis 2 of 2
• Boundary value analysis assumes that

a ny)
rm (not a single
o that the equivalence class is composed of a continuous value range
e
value or a group of discrete values) n( G
e rl i
o that (explicit – specified, and implicit, unspecified, e.g.,, B
b H MaxInt) boundaries can be
defined for the value range
Gm
i c es
v
• As an extension to equivalence class
y serpartitioning, boundary value analysis is
a method suggesting the choice gof representatives
h n o lo
i g te c
o Equivalence class partitioning:
t r e d
§ examines onen(typical) value of the equivalence class
3
202 value analysis:
o Boundary
©
§ examines the boundaries
value range: bottom value δ x δ top value
and the neighboring values
bottom value – δ lower boundary lower boundary + δ
§ uses the following scheme:
top value – δ higher boundary higher boundary + δ
δ is the smallest step defined for the value
for example: 1 for integer value; 0.01 for currency

angajaților
Defining boundary values

• Basic scheme used for requirements that call for a range of numbers
) ny
ma
o rages may also be date ranges, time ranges or letter ranges (a-m, rn-z)
(G e
o in this case no additional testing is needed for a value in e rlinmiddle of the value
the
, B
range H b
m
e
• If the EC is defined as single numericalicvalue, for example x=5, the sG
e r v
neighboring values will be used as
y swell
og
ol and its neighboring values) are: 4, 5 and 6
hnclass
o the representatives (of the
ec
t
n
• If multiple parameters d ig
are partitioned in ECs, each class of each parameter
3 t re
2
can have20their own boundary values
©
o boundary values can be combined in different ways to form different types of
coverage (e.g., every value, every pair of values, all combination of values)

angajaților
Boundary analysis for invalid EC

• Boundary values make little sense for invalid equivalence classes
o Representatives of the invalid EC at the boundary to a valid EC arerm a
already covered
ny)
G e
through the basic scheme n(
rl i
, B e
• For value ranges defined as a set of discrete values,
m bH no boundaries can be
created generally G
i c es
v
ser widowed
o for example: single, married, divorced,
y
g
h n o lo
• Picking three values i g tec boundary (instead of two) allows for a more
per
intensive test tat
r nd boundary. The required test intensity is chosen based
eeach
0 23
on a risk2analysis
©
o intensity chosen by the risk of the test object
o example: safety critical applications have a high risk (failure causes high damage)

angajaților
Boundary analysis example 3a

• Example 3a: )
ny
o value range for a discount in %: 0.00 ≤ x ≤ 100.00 r ma
Ge
o Definition of EC rl in (
Be
3 classes:
b H,
m
§ 1. EC: x < 0
e sG
v ic
§ 2. EC: 0.00 ≤ x ≤ 100.00
y ser
§ 3. EC: x > 100
ol og
o Boundary analysis te chn
ig
end
extends the representatives
tr
to:
3
202 -0.01; 0.00; 0.01; 99.99; 100.00; 100.01
2. EC:
©
o Please note:
Instead of one representative for the valid EC, and two for the invalid EC,
there are now six representatives (four valid and two invalid)

angajaților
Boundary analysis example 3b

• Basic scheme: choose three values to be tested – the exact boundary) and
n y
the two neighboring values (within and outside the EC) ma r
Ge
rl to the EC, only
• Alternative point of view: since the boundary value belongs in (
, B e
two values are needed for testing: one within andbHone outside the EC
s Gm
• Example 3b: ic e
e rv
y s ≤ x ≤ 100.00
o value range for a discount in %:g0.00
o lo
chn
o valid EC: 0.00 ≤ x ≤ 100.00
te
ig
o boundary valuedanalysis
r e n
t
additional3representatives are: -0.01; 0.00; 100.00; 100.01
0 2
© 2– same behavior as 0.00
0.01
99.99 – same behavior as 100.00
• Most (but not all) programming errors caused by a wrong relational operator
will be found with the two boundary values

angajaților
Exercise IV.2: Boundary value analysis

ny )
From a given specification please extract: r ma
Ge
rl in (
• all input values Be
b H,
m
sG
• equivalence classes for each of the input values
e
v ic
ser
o valid equivalence classes
y
o invalid equivalence classes
ol og
tevalues
• Determine the boundary chn
en d ig
02 3 tr
© 2

angajaților
Decision table testing 1 of 5
• Equivalence class partitioning and boundary analysis deal with isolated

) input
n y
conditions ma r
Ge
• However, an input condition may have an effect only in l
rcombination with in (
, B e
other input conditions, the combinations are alsobH
referred to as business rules
Gm
cestake into account the effects of
• All previously described techniques dovinot
r
e
dependencies and combinations ys g
h n o lo
• Using the full set of combinations of all input equivalence classes often leads
i g te c
to a very high number of test cases (full decision table - test case explosion)
t r end
3 delete columns that do not affect the outcome (e.g.,
• One way20is2to
©
impossible or invalid combinations)
• Another way to reduce the number of possible combinations to a subset
(collapsed decision table) is the usage of a cause-and-effect graph or other
test case reduction technique

angajaților
Example 5: Online-Banking )
ny
T01 T02 T03 T04 T05 T06
er
T07 maT08
( G
conditions (causes) enough coverage Yes Yes Yes Yes No
rl in No
No No
correct recipient Yes Yes No No
H , Be Yes
Yes No No
valid TAN Yes No Yes
G mNob Yes No Yes No
s
actions (effects) do transferal Yes -
rv ic e - - - - - -
mark TAN as used Yesse - - - - - - -
gy
deny transferal
hn o lo - - Yes Yes Yes Yes Yes Yes
te c
request again TAN - Yes - - - - - -
n d ig
3 t re
• Rule to 20 2
© compress decision tables:
o if two columns have identical input data patterns and each has a different output,
these columns can be combined
o if two input columns have identical output data pattern but differ only in one input
data item, they can be combined

angajaților
Example 5: Online-Banking
a ny)
T01 T02 T03 T04 T05 T06
er
T07 m T08
( G
conditions (causes) enough coverage Yes Yes Yes Yes No
rl in No
No No
correct recipient Yes Yes No No
H , B- e Yes No No
valid TAN Yes No -
G mb
No - No Yes No
actions (effects) do transferal Yes -
v i ces- - - - - -
mark TAN as used Yes
y ser - - - - - - -
deny transferal o lo g - - Yes Yes Yes Yes Yes Yes
hn
te c
request again TAN - Yes - - - - - -
n d ig
tre represents a test case
• Each table column
20 23
• Rule to
© compress decision tables:
o if two columns have identical input data patterns and each has a different output,
these columns can be combined
o if two input columns have identical output data pattern but differ only in one input
data item, they can be combined

angajaților
• Practical use
a ny)
mdecision
o The specification is divided into manageable parts, thus leadingrto
e
tables of a practical size n (G rl i
Be
b H,
o It is difficult to deduce boundary values out of a cause-and-effect diagram or
decision table m
sG
o It is recommended to combine test cases
r v icederived from decision tables with
se analysis
values derived from a boundary yvalue
lo g
o The number of causes and
e c hnoeffects that are examined will determine the complexity
t
of the cause-and-effect
ig diagram: for n conditions that may be true or false, 2 test
n
n d
cases can berecreated
3t
202 of larger size, this method is only manageable with tool support
o On systems
©

angajaților
• Benefits
a ny)
rm that might
o Systematical identification of input combinations (combined causes)
e
not be found using other methods n( G
rl i
, Be
o Test cases are easily derived from the decision table
bH
Gm e.g., at least one test case
o Easy to determine sufficient test case coverage,
s
created for each column of the decision ic e
rvtable se
y
o The number of test cases can
o l ogbe reduced by systematically merging columns of
the decision table hn ec
t
• Drawbacks n d ig
3 t re
2 2 a large number of causes leads to complex and extensive results
o Setting0up
©
o Thus, many errors can occur when applying this method
o This makes it necessary to use a tool
number of DT-columns tested

DT - coverage = × 100%
number of DT-columns identified

angajaților
Exercise IV.3: Decision table test

ny )
Create a decision table for printer maintenance r ma
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

angajaților
State transition testing 1 of 6
• Many techniques only take into account the system behavior in terms) of input
ny
data and output data ma r
Ge
• Different states that a test object might take on are not l in (
ertaken
,
into account
B
o For example, results of actions that happened in the
m bH past – actions, that caused
sG
the test object to be in a certain internal state
e
e r v ic
• The different states that a test object s
l o gy can take on are modeled using state
transition diagrams
e c hno
d ig t
• State transition nanalysis is used to define state transition-based test cases.
t r e
0 23
The notation of a state transition in a state transition diagram is an edge
with an
© 2
event, a guard condition (optional) and an action (optional)
• State transition testing is much used within embedded software industry and
technical automation in general, but can also be used in many other
situations, e.g., where a GUI-flow is tested

angajaților
• To determine test cases using a )

dead dead
ny
state transition diagram, a tran-
r ma
Ge
in (
sition tree is constructed:
married
rl„to die“ married
Be
„to die“
o The initial state is the root
of the tree b H,
m
sG
„to marry“ „to marry“
e
o For every state that may be
rv ic
divorced widowed
reached from the initial state, se
a node is created which is olo
gy „getting divorced“ „death of partner“
n
connected to the roottebych
a branch married dead
ig
o This operation r e ndrepeated and
is „to die“
comes 0to23
t
an end if: „to marry“
2
§ ©
the state of the node is an end state single dead
(a leaf of the tree) „to die“
§ the same node with the same state
„be single“
is already part of the tree
unborn

angajaților
Every path from the root to a leaf then represents )

ny
a test case for state transition testing
e r maD D
The state transition tree for this example leads
rl i n (G
to the following six test cases: H , Be M M
G mb
i c es
state 4 rv V W
se
state 1 state 2 state 3 state 5 end state
y
unborn single dead lo g dead
e c hno
ig t
unborn single married dead dead M D
unborn singleend married widowed dead dead
3 tr
unborn
2 0 2 single married widowed married married
unborn
© single married divorced dead dead
S D
unborn single married divorced married married

U

angajaților
The state transition diagram can be transformed into )

ny
a state transition table:
r maD D
Ge
be single to marry
getting death
to die rl in (
divorced of partner
Be M M
H,
unborn single
G mb
ic es
e rv
single married dead V W
ys
married
ol og
divorced widowed dead
divorced te
married
chn dead M D
ig
widowed tr end married dead
3
dead © 20 2 S D

angajaților
dead dead • The transition tree y ) of our

example may a n
e rm now be
extended ( G
married married
e r lin using invalid
„to die“ „to die“ B
transitions (negative test
b H,
„to marry“ Gm
„to marry“ cases, robustness testing)
es
divorced widowed
v ic
ser • Example: a possible
gy
„getting divorced“
hn olo„death of partner“ invalid transitions –
ec there are more
d ig t
married dead
3 re n
t „to marry“
„to die“
• Impossible transitions
© 20 2 between states can not be
single dead tested
„to die“
„be single“ „marry underage“
error
unborn

angajaților
State transition testing – summary 6 of 6
• Test exit criteria (also used to calculate the coverage level in %) )

ny
o Every state has to be entered at least once r ma
Ge
o Every transition has to be executed at least once
rl in (
Be
• Benefits / drawbacks of this method b H,
m
sG
e be described as state machines
o Good testing method for test objects thaticcan
s e rv
gy only if the object life cycle is available
o Good testing method to test classes,
lo
o
t e chn complex, i.e., a lot of parameters are necessary to
o Very often, states are rather
describe the state ig
d
rencases and analyzing test results can be difficult and time consuming in these
§ designing ttest
3
20
cases 2
©
o Only covering all states does not guarantee complete test coverage

angajaților
Exercise IV.4: State transition test

(Exercises A and B: 10 min. individual work / 15 min. discussion of results)
ny )
• Draw a state chart (group-work with the trainer) r ma
Ge
rl in (
• Design test cases (group-work with the trainer) Be
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

angajaților
Use case testing 1 of 4
• Test cases are derived directly from the use cases of the test object )
y an
e rm
o The test object is seen as a system reacting with actors
G
r l in ( to an end result of
o A use case describes the interaction of all involved actors leading
the system , Be
mmet in order to execute the use bH
o Every use case has pre-conditions that mustGbe
c e s
case (the test case) successfully rv i se
y describing the system after execution of the
lo g
o Every use case has post-conditions
use case (the test case) hno
i g te c
t r end
• Use cases are elements of the Unified Modeling Language UML*
023diagrams are one of 13 different types of diagrams used by UML
o Use case
2
© case diagram is a diagram describing a behavior, it does not describe the
o A use
sequence of events
o It shows the system reaction from the viewpoint of a user
* UML is a non-proprietary specification language for object modeling

angajaților
• Example of a simple use case diagram (source: Wikipedia) )

ny
r ma
Ge
o The diagram on the right describes the
rl in ( Restaurant
functionality of a simple restaurant system , BeFood
Order
bH
o Use cases are represented by ovals and Gm Prepare Food
the actors are represented by stick figuresices
v
ser Wait Staff
Serve Food
Chef
o The patron actor can “eat food”, “pay g y for
food”, or “drink wine”
h n o lo
ec
t“prepare
o Only the chef actor can
d i g food”. Eat Food
n
re patron and the cashier
Note that bothtthe
2 3
© 20
are involved in the “pay for food” use case
Drink Wine
System
Patron Boundary
o The box defines the boundaries of the
“restaurant system”, i.e., the use cases
Pay for Food
shown are part of the system being
modelled, the actors are not Cashier

angajaților
• Every use case describes a certain task (user-system-interaction) )

ny
er not limited to:
• Use case descriptions (use case specifications) include, butGare ma
in (
o pre-conditions
,B e rl
H
o expected results / system behavior
G mb
o post-conditions ic es
s e rv
y
n o lo g
• These descriptive elements (attributes) are also used to define the
corresponding test cases
ec h
d ig t
n
• Every use case
3 tremay be used as the basis for a test case
20 2
• Every©alternative and every exception within the specification corresponds
to a separate test case (coverage in %)
• Typically, information provided with a use case has not enough detail to
define the test cases directly. Additional data is needed (input data, expected
results) to define a test case

angajaților
Use case testing – benefits and drawbacks 4 of 4
• Benefits
ny)
ma use case
o Well suited for acceptance testing and system testing, because reach
e
describes a user scenario to be tested n( G
e participation rl i
o Useful for designing acceptance tests with customer ,/ B
user
bH
Gmin UML
o Well suited if system specifications are available
ic es
e rv
o May be combined with other specification-based test techniques
ys
og
• Drawbacks nol
t ech
e n d ig
o No derivation of additional test cases beyond the information provided by the use
case t r
2 0 23
o Therefore,
© this method should be used only combined with other methods of
systematic test case design
number of UC-behavior tested

UC - coverage = × 100%
number of UC-behavior identified

angajaților
Black-box / behavior-based techniques – general conclusions

• Testing system functionality is the main goal of black-box testing )
y a n
rm
o Therefore, the test result depends on the quality of the system specification
e
(G test cases)
(e.g., completeness, missing or wrong specifications lead to nbad
rl i
, Beis only performed for the
§ If specifications are wrong, tests will be wrong, too. Testing
Hfunctionality
described functions. Missing specifications of required
m b will not be discovered
during testing s G
e
v ic
ser
o If the test object holds functions ythat have not been specified, they will not be
g
examined o lo
hn
i g te c
§ Such superfluous functions may cause problems in the area of stability and security (e.g.,
n d
software for automated teller machines (ATM))
t re
023 drawbacks, functional testing is still the most important
• Despite 2these
testing©activity
o Black-box techniques are always used in testing
o The drawbacks can be compensated by using additional methods of test case
design, e.g., white-box testing or experience-based testing

angajaților
Black-box or behavior-based techniques – summary

• Black-box methods: )
ny
o Equivalence class partitioning r ma
Ge
o Boundary value analysis
rl in (
Be
o decision tables (and cause effect graphing)
b H,
m
o State-transition testing sG
v ic e
o Use case based testing
y ser
o l og
n specified functions: if functions are not
• Black-box testing verifieschthe
specified, they are ig t e
not tested
r e nd
• Additional 3 t (e.g., code that should not be there) cannot be detected
02code
2
using ©
black-box testing

angajaților

ny
Ge
Be
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

3. White-box test techniques Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
White-box or structure-based test techniques 1 of 2
• The following techniques will be explained in more detail: )

ny
o Statement testing and coverage
er ma
( G
o Decision testing and coverage (comparable to branch coverage)
e rl in
, B
o Condition testing and coverage
m bH
G
o Path testing and coverage
i c es
v
• Remark: y ser
g
These techniques are dynamic h n olo test techniques. They relate to static analysis
which was described i g tec (control flow graph)
earlier
t r end
0 23
• Other white-box techniques include but are not limited to:
2
© (Linear Code Sequence And Jump)
o LCSAJ
o Data-flow based techniques

angajaților
White-box or structure-based test techniques 2 of 2
• It based on an identified structure of the software or the system:

a ny)
rm
o Component-level: the structure of a software component, i.e. statements,
e
decisions, branches, distinct paths n (G rl i
Be in which modules call
H,
o Integration level: the structure may be a call tree (a diagram
b
other modules) m
sG
ice structure, business process or web
o System level: the structure may be a rmenu
v
e
page structure ys
n o lo g
• Code-related structural t ch design techniques for code coverage based on
etest
ig
r e nd and branches
statements, decisions
t
2 0 23
©

angajaților
White-box or structure-based test techniques – tools 1 of 2
• During white-box testing, the program under test is executed, same as) in
n y
ma testing
black-box or experience-based testing. Those are part of dynamic
er
G
o Theory states, that all parts of the program should be executed
r l in ( at least once
during testing , Be
bH
• The rate of coverage of the program is measured
s Gm using tools (e.g., coverage
ic e
analyzers): e rv
ys
g in order to count execution paths, i.e. counters
h n o lo
o Code instrumentation is performed
ec
are inserted into the program
t
code of the test object
ig
o These counters
r e ndare initialized holding the value of zero, each execution path
t
23 the respective counter
increments
20
©
o Counters that remain zero after testing indicate program parts that have not been
executed

angajaților
White-box or structure-based test techniques – tools 2 of 2
• White-box techniques need the support of tools in many areas, there )are:
any
e rm
o Test design
(G
lincode
§ automatically generating a control flow graph from program source
r
o Test execution , Be
bH
s Gm
§ tools to monitor and control the program flow inside the test objects
r v ic e
e
• Tool support ensures the quality of
g y sthe tests and increases efficiency
o Because of the complexitynof o
olthe necessary measures for white-box testing, manual
e c h
ig t
test execution is
n d
tre consuming
§ time and resource
3
2
0 to implement and prone to errors
§ difficult
©2

angajaților
The main types of coverage

• Statement testing and coverage
a ny)
rmby the test
o the percentage of executable statements that have been exercised
e
cases n( G
rl i
Be
o can also be applied to modules, classes, menu items,, etc.
H
G mb
• Decision testing and coverage (comparable s to branch coverage)
r v ic e
e
o the percentage of decision outcomes,
g y s that have been exercised by the test cases
o
• Path testing and coverage c h nol
e
g t that have been exercised by the test cases
o the percentage ofdipaths,
n
3 t re
• Condition 02testing and coverage
©2
o the percentage of all atomic condition outcomes independently affecting a decision
outcome, that have been exercised by the test cases
o Condition coverage comes in various degrees, e.g., single, multiple and modified
condition decision coverage (MCDC)

angajaților
Statement testing and coverage

• The statements in the program code is the focus of attention
ny)
e ma
o What test cases are necessary in order to execute all (or a certain rpercentage ) of
the existing code statements? n ( G
rl i
Be
• Basis of this analysis is the control flow graph b H,
m G
o all instructions are represented as nodes and
i c es the control flow between the
v
er(arrow)
instructions is represented as an edge
ys
n o log in a single node if they can only be executed in
o multiple instructions are combined
one particular sequenceech t
ig
end
• Aim of the testtr(test exit criteria) is to achieve the coverage of a selected
percentage
2 023of all statements, called the statement coverage
©
(C0 code coverage)
number of executed statements

statement coverage (C0) = × 100%
total number of statements

angajaților
Control flow graph

Directed graph )
ny
• Nodes represent statements or sequences of statements r ma
Ge
rl loops
• Edges represent control flow transfer, as in decisions eand in (
,B
• Represents the code structure of a program / m bH
component
e sG
rv ic
• Tools based construction
y se
no lo g
h
ig te c
A control flow graph
r e ndcan be seen as a simplified version
t
23 with the following characteristics. Differences:
of a flow chart,
20
© graphical elements are used: nodes and edges
• only two
• conditions behind decisions are not explained
• details behind statements (nodes) are not relevant

angajaților
Statement testing and coverage – example 1 1 of 2
• We are assessing the following segment of )

ny
program code, which is represented by the
r ma
Ge
in (
control flow graph (see right side):
rl
Be
b H,
if (i > 0) { m
e sG
j = f(i); v ic
if (j > 10) { y ser
while (k > 10)h{no
lo g
...ig te c
e nd
02 3 tr }
© 2 }
}

angajaților
Statement testing and coverage – example 1 2 of 2
• Consider the program represented by the control )

ny
flow graph on the right
r ma
Ge
o Contains two if-statements and a loop (do-while)
rl in (
inside the second if-statement Be
b H,
• There are three different “routes” throughethe
s Gm
ic
program segment e rv
ys
og directions
o The first if-statements allows ltwo
no
h the first if-statement is
ecof
o The right-hand direction
t
ig
nd the second if-statement
divided again using
3 t re
0 2
©2
• All statements of this program can be reached
using the route to the right
o A single test case will be enough to reach
100% statement coverage

angajaților
Statement testing and coverage – example 2

• In this example the graph is slightly more complex: )
ny
o The program contains the if-statements and a loop r ma
Ge
(inside one if statement)
rl in (
• Four different “routes” lead through this programbH,
Be
m
segment
c e sG
i
e rv
o The first if-statement allows two directions
s
y
n o log another
o In both branches of the if-statement
ch two different directions
if-statement allows forteagain
ig coverage, four test cases
o For a 100% statement
r e nd
t
0 23
are needed
2
©

angajaților
Statement testing and coverage – general conclusions

• The measurement of coverage is done using specifically designed tools
)
a ny
rm
o these tools are called coverage analysis tools or coverageeanalyzers
in (G
• Benefits / drawbacks of this method
,B e rl
bHare never executed, will be
o Dead code, that is, code made up of statementsmthat
G
discovered ces rv i
se a 100% coverage cannot be achieved
§ if there is dead code within the program,
gy
lo which is necessary in order to fulfill the
c h nocode
o Missing instructions, that is,
e detected
specification, cannot tbe
ig
d with respect to the executed statements: can all code be reached /
§ Testing is only
t r endone
23
executed?
20
§ ©
Missing code cannot be detected using white-box test techniques (coverage analysis)

angajaților
Decision testing and coverage

• Instead of statements, decision coverage focuses on the control flow )with a
program segment (not the nodes, but the exits of all decision points
m anyin a
e r
control flow graph) n (G rl i
e the control flow graph
, Bof
o All exits of decision statements (alternatively: all edges)
have to be covered at least once b H
m
sG
e exit of decision statements
o Which test cases are necessary to cover
r v iceach
e
y s flow graph at least once?
(alternatively: each edge) of the control
n o lo g
• Aim of this test (test exit h
eccriteria) is to achieve the coverage of a selected
i g t
percentage of allndecisions,
d called the decision coverage (C1 code coverage)
t r e
2 0 23 number of executed decisions
© coverage (C ) =
decision 1 × 100%
total number of decisions
o Synonymous (only at 100% coverage) to:
number of covered branches
branch coverage (C1) = × 100%
total number of all branches

angajaților
Decision testing and coverage – example 1

• The control flow graph on the right represents )
ny
the program segment to be inspected
r ma
Ge
• Three different “routes” lead through the graph rl in (
, Be
of this program segment H
mb G
es
o The first if-statement allows two different directions
o One path of the first if-statement is s er
divided again
v ic
g y
lo
in two different paths, one of which
no
holds a loop
o All edges can only bete ch
reached via a combination
d ig paths
of the three possible
n
t re
023cases are needed to achieve a
o Three test
2
©
decision coverage of 100%
o Only using the two directions on the right,
nine out of ten edges can be covered
(branch coverage: C1=90%)

angajaților
Decision testing and coverage – example 2

• In this example the graph is slightly more complex: )
ny
• Four different “routes” lead through this program r ma
Ge
segment
rl in (
Be
o The first if-statement allows two directions
b H,
o In both branches of the if-statement anothersif-Gm
r
statement allows again for two different v ic e
directions
e s
o in this example, the loop is not gy
locounted as an
additional decision h n o
c te
ig
r e nd coverage four test cases are
o For a 100% decision
needed 3 t
02
© 2example, the same set of test cases is also
o In this
required for 100% statement coverage!

angajaților
Decision testing and coverage – general conclusions

• Achieving 100% decision coverage requires at least as many test cases
) as
n y
100% statement coverage – in most cases more ma er
G
in ( coverage!
o a 100% decision coverage always includes a 100% statement
rl
, Be
• In most cases edges are covered multiple timesbH
s Gm
e
• The test cases necessary to cover each
s e rvicexit of decision statements, for IF-
statements, there are two exits.gFor y a CASE-statement there can be multiple
o l o
exits n
t ech
ig
• Drawbacks
r e nd
t
2 23
o Missing0statements cannot be detected
©
o Not sufficient to test complex conditions
o Not sufficient to test loops extensively
o No consideration of dependencies between loops

angajaților
Statement / decision testing and coverage

• Both methods relate to the paths through the control flow graph
a ny)
rmcoverage
o they differ in the amount of test cases necessary to achieve 100%
i n (G e
• Only the final result of a condition is considered, although l
r the resulting
, Be
condition can be made up of several atomic conditions H
G mb
s
o the condition If ((a>2) OR (b<6)) may only
r v icebe true or false
e
o which path of the program is executed
g y s depends only on the final outcome of the
o
combined condition
c h nol
e
o failures due to a wrong
d i g t implementation of parts of a combined condition may not be
detected n
3 t re
0 2
©2

angajaților
Exercise IV.5: White-box techniques

(Exercises A – D: 10 min. individual work / 15 min. discussion of results)
a ny)
Using a control flow graph (part 1) and pseudo code (part 2) the m
erminimum
( G
number of test cases must be determined to achieve 100%
e rlincoverage of:
, B
o statements
m bH
G
o branches / decisions
i c es
v
o paths
y ser
g
h n o lo
i g te c
t r end
2 0 23
©

angajaților
Summary
• White-box and black-box techniques are dynamic methods, the test)object is
ny
executed during test ma r
Ge
• White-box techniques comprise:
rl in (
Be
o Statement coverage
b H,
m
o Decision coverage
e sG
v ic
o Path coverage
y ser
o Condition coverage (single, o log MCDC)
multiple,
e chn
• Only existing code can
d ig t be tested. If functions are missing, this fact cannot be
n
tre or superfluous code, however, can be discovered using
discovered. Dead
3
2
20testing
white-box
©
• White-box techniques are used mainly in lower test levels like component
testing or integration testing; however, they can easily be transferred to use
for business processes as well
• The techniques differ in their intensity of test (test depth) and consequently
the number of test cases differs
angajaților

ny
Ge
Be
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

4. Experience-based test techniques Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Definition of experience-based test techniques

• Practice of creating test cases, )
ny
based on the intuition, skills and ma
erpartitioning
experience of the tester Equivalence
( G
in value analysis
e rl
Black-Box
Boundary
Quality Control (Analytical QA)
P
o Where have errors , BState
H transition testing
accumulated in the past? G mb Decision tables
ic es
e rv
o Where does software often fail? Use case based testing
dynamic
ys
• Experience based testing is oalso
n lo g Experience-based techniques
echincludes:
called intuitive testing tand
ig Statement coverage
nd point oriented
White-Box
error guessing (weak

e
P
t r Decision coverage
0 23
testing), exploratory testing (iterative Condition coverage
2
testing©based on gained knowledge Path coverage
about the system) and checklist-
based testing Reviews/walkthroughs
P
static
Data flow analysis


angajaților
Test case design

The tester must dispose of applicable experience or knowledge )
ny
• Intuition – Where can defects be hiding? r ma
Ge
o Intuition characterizes a good tester
rl in (
Be
H,
mbwhere in the past?
• Experience – What defects were encountered
o Knowledge based on experience e sG
sedefects
o An alternative is to set up a list of recurring
rv ic
o gy
c h nol
§ Testing a leap year after 2060
te
(known problems of the past)
ig
t r end input
§ Empty sets within values
3
(a similar application has had errors on this)
© 20 2
• Knowledge / awareness – Where are specific defects expected?
o Specific details of the project are incorporated
o Where will defects be made due to time pressure and complexity?
o Are inexperienced programmers involved?

angajaților
Intuitive test case design – possible sources

• Test results and practical experience with similar systems
a ny)
rm functionality
o Possibly a predecessor of the software or another system with similar
in (G e
• User experience
,B e rl
H
mb
o Exchange of experience with the system as a user
G
ic es
• Focus of deployment
s e rv
y
ogused the most?
o What parts of the system will lbe
ec hno
ig t
• Development problems
d
n
treweak points out of difficulties in the development process?
o Are there any
3
2 02
©

angajaților
Error Guessing in practice

• Check defect lists )
ny
o List possible defects
er ma
( G
o Weight factors depending on risk and probability of occurrence
e rl in
, B
• Test case design m bH
o Creating test cases aimed at producing the c sG
edefects on the list
r v i
s e
l o gy value
o Prioritizing test cases by their risk
no
ech testing
• Update the defect list talong
ig
o Iterative procedure
r e nd
t
023 collection of experience is useful when repeating the procedure in
o A structured
2
© projects
future

angajaților
Exploratory testing
• Test case design procedure especially suitable when the information) basis
ny
is weakly structured ma r
Ge
• Also useful when time for testing is scarce rl in (
Be
b H,
• Procedure: m
e sG
rv ic
o Examine the single parts of the test object
se
y
guessing n o log on the parts to be tested, applying error
o Execute few test cases, exclusively
h ec
ig t
o Analyze results,ddevelop a rough model of how the test object functions
tr en
23design new test objects applying the knowledge recently acquired
o Iteration:
20
©
o Thereby focusing on conspicuous areas and on exploring further characteristics of
the test object
o Capture tools may be useful for logging test activities

angajaților
Exploratory testing – principles

• Choose small objects and / or concentrate on particular aspects of a test
object (called “charter”) a ny)
m er
G
in (max. 1 hour)
o A single iteration should not take more than 2 hours (currently
o Possible ideas for charters are e.g., test conditions, H ,B
personas,
e rl
heuristics and tours
G mb
• The results of one iteration form the information s basis of the following iteration
r v ic e
o Additional test cases are derived y se the particular test situation
from
lo g
• Modeling takes place during c h notesting
te
igwhite-box
o Test design (black-,
n d
ebased
and experienced-based), test execution, test logging
t r
2 0 23
and learning, on a test charter containing test objectives are concurrent and
carried
© out within time-boxes (sessions)
• Preparing further tests
o Herewith, knowledge can be gained to support the appropriate choice of test case
design methods
• Documentation, typically “light weight”, e.g., on session sheets

angajaților
Checklist-based testing – principles

• Test design, implementation and test execution based on checklist items (e.g.,
test conditions) a ny)
m er
G
• Checklist items may have been defined based on specific
r l in (or high-level
Be
experience and information: H, b
Gm for the user
o What does the user typically do, what is important
es
v ic
ser
o What makes the software fail
g y
olocorrected for specifics of the current project
• Checklist can be updated and
e chn
ig tdifferent testing types (functional, non-functional, etc.)
• Checklists may serve
d
ren variation in test execution a better coverage may be
• Because of 3a twider
2
20but less repeatability
achieved,
©

angajaților
Examples for experience-based testing techniques

• Adam Tester knows that John has written the code. Because John is )
ny
ma
inexperienced (he just finished university) his code is tested intensively
er
• Brian Tester has a list of previous errors that he has come (G
in across during his
e r l
B
H, towards critical areas
past testing activities. He uses this to focus his testing
b
(where problems occurred before) Gm
ic es
erv common errors that C++ developers
• Clarissa Tester uses a list with the most
s
y
log towards these high-risk areas
make to focus her testing activities
no
t ech
• Dave Tester uses his knowledge about the previous version of the product to
i g
d towards former problem areas
direct his testing neffort
t re
• Eve Tester
2 02is3 responsible to test the behavior of an application when used with
invalid©input data. She designs an initial test scenario to get more information
about the systems reaction and uses this information to make up further tests.
Which method is used by each one of these testers?


angajaților
Experience-based versus specification-based techniques

• Intuitive test case design is a good complement to systematical approaches
)
any
e rm
o It cannot give proof of coverage
in (G
o the number of test cases can vary considerably
B e rl
b H,
Gm
• Test are executed in the same way as with systematically defined test cases
s
ic e
ervtest cases were designed / identified
• The difference is the way in which sthe
y
og
c h nol can be detected that may not be found
• Through intuitive testing defects
through systematical te
ig testing methods
e nd
02 3 tr
© 2

angajaților
Summary
• Experience based techniques complement systematical techniques to)
ny
determine test cases ma r
Ge
rl
• They depend strongly on the individual ability of theetester in (
B
H,
G mb
• Error guessing, exploratory testing and checklist-based testing are three
s
ic e
of the more widely used techniques ofvexperience-based testing
y ser
• The international standard (ISO g29119-4) contains descriptions of test
h n o lo
techniques and their corresponding coverage measures
i g te c
t r end
2 0 23
©

angajaților
Keywords
• test technique
• black-box test technique )
• use case testing ny
• boundary value analysis r ma
Ge
• white-box test (technique
• checklist-based testing rl in
Be
• coverage b H,
m
e sG
• decision coverage v ic
• decision table testing y ser
ol og
• error guessing e chn
i gt
end
• equivalence partitioning
tr
3
20 2
• experience-based
©
test technique
• exploratory testing
• state transition testing
• statement coverage

angajaților
ny )
r ma
Ge
rl in (
Be
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
V. Test management »
angajaților

• V/1 Test organization )
ny
• V/2 Test planning and estimation r ma
Ge
• V/3 Test monitoring and control rl in (
Be
m
e sG
• V/5 Risk and testing v ic
ol og
te chn
ig
tr end
3
© 20 2

angajaților
Learning objectives for test management: 1 of 2
5.1 Test organization

a ny)
• FL-5.1.1 (K2) rm
Explain the benefits and drawbacks of independent testing
(G e
• FL-5.1.2 (K1) l in
Identify the tasks of a test manager and tester
r
Be
5.2 Test planning and estimation H,
G mb
• FL-5.2.1 (K2) Summarize the purpose andecontent s of a test plan
r v ic
• FL-5.2.2 (K2) e
Differentiate betweensvarious test strategies
g y
o
• FL-5.2.3 (K2) Give examples
c h nolof potential entry and exit criteria
• FL-5.2.4 (K3) Apply te
ig knowledge of prioritization, and technical and logical
n d
3 t re
dependencies to schedule test execution for a given set of test case
• FL-5.2.5 2 02
(K1) Identify factors that influence the effort related to testing
©
• FL-5.2.6 (K2) Explain the difference between two estimation techniques: the
metrics-based technique and the expert-based technique
angajaților
Learning objectives for test management: 2 of 2
5.3 Test monitoring and control )

ny
• FL-5.3.1 (K1) Recall metrics used for testing
er ma
G
• FL-5.3.2 (K2) in ( for test reports
Summarize the purposes, contents, and audiences
rl
Be
5.4 Configuration management
b H,
• FL-5.4.1 (K2) Gm
Summarize how configurationsmanagement supports testing
e
v ic
5.5 Risks and testing
y ser
g
• FL-5.5.1 (K1) oloby using likelihood and impact
Define risk level
• FL-5.5.2 (K2) chn
te between project and product risks
Distinguish
ig
• FL-5.5.3 (K2) r e nd
Describe, by using examples, how product risk analysis may influence
02 3t the thoroughness and scope of testing
© 2
5.6 Defect management
• FL-5.6.1 (K3) Write a defect report, covering defects found during testing

angajaților

ny
Ge
Be
m
sG
• V/5 Risks and testing v ic e
ser
• V/6 Defect management
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
1. Test organization Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Test management as a part of the test process

• Test management is project management of test projects )
ny
er
• Testing is a set of activities covering the complete softwareGdevelopment ma
in (
process e rl B
H,
• The test management activities are needed G mb
throughout the entire test process:
es
v ic
y ser
activity
ol og product
work
test planning te chn master test plan (static)
ig
test planning r e nd test plan (dynamic)
3t
test© 20 2
control status reports, control action
test acceptance product release

angajaților
Testing can be done independently

Advantages
a ny)
• unbiased, not personally attached to test object, can report issues
( G e rm
without (political) pressure
e rl in
, B
• They may question facts on the test basis and verify
m bH assumptions
made while designing tests G
i c es
v
• Independent testers may be more effective
y ser in finding defects and find different
o g
nol
kind of failures
e c h
Drawbacks d ig t
n
3 t re
2 2
• Increased0communication efforts, “having the last word on”-conflicts
©
• Developers loose sense of responsibility, little collaboration test and devs
• Lack of information, testers may be seen as bottleneck for releasing
• Developers may be more efficient in finding defects (they know where to
check)
angajaților
Other ways of setting up test teams

• Developers test their own programs )
ny
• Testers are also members of the development team r ma
Ge
l in (
er
, Borganization
• Testers are also members of the project team or line
b H
(independent test team) Gm
i ces
• Testers from the business or user group
s erv or specialists for specialized
tasks (test types) like usability, gy
losecurity, performance, portability or
h n o
regulatory compliance ec
d ig t
n
tre (in- or outsourcing)
• External test teams
3
0 2
©2
The above order reflects the degree of independence
• With multiple test levels, some levels are handled by Independent testers
• In agile teams testers might be part of the development team and product
owners might do user acceptance testing during sprint review.
angajaților
Profiles of test personnel

• Testing requires persons with a wide variety of skills and qualifications)
m any
• A test team may have the following roles covered within theGtest r process:
i n ( e
rl
o Test manager
, Be
H
o Test designer
G mb
s
o Test automation engineer
r v ic e
e
o Test administrator / test system g ys
administrator
o
o Tester
c h nol
o Technical expert dig
te
n
3 t re
• Notes: 202
©
o additional roles may be specified, e.g., database administrator, load tester
o In Agile, especially Scrum, people are typically called “developer” even though they
may focus mainly or entirely on testing tasks
angajaților
Soft skills
• In addition to their technical skills, test team members require the following
qualification and experience: a ny)
m r
Ge
o Team members: (political and) diplomatic instinct
rl in (
Be
o Willingness to question seemingly obvious facts
b H,
m
s Gand collaboration
o Persistence, strong standing in communication
rv picture in mind
o Exactness and creativity, keeping theebig
ic e
ys
o Skillful handling of complex o lo g
situations
ec hn
ig t
o Fast learning skills
e nd
02 3 tr
© 2

angajaților
Organization of (test) teams

• Use an appropriate organization for each individual project
a ny)
• The roles and corresponding tasks relate to the chosen SLDCerand m the product
n ( G
• Not every role must be filled in by a separate person. eInrlismall teams one
, B
person may take on multiple roles
m bH
G
i c es
v
y ser
g
h n o lo
i g te c
t r end
2 0 23
©
angajaților
Tasks of the test manager – overview

• Writing and / or reviewing the test policy and the generic test strategy)
any
rm testers)
• Organizing the test team (skill and career development, advocate
Ge
r l in (
• Test planning (in accordance with the test policy and e
Bgeneric test strategy)
b H,
m
• Test cycle planning
c e sG
s rv i
eautomation
• Test approach incl. decision on gtest y (tool selection and
l o
implementation) and test h no
environments
t ec
• d ig
Initiation of testernactivities
e
(analysis, design, implementation, execution of
t r
0 23
tests and check the status of exit criteria)
© 2
• Test progress monitoring and management
• Introduction of an adequate defect and configuration management system
• Results / progress reporting for the stakeholders, incl. promote and advocate
the testers, the test team, and the test profession within the organization
angajaților
Tasks of the test manager – test planning / test cycle planning

• Test approach: Implementation of a test strategy for a specific project
)
m any
• Test cycle: a cycle through the fundamental test process regarding r a
i n (G e
particular test object rl
, Be
H
• Activities of the testing process (reminder): Gmb
s
o Test planning and controlling r v ic e Test Planning
s e
gy

o Test design o
o l
o Test implementation chn
Test Analysis
t e
o Test execution ndig
3 t re Test Design
2
o Test completion
0
©2 Test Implementation
Test Execution
Test Completion
angajaților

• Test planning )
ny
o Planning of the test process:
r ma
Ge
in (
shall be made up during the early phase of the project.
The results shall be put in a document (master test plan) rl
Be
b H,
• Test cycle planning m
sG
o Detailed planning of a test cycle:
r v ic e Test Planning
e
Static master test plan will be detailed tosdescribe
ythe particular
o g
nol progress,
a specific test cycle. Details depend on
Test Analysis
c h
project situation (e.g., development
test results, availableig te
resources)
t r end Test Design
023test manager:
• Tasks of the
2
©
Initiating, controlling and Test Implementation
supervising tests and test cycles
Test Execution
Test Completion

angajaților

• Test planning starts at the beginning of a project )
ny
• Milestones, budget and priorities of the various test activities r
eneed to be ma
n ( G
i
addressed, risks need to be understood e rl B
H,
• Project development (dependencies) needs G mb
es
to be taken into account v ic r Test Planning
y setest
o During the project, delays may occur and

planning needs to be adapted olo
g
h n Test Analysis
ec
• Select tools to bedused ig t and decide on
n
test automation3 t re Test Design
02
© 2 tools and automation degrees for
o Different
Test Implementation
different test levels
• Resources must be planned
Test Execution
o Sparse and usually need to be assigned individually
o During test cycles, delays might occur in such a way
that planning of testing resources need to be revised Test Completion
angajaților

• Evaluation of previous tests should be taken into account
andepending on y)
r m
o Actual results of test activities may influence planning of other test activities, e.g.,
e shorter or longer
(Gbe
the number of errors found in the first test cycle, the second test cycle will
in
B e rl
• Control of the ongoing test activities is done
b H,
using metrics stated and agreed upon in s Gm
e
the test plan rv ic e
Test Planning
ys
ol og
chn
Test Analysis
e
ig t
e nd
3 tr
Test Design
© 20 2
Test Implementation
Test Execution
Test Completion

angajaților
Tasks of the test manager – test monitoring and control

• Test management brings transparency into the progress of testing and
supplies indicators for the project management a ny)
m r
Ge
rl
• Reports generated during test execution (e.g., defectereports, summaries by in (
, B
bHreports to the customer are
defect classes, statistics), the defect tracking and
m
an important source of information for the s G
e
project leader and the company ervic
Test Planning
s
gytime

management (e.g., as a basislofor
and resource planning) chn
o Test Analysis
e
d ig t
• Using tools and n
3 tre templates will increase Test Design
02may reduce the workload
quality and
©2 Test Implementation
Test Execution
Test Completion
angajaților
Tasks of the test manager – test monitoring and control

• Test management includes acceptance of the project results, meaning:
a
the product must meet the defined requirements and specification ny)
m er
o The project manager, in accordance with the test manager, decides on (G acceptance of the
in the
test objects (e.g., transfer to the next test level) e r l
H ,B
• A test summary report ensures G mb
ic es Test Planning
the completion of the test activities
e rv
ys
ol og
chn
Test Analysis
e
ig t
e nd
3 tr
Test Design
© 20 2
Test Implementation
Test Execution
Test Completion

angajaților
Tasks of the test manager – initiating test analysis and -design

• One of the goals of testing is to find as many relevant defects with as)little
ny
effort as possible! ma r
Ge
rl
• All tests documented in the master test plan are specified, i.e., it is in (
, B e
stated, how each test case is furnished and
m bH
how it should be executed. This process iss G
ic e Test Planning
initiated by the test manager e rv s
y

lo g
o test cases are made up of single steps
o
n help of personnel Test Analysis
t e chwith
o test cases should be derived
igrequirements of the software!
knowing the functional
nd
tre be designed keeping their
o test cases should
3
Test Design
2
20 in mind
repeatability
© Test Implementation
Test Execution
Test Completion
angajaților
Tasks of the test manager – test implementation and -execution

• Comparison of expected and actual results within the project )
ny
o Every test cycle needs to be adjusted to the test plan
r ma
Ge
o Did delays or changes occur?
rl in (
Be
H,
o Are the results within the range of expectancy?
Number of detected defects, time for corrections, b
Gm
retests, etc. es
rv ic
Test Planning
s e
• All deviations are to be reportedgand y to be
o l o
chn
taken into account Test Analysis
o Corrective measuresig t e
have to be taken on a
n d
3 t re
regular basis regarding the test plan and Test Design
20 2
current test activities, e.g.:
§ ©
Adjust dates for planned tests Test Implementation
§ Adjust resources for test execution
§ Perform additional / skip test cycles Test Execution
§ Change priority of test cycles
Test Completion

angajaților
Who takes the test manager role?

• The way in which the test manager role is carried out varies depending) on
n y
the software development lifecycle ma er
• In sequential SDLC models, the test manager role canlibe
r n (Gexecuted by a
Be
b H,
professional test manager, a project manager, a development manager or a
m last 3 is not a good idea!). In
quality manager (statement from the trainer:Gthe
e s
vic handled by test coordinators
bigger projects some of the tasks arerbeing
e
s
gy
coordinating small teams of testers
n o lo
ech of the tasks of the test manager are handled by
• Agile development: some
t
the Agile team d ig en
3 tr with the day-to-day testing done within the team, often by a tester
o tasks concerned
0 2
© 2 within the team
working
o tasks that span multiple teams or the entire organization, or that have to do with
personnel management, may be done by test managers outside of the development
team (called test coaches)
angajaților
Tasks of the tester* – overview

• Assists in implementing test planning activities and review the test plan
)
any
• Analyse, review, and assess test basis for testability G e rm
r l in (
• Performs test case design and test execution (incl. ,result Be evaluation and
b H
including functional and non-functional characteristics) m
c e sG
i
• Prepare and create test data s e rv
y
• Reviews test cases designed o og other testers
lby
n
t ech
ig
• nd
Assists in test reporting
r e
t
• 2
Assists in 23
0implementing test automation
©
• Perform “doing” tasks related to test automation and test infrastructure
* Tester is used as a generic term and may include various roles other than the test manager

angajaților
Who takes the tester role?

Different people may take over the role of tester at different test levels and
phases. People who work on test analysis, test design, specific test a ny) or
types,
test automation may be specialists. ( G e rm
n rl i
Be
• component / integration testing level H,
- developers
b
• system testing level Gm
- sindependent test team
e
• system integration testing level e r vic - independent test team
s
gy
• acceptance testing level: nolo - business analysts, users,
h
te c subject matter experts
n d ig
t re
• operational acceptance
3 testing level - operations and / or systems
© 20 2 administration staff
Roles being taken depending on the risks related to the product and the project,
and the software development lifecycle model selected.
angajaților
Summary
• The effectiveness of finding defects increases with the independence of the
test team. Independence comes in various degrees a ny)
m r
Ge
in (
erl and
• The test manager sets up the test team at an early state
,B
H
mb
o Plans all tests
G
o Creates a test strategy
ic es
e rv
o Organizes defect management
g ys
o
c h nol
o Organizes testware and configuration management
t e
ig
o Controls test execution
r e nd
t results
o Evaluates3test
2 02
© manager reports to stakeholders, to company management and to
• The test
the project leader
• The testers support the test preparation activities, execute the tests, create
defect reports and test results documentation. They also assists in the
implementation of test automation
angajaților

ny
Ge
Be
m
sG
• V/5 Risks and testing v ic e
ser
• V/6 Defect management
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
2. Test planning and estimation Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Test planning activities

Test planning is project planning Test Planning
ny )
ma
• All tasks and activities have to be r

(Ge Analysis
Test
planned in advance
rl in
Be
• For the different tasks defined, resources b H, Test Design
should be allocated (personnel, budget, s Gm
tools, test environment, etc.) r v ic e
e
ys
Test Implementation
• Summarizing test activities in
n o laog
ech
master test plan and tcoordinating them Test Execution
i g
nd
with the project plan
t re
023level of quality (e.g., testing depth)
• Defining2the Test Completion
for the©various test levels
• Test planning is a continuous activity, it needs to be controlled and adjusted
on an ongoing basis
• Feedback from test activities might impose adjustments on the master test
plan in order to deal with changing risks
angajaților
Test planning is part of the overall quality planning

• Test planning is an important part of quality assurance, but not the only
) part
m any
• Structure and contents of a quality plan can be found in IEEE r730
i n (G e
rl
• Elements of a quality assurance plan according to IEEE , Be 730: planning and
H
description of G mb
s
o Project organization r v ic e
e
o Documents covering the development g y s life cycle
o
o Standards, methods and c h nol
conventions as well as a mechanism to ensure, that they
are followed i g te
r e nd
t
0 3
o Reviews2and audits during the software development life cycle
© 2
o Testing
o Error documentation, remedies
angajaților
Master Test Plan (static) (German: Testkonzept)

• After defining the role of testing within the QA-activities, the test process starts
with its planning phase a ny)
m r
Ge
in (
e rl
• First step of planning is the creation of a static test plan
, B
bHof the testing process
o The master test plan covers all phases and activities
s Gm
r v ice resources, testing activities,
o Rules are laid down regarding test objectives,
milestones, etc. se y
o lo g
t e chn on extended to cover results from the detailed
• The master test plan is later
planning phase ig
nd
tre planning more information becomes available, planning can be
o As during 3project
0 2
done 2in more detail
©
o The master test plan gets a dynamic extension, which will be adjusted during project
life cycle, if necessary
o A template for a master test plan is given in ISO 29119

angajaților
Master Test Plan according to ISO 29119

1. Overview )
ny
2. Document specific information r ma
Ge
3. Introduction rl in (
Be
4. Context of the testing b H,
m
sG
5. Testing communication v ic e
ser
6. Risk register
lo gy
7. Test strategy hno
i g te c
t r endand estimates
8. Testing activities
9. Staffing
2 0 23
©
10. Schedule
angajaților
Test planning activities

• Test planning starts at the beginning of the development project and is
adjusted throughout the project life cycle a ny)
m r
Ge
in (
, B erl plan. These following
• Test planning also covers creating and updating the test
activities are explained in more detail: bH
Gm
o Test strategy es
e rv ic
o Resource planning
g ys
o Priority of tests
hn o lo
ec
o Tool support ig t
e nd
02 3 tr
© 2

angajaților
Test planning activities – test strategy

• Based on project characteristics, risks and goals, several test approaches
) can
n y
be considered ma r
Ge
• Test strategy describes the test levels to be performed rland the testing in (
, B e
intensity within those levels bH
Gm
ces for each test level, including the
• Test strategy also states entry and exitvicriteria
r
metrics to measure these criteriay se
g
h n o lo
• Because testing a system in full is not feasible, a test strategy is needed.
i g te c
Testing with all possible test data combinations, internal states and timing
constraints2is t r end
practically impossible
2 0 3
©
• Risk assessments help to focus testing activities on those areas, which bear
the highest risk of failure
angajaților
Test planning activities – resources planning

• The main goal of resource planning is to estimate the efforts of team )
anyetc.
members, including their time requirements, tools, support activities
m
e r
These estimations become part of the (dynamic) test plan n (G i
, B e rl
• The master test plan also has a detailed plan timetable
m bH including milestones,
assignment of people to activities. This plan Gis an instrument to manage the
i c es
overall task of test execution with alleits v
r activities
g ys
h n o lo
i g te c
t r end
2 0 23
©

angajaților
Test planning activities – priority and time management

• Prioritize tests: since releasing a product without sufficient testing bears ) a
high risk, it is necessary to prioritize test activities. This should be n y
adone in
e rm
G
in (
such a way, that the most important test cases are executed early. This way,
r l
e activities have to be
critical parts of the programs are tested even if the, Btest
aborted prematurely b H
m G
i c es
v intense time problems towards their
ser
• Manage time: Many projects experience
g y
final phases. This may lead tolodecisions on reducing test activities or skipping
tests completely h n o
i g te c
r end
• Select tools: tDeciding on which tools should be used for testing, are the
3
2 sufficient or are additional tools need
available20tools
©
• Document: Defining the level of detail, structure and templates for the test
documentation
angajaților
Test approaches / test strategy 1 of 2
The test approach is the implementation of the test strategy for a specific
project. The test approach is defined and redefined in the test plans a n y) test
and
designs. It typically includes the decisions made based on nthe( erm goal and
Gproject
rl i
risk management. , Be
m bH
There are a variety of different approaches to G
testing.
i c es
• Reactive er v
g ys
o First software / system design,
h n olothen test design (e.g., exploratory testing)
• Analytical i g te c
nd
tre prior to testing, e.g., requirements analysis or risk based testing
o Analysis is3 done
2 02
• Methodical
©
o Testing making use of a pre-defined list of taxonomies, quality characteristics, look-
and-feel standards or test conditions
• Consultative or directed
o external experts (domain, technology, etc.) guide the test process

angajaților
Test approaches / test strategy 2 of 2
Other approaches / strategies:

a ny)
• Regression-averse: Use test suites, tests and test data fromeprior rm projects in
( G
order to make fast advances, strong automation of regression
e rl in testing
• Process- or standard-compliant: Strategy ruled , Bsoftware development
Hby
b
standards, e.g., agile methods, industry- ors companyGm standards
i c e
• Model based: Testing on the basisse ofrvsome model for some aspect of the
y
system, examples of modelsocan
n log be business process models, state models,
h
reliability-growth-models
i g te c
t r end
2 023 may be defined
More approaches
©
• Failure focused: Error Guessing, fault attacks
• Preventive: Tests are designed as early as possible
• Check list based: Usage of check lists from prior projects or from test
planning
angajaților
Test approaches and strategy - consequences

The test strategy and the selected approaches are the baseline for defining
many further issues in the testing project (context specific): a ny)
m er
G
• A combination of approaches focusses on the specifics
r l in (of the project and
Be serves to find
H,
the assignment the team has, a well-chosen combination
b
multiple different defects Gm
ic es
erv for choosing:
• The chosen strategy is the foundation
s
y
logof done items
o entry- and exit criteria, definition
no
o test levels and types tech
d ig
re n
o test techniques
3t
2
• And as a20consequence, also:
©
o Test environments
o Testing skillsets

angajaților
Entry criteria – definition and examples

• Entry criteria define when to start testing such as at the beginning of a) test
level or when a set of tests is ready for execution. In Agile several nythese
aof
r m
topics may be considered within a definition of ready. n (Ge
rl i
, Be
• Typically, entry criteria may be: bH
o Testable requirements in any form available s Gm
ic e
o Test environment available and readiness s e rv
y
o Test tool readiness in the test o l og
environment
n
t
o Testable code availability ech
ig
r e nd
t
o Test data availability
2 0 23
o Availability
© of well skilled and trained people
o Testers are ready and prepared
angajaților
Exit criteria - definition 1 of 3
• Exit criteria indicate the end of a testing phase, testing level or activity.
In Agile this is typically called a definition of done. a ny)
m r
Ge
• Metrics are needed to control these exit criteria:
rl in (
Be
o planned tests have been executed
b H,
m
o a defined level of coverage (e.g., requirements,
c e s G user stories, acceptance criteria,
i
risks, code) has been achieved e rv s
y
o number of unresolved defectsloisgwithin an agreed limit
no
ech
o number of estimated tremaining defects is sufficiently low
ig
endof relevant quality characteristics are sufficient
o evaluated levels
02 3 tr
© 2
Exit criteria - examples
• Code coverage
o x% of program code have been executed
o x% of all functions/all menu selections have been covered

angajaților
Exit criteria - examples 2 of 3
• Risk coverage
a ny)
rm all been
o test cases of a predefined risk class (e.g., the highest risk level) have
e
executed successfully n( G
rl i
Be
H,
• Test abortion due to time, cost or quality reasons
b
o Test activities are stopped when delivery days is Gmreached or the test budget is
i c e
ev
exhausted. Very often this is reality in rprojects, often this costs a lot of time and
s
money later on gy
lo
o If minimum quality is notch no testing may be aborted or not even start (too many
met,
e
critical errors) ig t
t r end
023 rate
• Error finding
2
© of new errors discovered per hour falls below a predefined value, e.g.,
o Number
testing can be stopped, if less than one error per hour is discovered
o The economies of testing have to be taken into account. Beyond a certain error
discovery rate it might be a better option to release the software application to
production and concentrate only on fixing failures reported by the customers
angajaților
Exit criteria 3 of 3
• Economies of testing )
ny
o An increasing degree of quality means lower r ma
Minimum cost
Ge
cost of error, but higher error prevention costs
in (
of quality
rl
Be
o Cost of review initially increase, then decrease
b H, To
(less reviews are performed during the late m ta
sG lc
osts
project phases) i c e os
t
v
ser
of
on c
qu
l o gy then
o Total quality cost decreases initially, ali
ty
Costs
enti
c h no
increases again – quality costs are lowest Er
ro
prev
e
where the curve is in tits minimum rc
os
ig
ndto supply a certain t
r
o If it is necessarye
Erro
t r
minimum
2 023quality to be able to stay in
©
business, this can mean:
Tests have to be continued although
error prevention costs rise more than Degree of quality
error costs decrease
Source: Qualitative data from about 20
trendig technology services projects

angajaților
Test execution schedule

• Defines the order in which the test suites / -cases are executed )
ny
• The order of execution is influenced by the priority, dependencies, er efficiency ma
( G
in
as well as confirmation and regression tests. e rl B
b H,
• In practice there might be trade-offs between G mdependencies and priorities
e s
of test cases. Examples could be: v ic er
s
o A high priority test case depends
l o gy on the result of a test case of lower priority.
nomust be executed first
So the low priority test case
ch
o Confirmation tests te / or regression tests must run early in order to give
igand
d
en developers about the state of repair
feedback totrthe
3
202 might be grouped to increase efficiency (e.g., all test cases need the
o Test cases
©
same initial setup of the data base), which might result in a deviation from their
priority of execution
angajaților
Test execution schedule – exercise

The following test cases have been designed and assigned various priorities:
)
any
T1, T3, T5 (High priority)
(G e rm
in
T2, T7, T9 (Medium priority)
,B e rl
H
mb
T4, T6, T8 (Low priority)
G
ic es
e rv
Constraints coming from the test g ys
environment require test case T5 and T8 to be
l o
e c hno
tested last. Functional dependencies require test case T4 being executed
before T3 and T7 being t
ig executed after T6, also T2 need to be executed
n d
t re
immediately after T3.
3
2 02
©
What is a valid order that considers all of the above. (Constraints and
dependencies are more important than priorities)

angajaților
Test estimation – factors influencing test effort (extract)

• Characteristics (e.g., complexity) of the product )
ny
• Quality of the test basis r ma
Ge
r l in (
, Be
• Requirements on reliability and safety of the product
H
• Complexity of the development process es G
mb
ic
s
• Stability of the organization, maturityerofv used processes
gy o
h nol
tec pressure
• Personnel involved, time
n d ig
t re
• Team and leadership
3
02
©2
• Methods to estimate test effort:
o Metric based estimation (analogies and ratio)
o Expert based estimation (also: task-based approach)
angajaților
Metrics based estimation - analogies 1 of 5
• Method )
ny
o Categorize the required testing tasks r ma
Ge
r l in ( a testing task
o Find a project which has been executed in the past containing
similar to the one specified , Be
bH
Gm
o Use the real efforts for this task as a basis for the estimation
es
o By using metrics (burn-down charts of v ic
erprevious iterations, lines of code, number of
y s
modules, number of test cases,
value n o logetc.) as a basis, calculate the overall estimation
h ec
t
digfactors
o Allow for correcting
n
3 t re
© 20 2

angajaților
Metrics based estimation - analogies 2 of 5
• Advantages )
ny
o Method is simple and effective
er ma
( G
o With enough experience, quite accurate estimation values can
e rlin be achieved
, B
• Drawbacks m bH
o Need of experienced estimators and/oricaedetailed sG data base on actual project
v
data for the tasks to be estimated ser
l o gy
o Criteria to categorize projects
hn o may not cover all aspects of a project
e c
d ig t
o Often leads to discussions with management on the validity of the estimation
n
3 t re
0 2
©2
angajaților
Metrics based estimation - ratio 3 of 5
• Method
a ny)
rm activities
o Efforts for testing activities are estimated on the basis of overall project
( Ge
lin on experience
o The value for the ratio (fraction) needs to be determined based
er
B
o Example: Spillner/Linz talk about a ratio of 50% testing
b H, activities compared to the
m
overall project activities (see also: “Basiswissen
c e s G Softwaretest”, dpunkt Verlag, 3.
Auflage, S. 181) rv i se
gy for part of the work (e.g., estimation test
o This method may also be usedoonly
l test
n o
management costs, estimating
h efforts for systems test)
ec
ig t does not take into account regression test efforts, which can
o Ratio based estimation
d
n
tre part in maintenance testing and change related testing
be a substantial
3
2 02
©

angajaților
Metrics based estimation - ratio 4 of 5
• Advantages
a ny)
rm too much
o Very simple and powerful estimation technique, which does not need
e
input data n( G
rl i
• Drawbacks H , Be
G mb
o Not very accurate, since it does not take into s account particular project facts
r v ic e
e
y s by the estimator in order to derive valid
o A lot of experience and intuition is need
g
o
nol
estimations
c h
e may lead to difficult discussions
o Deciding on the ratio tvalue
d ig
n
tre activities, which are already part of project planning estimations
o Take into account
3
2
(e.g., is0developer test effort part of the development estimation or part of the testing
©2
estimations?)
o Ratio varies widely between new development projects and maintenance projects
angajaților
Metrics based estimation – other options 5 of 5
• in agile development
ny)
ma to feed into
o e.g., burndown charts, as a means to capture and report on effort, rused
e
the team’s velocity n (G rl i
Be
• in sequential projects
b H,
Gm and report on volumes of
o e.g., defect removal models, as a means to scapture
c e
defects and time to remove them rv i e
g ys
hn o lo
ec
ig t
e nd
02 3 tr
© 2

angajaților
Expert based estimation 1 of 3
• Method
a ny)
rm
o Identify all tasks to be performed (typically using a top down approach)
o Get estimations on all tasks by their owners or by expertslin (G e
, B (if there are experiences er
o Add up all values for the tasks. Include correcting H
factors
b
regarding the accuracy of certain estimators) Gm
es
rvic for overlooked or underestimated
o Include buffers/additional position toecover
s
y
tasks og ol
hn
te c
en d ig
02 3 tr
©2
angajaților
Expert based estimation 2 of 3
• Advantages )
ny
o Estimation activities can be closely linked to project planning r ma
Ge
r l in ( and adjusted
o Estimation creates a detailed data base which can be controlled
throughout the project life cycle , Be
m bH
o Tasks may be assigned to groups (e.g., small,
c e s G medium, large) and efforts are
rv i
estimated for a few group representatives only
e
g ys
• Drawbacks
hn o lo
tec and expensive
o This method is extensive
ig
end a clear idea at an early stage on test strategy and test
o This method rrequires
t
0 23
activities
2
©
o Experience shows that estimations are, in most cases, to low. This might be due to
overlooking certain tasks or coarsely underestimating tasks
o Build-in buffers are cut down during project planning activities
o Errors regarding project planning are inherited

angajaților
Expert based estimation – other options 3 of 3
• in agile development
a ny)
rm based on their
o planning poker, team members estimate the effort to deliver a feature
e
experience n( G
rl i
• in sequential projects , Be
bH
Gm
o Wideband Delphi, groups of experts providesestimates based on their experience
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
angajaților
Comparison estimation methods and development models
ny )
Agile Metrics Agile Experts er ma
( G
burndown charts planning poker erlin
B
• measured effort is used to • team members
b H, use their
m to estimate effort
determine the team's velocity sG
experience
r v ic e
e
Sequential Metrics y sSequential Experts
defect removal models n o lo g Wideband Delphi
• number of defects and t echtheir • groups of experts combine
removal times e digtracked and
nare their experience to estimate
t r
reported0 23 tasks
©2

angajaților
Summary
• Test planning is part of corporate quality planning )
ny
• The master test plan is the basic element of all test planning eractivities. It ma
( G
in
should be created early in the project e rl ,B
H
mb
o template of a (master) test plan: ISO 29119
e sG
e r vic methods. two common methods
• Test estimation can be done using various
s
are: gy lo
o metric based estimationchno
te
d ig
o expert based estimation
en
3 tr
0 2
©2
angajaților

ny
Ge
Be
m
e sG
• V/5 Risks and testing v ic
ol og
te chn
ig
tr end
3
© 20 2

3. Test monitoring and control Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Test monitoring and control

• Test planning: activities must be initiated Test Planning
ny )
ma

• Test monitoring: control of test activities in r
(Ge Analysis
Test
order to detect deviations form the plan
rl in
• Test controlling: correcting the direction of H , Be
test activities when necessary G mb Test Design
s
v ic e
• Monitoring should be done based on
y ser Test Implementation
measurable considerations olog
• Gathering of data either
hn
ecautomatic or manual Test Execution
ig t
n d
• decide whether
3 treexit criteria or DOD* is reached
2
20metrics
• groups of
Test Completion
©
o defect-based metrics (taken from the defect management system)
o test case-based metrics (taken from the test management system)
o cost-based metrics (e.g., taken from the project control system)
• The results measured are to be reported regularly for feedback and visibility
* DOD = definition of done
angajaților
Metrics examples:
• Defect based metrics (taken from the defect management system), e.g.
)
o defect detection rate any
o defects found / fixed (G e rm
in
o re-test results ,B e rl
H
mb
s Gmanagement system), e.g.,
• Test case-based metrics (taken from the test
e
v ic
ser
o coverage of test cases
y
o coverage of requirements
n o lo g
ech
o good / bad ratio of test cases
t
o code coverage ndig
t re
23
o risk coverage
20
©
• Cost-based metrics (often taken from the project control system), e.g.
o cost of finding errors
o regression test costs
o cost of external resources

angajaților
Test control
• Test controlling is a management task )
ny
o The test manager r ma
Ge
in (
erl activities
o The team in agile projects monitoring an adjusting their own
B
b H,
Gm from the plan
• Correcting measures as response to deviations
s
e
rvicthat are undertaken during testing
o Test controlling incorporates all measures
se
y
o l og and, where needed, initiating a new planning
o Adjustment of planned test activities
cycle in the project plan hn
i g te c
t r end
• Test closure evaluation
o Test exit 3
02criteria are also recorded with the test progress metrics
© 2
o Test exit criteria that were reached are documented in the test report for approval
angajaților
Test control measures

• Provide more resources )
ny
o More human resources (overtime, increase size of test team) r ma
Ge
o More budget (get external specialists / consultants)
rl in (
Be
o Tool deployment for automating tasks
b H,
m
• Reduce the work efforts e sG
v ic
ser
gy
o Leave out variations of test cases
lo
hno / omit testing of selected objects
o Simplify complex test objects
ofig
o Reduce amount d test data
te c
tr en
23test cases / test sets
o Leave out
20
©
• Test controlling measures are reported to inform the project management /
customer about changed risks for product deployment

angajaților
Test reporting 1 of 4
• Information about test activities is consolidated for test reporting )

ny
• Sample for the contents of a test status report (as per ISO r ma
Ge29119-3)
rl in (
, Be
o Overview and unique ID
H
mb
o Test status (incl. traffic light or weather conditions)
G
s
ic e
o Reporting period (incl. test level, test cycle)
s e rv
l o gy metrics, incl. test objects, resources
o Progress against the test plan (using
used/budget spent, number noof documented defects, number of test cases executed)
ch
e
ig t
o Factors blocking progress
e nd
3 tr
o Test measures
2
o New 20 changed risks (risk assessment)
© and
• Optional:
o Milestones reached (e.g., acceptance of test objects on particular test levels)
o Defects report (number of defects uncovered, number of defects corrected)
o Forecast: activities planned for the next reporting period
angajaților
Test summary report (test completion report) 2 of 4
Test progress reports )

ny
• summarize all project test activities performed r ma
Ge
in (
erl include
• are issued regularly for the stakeholders and might also
,B
H
mb
o status against test plan, impediments
o planned work for the next reporting periodces
G
i
e rv
y stest object(s)
o information about the quality of the
ol og
e chn
Test summary reportdig t
en
0 2 3 tr at the end of a test activity, e.g., the end of a test level or
• usually prepared
© 2of overall testing (when test exit criteria are reached)
the end
• test closure report at the end of the project

angajaților
Test reporting 3 of 4
• Reporting frequency
a ny)
rm are longer
o At the beginning of a project / in the preparation phase reporting cycles
e
(monthly, every 2 weeks) n( G
rl i
Be (weekly, even daily)
o The “critical” phases of test execution require shorter ,cycles
m bH
o Test closure report at the end of the project
sG
v ic e
• Evaluation of test reporting ser
o gy
nol
o Is progress made appropriate?
ch
e and efficient?
ig t
o Is test execution effective
e nd
3 trin line with the test goals, are the test goals being reached?
o Are activities
2
o How 20 is the level of confidence in the software based on current test progress?
© high
angajaților
Test reporting contents and audience 4 of 4
Contents of a report depends on the software development lifecycle, )

ny
organizational requirements and the project itself. ma r
Ge
rl
• More reporting needed in complex or safety criticaleprojects in (
,B
bH down charts and the
• Agile projects might use the task board, the burn
m
G
ces
stand-up meetings to make progress visible
vi
er
ys
og
nol test reporting
Target audience also influences
ch
te trends, defects distribution and defect types is
• Detailed informationigabout
d
n staff only
t re
relevant for technical
23
0 needs a summary report comprising status of risk coverage,
©2
• Management
defect repair, summary of defects, budget, schedule and test progress data
(e.g., passed / failed)
Templates of test progress and test completion report are available in ISO
Standards (ISO 29119-3).

angajaților
Summary
• Test progress monitoring is based on measurable criteria and provides
) the
n y
information needed to manage the test process ma r
Ge
• Deviations from the plan require correction measureserl in (
B
H,
• Regular test reporting informs the project and
G mbthe company management
es
about the testing progress v ic er
s
gy
hn o lo
te c
en d ig
02 3 tr
©2
angajaților

ny
Ge
Be
m
e sG
• V/5 Risks and testing v ic
ol og
te chn
ig
tr end
3
© 20 2

4. Configuration management Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Purpose
• A large amount of data / information / results (artifacts) are produced) during
ny
software development, e.g.: ma r
Ge
o requirements, specifications, system design documents in (
B e rl
H,systems
o individual components, integrated modules, complete
b
m
o test data, test specifications, test results
e sG
e r v ic
• A large number of participants with s different roles working on the variety of
l o gy
hno
system components
e c
• Configuration management
d ig t is responsible for the naming conventions of all
r e n
artifacts and t their administration
for
2 0 23
©
o successive versions numbers are assigned
o clearance for further development is recorded
o old versions are archived for future control
o access to the artifacts is recorded
angajaților
General remarks
• Configuration management has a supporting role within a project. )All
changes must be recorded at a common place and communicated m anyusing a
e r
defined process n (G i
, B e rl
• Depending on the type and scope of a project, the
m bHexpectations on
configuration management might stronglysvary G – a specific configuration
i c e
management plan must be made er v
g ys
o
• Configuration management
c h nol is not a particular testing activity, it is needed
throughout all project te
ig phases
d
• Configuration 3 re n
tmanagement without an appropriate tool is only possible on
2 0 2
very small
© projects

angajaților
Configuration management
CM refers to a set of measures that supplement software development:)
m any
r
• Change management follows all activities, e.g., changes on
i n (Gethe source code,
for each change request rl
, Be
H
• Build management describes all steps leading G mbto creating a software version
to be delivered, concerning the software es a whole or individual sub-systems
icas
e r v
g y s definition of isolated versions of each
• Release management enables o the
c h
artifact composing a complete
nol version of the product to be tested, be
te
delivered, etc. ndig
3 t re
2
0management (as part of CM) records all access information for
©2
• Versions
each artifact: current version number, last change, last user, etc.
angajaților
Requirements on CM from the testing point of view

For all test items CM ensures the following: )
ny
• Unique IDs for all testware items r ma
Ge
o identify all documents and software items unambiguouslyerl in (
H ,B
• Version control mb
e sG
rvic versions of an object (V1.0, V1.1 etc.)
o Categorizing, saving and recalling different
e
g ys
• Keep track of changes
hn o lo
ec document, who changed it, restore old versions
o what was changed intthe
ig
nd
t re
o Protocols,3comments and reasons for changes made
2 02
©
• Maintain traceability throughout the test process
o relate the testware to the correct versions of the test items
This is valid for all artifacts being produced in the software lifecycle, including
testware!

angajaților
Summary
• Configuration management is needed to administer changes on test) objects
ny
and their respective versions ma r
Ge
in (
e rl
• Build- and release information is stored in order to reconstruct older releases
B
H,
mb software development
• Configuration management applies to the complete
G
process, not only to the test process vices
y ser
g
• Configuration management islohardly possible without the appropriate tools
h n o
tecuniquely identified, version controlled, tracked for
• All items of testwareigare
changes and related
t r end to each other as to versions of the test object so that
023can be maintained throughout the testing process
traceability
2
©
angajaților

ny
Ge
Be
m
e sG
ol og
te chn
ig
tr end
3
© 20 2

5. Risk and testing Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Risk
• Risk (from: German Wikipedia)
ny)
main case of a
o A risk is a calculated prediction of a possible damage respectively rloss
e
G
negative outcome (danger) or a possible advantage respectively
r l in ( gain in case of a
positive outcome (chance) Be
H,
G mb
o Risk is the probability of a negative outcome (mathematical), or the probability of a
s
negative event happening multiplied by theiceamount of financial damage (economic)
v
y ser
• Risk (from: “Waltzing with bears”, g Tom DeMarco/Timothy Lister)
h n o lo
o A possible future eventec
i g t that will lead to an undesired outcome (cause) respectively
this undesired outcome itself (effect)
r e nd
t
023 product risks should be taken into account when planning and
• Project-2and
© test cases, when prioritizing test cases, when choosing methods
designing
and while executing the tests
angajaților
Risk assessment 1 of 3
Categorizing risks allows comparing the risks with each other. This can) be done
ny
using a risk matrix to determine probability and damage. ma r
Ge
• Example for risk matrix:
rl in (
Be
b H,
m
sG
Probability of occurrence
e
v ic
y ser
3: „high”/ >40% 3
ol og
te chn
d ig 2
2: „medium” / 10 - 40%
n
3 t re
©/ <10%
1: „low” 20 2 1
Damage caused
Factors 1 2 3 1: „low” / <0.1 Mill. €
2: „medium” / 0.1 - 1 Mill. €
3: „high” / >1 Mill. €

angajaților
Project risks 1 of 3
Possible negative impact on project success )

ny
• Project issues
er ma
G
r l in (
o Insufficient cooperation between departments / conflict of interests
Be
o Unrealistic project schedule and estimates
b H,
o Changes regarding legal requirements
s Gm
e
o Rework due to late changes rv ic se
y
lo g
hno
• Organizational issues
ec availability of staff
o Qualification, trainingtand
d ig
n between teams / team members
t re
o Personal problems
3
o Wrong 2
0business priorities leading to lack of user involvement
©2
o Concurrent access to external resources
angajaților
• Political issues
y)
o Inadequate communication of test results and needs by the testers man
o Failing to follow up on information found in testing and reviews ( G er not improving
(e.g.,
rl in
development and testing practices) , Be
H
mb
o Improper attitude towards, or expectations of, testing
G
ic es
• Technical issues s e rv
y
o Wrong, incomplete or unrealistic
n o logrequirements
ech methods, tools, etc. for software dev.
o New or uncertain technology,
t
dig of the work products
o Shortfalls in thenquality
e
t r
023 of complex test environment
o Availability
2
© data conversion, migration planning and their tool support
o Late
o Accumulated defects and other technical debt due to e.g., poor defect management
• Supplier issues
o Shortfall in externally provided components (time, quality, cost)
o Acceptance problems and other contract issues with vendors
angajaților
• Project risks influence the success of a product, they have to be managed

)
any
• Estimate probability and potential damage G e rm
r l in (
• Implement appropriate measures to deal with the identified Be risks:
b H,
o Risk mitigation (actively prepare measures to G m probability and / or potential
reduce
e s
damage) ic
s e rv
o Risk control (prepare for measures y
and money available) hnol
og necessary if risk becomes a problem, have time
c
tethat
o Risk ignorance (hope,
d i g the risk does not become a problem, pray, keep fingers
n
t re
crossed, etc.)
3
0 2
©2
o Risk transference (moving the risk to another area / organization)
o Risk avoidance (avoid the risky situation)
When analyzing, managing and mitigating these risks, the test manager is
following well established project management principles. The ISO 291119
outline for test plans requires risks and contingencies to be stated.
angajaților
Product risks
• Product risks result from problems regarding the supplied product )
any
e rm
o Insufficient functionality of the supplied product
o Insufficient non-functional attributes (quality risks) in (G
,B e rl
o poor data integrity and quality H
mb
s G it cannot be brought into operation
o The product is not fit for its intended use; ehence
c
v i
(e.g., user experience / UX) er s
y
ogproperty
o The product causes damage lto
no
ech regulations, standards and norms
o Non-compliance withtrules,
ig
end accidental bodily injury or death
o The product rcauses
t
2 0 23
• Testing
© is done to reduce product risks
o Risk = probability of occurrence x potential damage
o Testing reduces the probability of error occurrence
o For high potential damage more intensive tests are need

angajaților
Managing product risks 1 of 2
• Managing product risks using risk-based testing )

ny
o Identify, analyze and prioritize risks
er ma
( G
o Risk influence already taken into account during test planning
e rl in
§ Select test methods to mitigate risks , B
m bH
§ Assign test scope (depth) according to risk level G
s
r v
§ Adapt execution order of test cases (importantice test cases first, in order to find critical defects
e
early!) g ys
o
o Update the risk assessment c h nolworksheet frequently
e
§ Risks may disappear
d ig t(vendor did supply in time)
§ New risks 3 renarise (customer orders additional functions)
tmay
20may2
§ ©
Risks change (influenza epidemic)
angajaților
• Risk is used to focus the effort required during testing. High risk test object are
tested more intensively. a ny)
m er
G
r l in ( of bugs occurring
• Testing reduces the product risk by reducing the probability
Be
at the customer's side. H, b
m
• In a risk-based test approach, product risk
c e s Ganalysis results are used to
i
determine: e rv s
y
log (e.g., systematic, heuristic)
o the test techniques to be employed
no
o the particular levels and h
ectypes of testing to be performed (e.g., security testing,
i g t
nd
accessibility testing)
t re
023of testing to be carried out (e.g., scope, depth)
o the extent
2
o the©order in which tests are executed (prioritize testing, trying to find the critical
defects as early as possible)
o whether any additional activities could be employed to reduce risk (e.g., providing

angajaților
• Benefits of risk-based testing:

a ny)
rm identified risks
o Test techniques, types, levels are chosen particularly to mitigate the
( Ge
rlin way, test efforts are
o The scope of testing takes care of the identified risks. In this
e
focused to address risk reduction potential ,B
bH
o Risky defects are discovered early, thus making
s Gmit more economical to correct
e
them rv ic se
y
o l og it is ensured that the most important test cases
o Even in the case of a test abortion,
hn
have been executed (risk-based prioritization of tests)
ec
t
en d ig
3 tr
0 2
©2
angajaților
Risk management and mitigation

• All relevant stakeholders should take part in risk identification and estimation
)
any
• Four main activities:
(G e rm
in
o Risk identification
,B e rl
H
o Risk assessment
G mb
o Risk mitigation ic es
e rv
o Risk management g ys
hn o lo
ec
ig t
e nd
02 3 tr
© 2

angajaților
Summary
• Project and product risks endanger the success of the project, they) have to
ny
be managed ma r
Ge
in (
erl risks
• Risks can be organizational, technical or environmental
B
b H,
• Risk number = probability of occurrence times
Gmpotential damage
vi ces
ser
• Risk mitigation: four basic principles
gy and / or impact (e.g., by testing)
• Preventive measures to reduce likelihood
o lo
chn impact if the risk becomes reality
• Make contingency plans to reduce
te
dig other party to handle
• Transfer the risk to some
en
3 tr the risk
• Ignore or accept
© 20 2
• “Risk management is project management for adults”*
* “Waltzing with bears”, Tom De Marco / Timothy Lister
angajaților

ny
Ge
Be
m
e sG
ol og
te chn
ig
tr end
3
© 20 2

6. Defect management Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Finding defects during testing

• Testers execute specified test cases and record the results )
ny
• they analyze the deviations between expected and actual G er
results ma
rl i n(
, B e
o Defects and or failures are identified (e.g., defects in documents, in code, in output
data of a test object, in a help text) bH
Gm
r v cesmay fail when a network connection is
o The reporting of false positives (e.g., a itest
se
broken or times out) should be minimized
y
o l og
n created and filed in the defect management
e h
• The defect report shouldcbe
system with all relevant t
ig information
r e nd
t
023in time, the task of the tester is finished for the time being
• At this point
2
© waits for the corrected version of the program to perform a confirmation test
o Tester
(retest)
• The further tracking of the error is done using a defect management system
angajaților
Who does what? 1 of 2
• Tester )
ny
o Executes test cases to discover failure r ma
Ge
o Records the results in a test log in (
, B e rl
o Enters the defect into the defect management system
m bH and assigns initial values to
defect characteristics based on the agreements
c e s G the organization and / or project
in
i
• Test manager s e rv
y
o Evaluates the problem report n o lo g
t ch
edefect
o Assigns priority to ithe
g (in accordance with project management, customer,
n d
etc)
t re
o Writes20 23 progress reports on the basis of current state of correctional work
work
©

angajaților
Who does what? 2 of 2
• Change Control Board (CCB) or in agile Triage )

ny
o Decides on changes of the requirements and their priorities r ma
Ge
• Developer or author of the relevant artifact rl in (
Be
o Analyzes failure, locates defect cause b H,
m
o Corrects defect cause in accordance with e sG
icassigned priority
v
o Performs all approved changes gy ser
o
nol
ech
• Note: these tasks are tperformed in an iterative manner:
o Tester en d ig
3 tr
20 2
o Test manager
©
o Change Control Board (CCB)
o Developer
o Tester …
angajaților
Purpose of defect reports

• Provide information about any unexpected or undesired behavior for quick
and simple isolation in order to solve the problem a ny)
m r
Ge
rl
• Provide for a means of tracking the quality of the workeproduct and the impact in (
, B
on the testing, e.g.: bH
Gm
es more confirmation testing, instead of
o Spending time on lots of reported defectsicand
r v
running tests sey
n o lo g
• Provide ideas for development and test process improvement
t ech
ig
r e nd
t
2 0 23
©

angajaților
Structure of a defect report

• The defect report describes the failure, not its cause; or a found defect) (e.g.,
n y
during static testing) ma r
Ge
in (
e rl
• A template for a defect report is found in ISO 29119 (incidents)
,B
H
• Details of a defect report may include: mb
e sG
o Defect data
s e rv ic
gy
§ Unique defect number (usually automatically
lo
generated)
o step
hntest
§ Test object (name, version),
§ Test environment ig te c
nd
tre report author
§ Name of3defect
2
20of first occurrence
Date
§ ©
o Classification of the defect

§ defect class (also: defect severity)
§ defect state (new defect, retest, etc.)
§ Priority (assignment of urgency)
angajaților
Structure of a defect report

• Details of a defect report may include: )
ny
o Description r ma
Ge
§ Test case (provides all details about preconditions) in (
B
§ defect result / failure mode (using description of actual and, expected result)
e rl
bH
m reports, screen shots, application
sG
§ Description of the deviation to enable resolution (including
error messages, etc.) i c e
v
§ Cross references to related reportsy ser
ol og
chn
§ Comments
e
ig ttaken
§ Corrective measures
tr end
23
o History Log
20
§ ©
Time and user to perform changes
§ Most systems automatically track changes in the
life cycle of the defect

angajaților
Defect class and defect priority

The severity of a failure is expressed by assigning a defect class
(synonymous: failure class) a ny)
m r
Ge
rl Three or four
• Defect classes used: critical, major, medium, minor defect. in (
, B e
classes are usual bH
s Gm
• Basis for the classification can be the v ic e
influence on the product usability
y ser
The priority takes into accountlo g effect of the failure
the
h n o
• Impact on the program
c
tefunctionality
i g
t r end
0 23
• Impact on the project, on the customer
© 2
• Possibility to provide an immediate fix or correct the problem within the next
release
Priority rules the urgency of the correction
angajaților
Defect state 1 of 3
• The defect state gives information on the progress of work that has been
performed for this defect a ny)
m r
Ge
in (
, B e rl
• The representation of defect states can be different depending on
bH
organizations, products, projects, and software development models
m
G
cesto:
• Possible states include, but are not limited
rv i
e
y s system
o New – tester entered defect into the
g
o (by test manager or developer)
nol
o Open – problem report confirmed
ch
te rejected (by test manager or developer)
ig report
o Rejected – problem
e nd
3 t–r Developer tries to identify the defect
o Inspection
02
©2
o Surveillance – Defect cannot be reproduced, it is under surveillance
o WorkInProgress – Defect is located and cleared for correction
o Retest – Developer has corrected the error cause
o Closed – Tester has verified correction by performing a retest
o Reopen – Tester disapproved the correction, defect still there

angajaților
Defect state 2 of 3
• Typical states and transitions for a defect management workflow )

ny
r ma
new rejected Ge
rl in (
Be
b H,
m
open
e sG observing
se rv ic
y
no lo g
analysis
h
te c
en d ig
02 3 tr in progress
©2
not solved /
retest closed
reopen
Remark:
Number of states supported by tools varies widely (could be three, could be twenty)
angajaților
Defect state 3 of 3
• Only a tester can mark a defect as “closed”! )

ny
• Usually, the test manager decides if a defect is to be corrected eror rejected – ma
( G
alternatively the change control board may decide on etherlincorrection of a
, B
bH implications
defect considering the cost of repair and the business
s Gm
iceto be recorded within the defect
• All changes (including commentaries) vare
r
e
management system ys
l og
hno of defect correction is ensured
o Ongoing control on the status
ec
ig tcan be planned
o Further test activities
r end
o Sometimes,
0 2 3 tadditional test cases must be generated in order to track down the
cause2of a failure
©

angajaților
Defect reports analysis

• All defect reports are analyzed in a systematic manner in order to assess
progress of defect correction activities, project plan conformance a ny) software
and
quality: ( G e rm
n rl i
• Typical points of attention are: H , Be
G mb
o Is a reduction in the number of newly foundesdefect visible? Or is the number
ic
increasing during the project life cycle?
s e rv
y
l
o Are particular test objects showing
o og a high number of defects? Are any test objects
n number of defects?
ech
showing a lower-than-average
t
ig
o How many severe
r e nd defects/high priority defects are still open?
t
o How long
2 023does it take for a defect to get fixed? What is the average time for defects
© fixed?
to get
Defect management tools offer a wide variety of reports on defect statistics
angajaților
Exercise V.1: Defect management system

)
• Design a simple screen layout of a defect management system m any
r
i n (G e
• State the minimum requirements for such a system Ber l
b H,
m
c e sG
i
s e rv
y
n o lo g
t ech
ig
r e nd
t
2 0 23
©

angajaților
Summary
• Defect management is the management of the deviations / defects )found
ny
during testing ma r
Ge
• Defect management is a process of its own, consisting rl of a particular in (
, B e
workflow bH
Gm
• For defect management powerful tools es available, which also cover the
icare
r v
tasks of change management y se
g
h n o lo
• The expressions deviation / incident / problem management are
i g te c
sometimes used nas a synonym for defect management
t r e d
2 0 23
©
angajaților
Keywords
• test estimation
• configuration management )
• test manager ny
• defect management
• test monitoring(Ger
ma
• defect report r l in
, Be
• test plan
• entry criteria
•G
bH
m planning
test
s
• exit criteria
r v ice • test progress report
• product risk y se
ol og • test strategy
• project risk
te chn • test summary report
ig
• risk
tr end • tester
• risk level2023
©
• risk-based testing
• test approach
• test control

angajaților
ny )
r ma
Ge
rl in (
Be
b H,
m
sG
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2
VI. Tool support for testing »

angajaților
Chapter VI. Tool support for testing

• VII/1 Test tool considerations )
ny
• VII/2 Effective use of tools r ma
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

angajaților
Learning objectives for tool support for testing:

6.1 Test tool considerations
• FL-6.1.1 (K2) Classify test tools according to their purpose and therm a
test
ny)
G e
activities they support n( rl i
• FL-6.1.2 (K1) , Be
Identify benefits and risks of test automation
bH
• FL-6.1.3 (K1) Remember special considerations
s Gmfor test execution and test
e
management tools rv ic se
y
6.2 lo g
Effective use of tools
o
n principles for selecting a tool
• FL-6.2.1 (K1) Identify e chmain
the
igt
• FL-6.2.2 (K1) nd the objectives for using pilot projects to introduce tools
Recall
t re
0 23
• FL-6.2.3 (K1) Identify the success factors for evaluation, implementation,
©2 deployment, and on-going support of test tools in an
organization

angajaților

ny
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

1. Test tool considerations Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
General remarks
• Test tools may be used to support test activities )
ny
o test execution support is referred to as test automation r ma
Ge
o test tools may also support other test activities
rl in (
, Be
bH tools, test data generation (or
§ Tools that are directly used in testing such as test execution
preparation) tools and result comparison tools Gm
s
v
§ Tools that help in managing the testingrprocessice such as those used to manage tests, test
e
results, data requirements, incidents,
g y sdefects, etc., and for reporting and monitoring test
o
execution
c h nol
§ Tools that are used tote
d ig investigate, explore and evaluate test and test results
n
tre aid in testing (e.g., spreadsheets)
§ Any tools that
3
0 2
© 2 are named after the type of support they provide
• Test tools
o tools are available for each level and activity of the testing process

angajaților
Meaning and purpose of tool support for testing

• Tool support for testing can have one or more of the following purposes
depending on the context: a ny)
m er
G tasks and
r l in (
o Improve the efficiency of test activities by automating repetitive
, Be
supporting manual test activities like test planning/design/reporting
H
G mb
o Automate activities that require significant resources when done manually (e.g.,
e s
static testing) ic
e rv
o Increase quality of test activities
g y sby making defects reproducible, testing more
o
repeatable and less prone ntool(typing) errors
ch
te can not be executed manually (e.g., reliability testing,
o Automate activitiesigthat
n d
t re
load and performance
3
testing)
2 02 reliability of testing (e.g., by automating large data comparisons or
o Increase
©
simulating behavior)

angajaților
Test tool classification 1 of 2
• Tools used for special tasks vs. test tool suites )

ny
o Single tools support one particular test task or activity
er ma
( G
o Tool suites cover several tasks and integrate several single
e rlintools
B
• Intrusive test tools vs. test tools that do not alter H, test object
bthe
G m
o Intrusive tools may interfere in the execution i c es of the test object and may cause it to
v
y ser
differ from the object in the true environment (probe effect)
o g
c h nol and alter the interrupt handling
§ Debuggers introduce breakpoints
e objects with artificial (input) data
§ Test drivers provide ttest
d ig
t r en
§ Coverage is determined by counters that are added to the code
o This is 0
2 23always desired
not
§ ©During performance testing, the test object must work as close to the real environment as
possible
§ During system testing, test objects must be embedded in a real time environment

angajaților
Test tool classification 2 of 2
• Test tools can be classified based on several other criteria )

ny
o Commercial, free, open-source, shareware, technology used r ma
( Ge
lin associated
o Classified under the activity with which they are most closely
er
B
H, activity
o Some tools support one activity other more than one
G mb
o Packages from a single provider that have
i c esbeen designed to work together (tool
suites) er v
g ys
• In house developed tools n
h o lo
e c
ig t
o e.g., Excel spreadsheets
nd
t re
o e.g., SQL 3scripts
20 2
o e.g.,
© databases for handling test data
o e.g., specific test result comparisons tools

angajaților
Tool support for management of testing and testware 1 of 3
• Test management tools )

ny
o Collecting, categorizing and administration of test cases r ma
Ge
o Evaluation / set of metrics describing the test cases
rl in (
Be
o Time and resource planning, budget planning H,
G mb
estests, documenting tests
o Creating test progress reports, evaluating
ic
erv tracking tools and requirement
o Interfacing to test execution tools, sdefect
y
management tools lo gno
h
tetoc the requirement specification (consistence)
o Tracing the test object
ig
d
t re n
o (Release management / configuration management)
2 0 23
• Requirements
© management tools
o Gathering requirements and their attached attributes
o Prioritizing requirements
o Referencing requirements to test cases for consistency checks
o Identify inconsistent requirements and / or that they are missing

angajaților
Tool support for management of testing and tests 2 of 3
• Defect management tools

a ny)
rm
o Recording and tracking incidents/defects/failures/anomalies etc.
o Also store and manage change requests in (G e
,B e rl
o Prioritizing, categorizing and sorting defects H
G mb
es of progress of testing
o Evaluations, i.e., metrics showing the degree
ic
v
ser changes of status, responsibility
o Workflow for the life cycle of a defect:
y
n o lo g
• Configuration management tools
t ech
ig versions of components:
o Tracking the different
r e nd by a particular version, operating environment, compiler in use,
3t
requirements met
etc. 202
©
o Version management of testware, configurations and other tools
o Source code and object code version administration
o References to test management / requirement management / change management

angajaților
Tool support for management of testing and tests 3 of 3
• Continuous integration tools (mainly used by developers) )

ny
o Process the creation of builds (build pipeline) r ma
Ge
o Can manage multiple tools in a tool chain, incl.: in (
B e rl
§ Static analysis
b H,
§ Automated builds (including compilation)
s Gm
ic e
erv system where appropriate
§ Automated unit tests
§ y s
Generation of defects in a defect management
§ Further automated tests (incl.o l og
regression tests)
c h n
§ Automated deployment te in further environments
en d ig
02 3 tr
©2

angajaților
Tool support for static testing

The most cost-effective way of preventing and finding defects is as early) as
any
possible in the development process with the help of static testingmtools.
r
Ge
• Static analysis tools in (
B e rl
H,coding
o Compliance with coding standards and also secure
b
Gm dead code) and data flow
o Analysis of code structure via control flows(e.g.,
i c e
e rv
(e.g., bad variable use) analysis, complexity, link check of HTML or XML code
• Model-based testing toolsolog
ys
h n
tec / consistency check
o Analysis of data models
ig
d
re n
o Analysis of specification
3t
documents / object design models / state diagrams
202 test cases based on the software model (optional)
o Generate
©
• Prerequisites
o The specifications are provided as a formal language documents
o Close integration into the software development process, thus mostly seen as a
development tool

angajaților
Tool support for test design and implementation

• Test design tools are used to generate test inputs or executable test) and / or
test oracles from requirements, graphical user interfaces, design m any
models or
e r
code. n( G
rl i
, Be
bH
• Test data preparation tools derive data from s Gmformal descriptions / structure
ic e
e rv
definitions like data base design, specifications, interface design or code
s
gy
o lo
o Will mask the data to ensure data security
hn
ec
gt
o Data generated automatically will often need to be reworked manually
di
n and scramble it before use (data masking)
t re
o May use real data
3
02
©2

angajaților
Tool support for test execution and logging 1 of 6
• For all test levels, tools may be introduced to support test execution )
m any
• Test execution tools may cover the following: r
Ge (
in
o Delivering data
,B e rl
o Receiving data or writing logs of output behaviormbH
G
o Documenting test execution ic es
s e rv
y
loglogging tools:
• Examples of test execution and
o Test robots ec hno
o Test executionen d / debugger
tools ig t
3 tr
202 / test framework tools
o Test harness
©
o Test comparators
o Coverage measurement tools (code, requirements)

angajaților
• Test robots )
ny
o May address external interfaces of the test object directly
er ma
( G
rl in
o May accept and / or supply data, the test progress runs automatically
e
, B
m bH
o Often provide a function to compare actual with expected results
G
o Often capture / replay tools are used as test
i c es robots. They record test execution
v as a script file
s r
steps via the user interface and saveethem
g y
o Allow for automatic repetition
h n oloof the test sequence, using the recorded script
i g tec testing and exploratory testing
o Well suited for regression
t r end
2 0 23
©

angajaților
• Test execution tools – debugger )

ny
o Tool for finding errors in a program code r ma
Ge
o The sequence of program execution can be interrupted in (
, B e rl
o Single statements and conditions can be checked H
G mb
s
o Variables can be defined individually and referenced
r v ic e
e
• Test comparators g ys
o
o They compare expectedhand
c nolactual results based on files or databases of different
e
formats
d ig t
n
treto be compared are selected using filter functionalities
o Relevant data
3
0 2
© 2part of larger test-frameworks, but may be also a stand-alone tool
o Often

angajaților
• Test harness / test framework tools )

ny
o The testing of components or parts of a system
er ma
( G
e r n
o Simulating the environment in which the test object will run,lithrough the provision of
mock objects will run like stubs or drivers , B
m bH
o They are a replica of the productive environment
s G (or a part thereof) and are
needed, when security considerations e
icprevent the use of the target productive
s e rv
environment y
o l og
c n
o The representation of thehproductive environment should be as close as possible
e
ig t
o If they focus on adcomponent test, they may be called unit test framework
r e n
t
2 0 23
©

angajaților
• Drivers
a ny)
rm
o enable access to the test object, when interfaces have not been implemented
o regulate data input, data output and log the test progress in (G e
,B e rl
o record actual results H
G mb
es
o often provide their own system environment
ic
e rv
• Stubs g ys
o simulate functionality of h
ann evoked componento lo
t ec
n d ig
3 t re
© 20 2

angajaților
• Coverage measurement tools )

ny
o may be intrusive or non-intrusive
er ma
( G
o measure the percentage of specific types of code structurelthat
e r in have been
exercised by a set of tests , B
m bH
o count statements, decisions, conditions, paths,
s G modules or function calls
ic e
o Check the features of these tools in the
s ervtopics on static and white-box testing
y
o l og
n
t ech
ig
r e nd
t
2 0 23
©

angajaților
Tool support for performance measurement and 1 of 3

dynamic analysis
y)
• they support or automate dynamic analysis and testing tasks rman
i n (G e
• they are named according to their use er l
H ,B
o Dynamic analysis tools
G mb
o Performance / load / stress testing tools ices
e rv
g ys
hn o lo
ec
ig t
e nd
02 3 tr
© 2

angajaților

dynamic analysis
ny )
• Dynamic analysis tools
er ma
G
o Find defects that are evident only when software is executed
r l in (
o Find time dependency defects or memory leaks H, Be
o Memory was allocated but not released G mb
s
o v ic e
Find defect in pointer allocation or arithmetic
r
e
o g y s of systems
Important for multi systems or systems
o
o h
Typically used in component,
c nol component integration testing or middleware
te the internal state of the test object
iglog
testing, control and
e nd
02 3 tr
©2

angajaților

dynamic analysis
ny )
• Performance testing / load testing / stress testing tools
er ma
G
o Monitoring, measuring and reporting system behavior underlina (variety of simulated
e r
B
frequency, and relative percentage of transactionsb H, their ramp-up pattern,
usage conditions, e.g., of number of concurrent users,
m
sG
r v ice set of transactions, spread across
o Creating virtual users carrying out a selected
various test machines commonly yknownse as load generators
og
ol parallel user transactions or network traffic (it’s not
o Generate a synthetic loadhn
like
e c
g tresources)
possible with human
di
n
t re
o Finding “bottle-necks”
3
2 02
©
Apart from the tools that support general testing needs there are tool for
specific testing of the non-functional requirements:
e.g., usability, accessibility, security, efficiency

2. Effective use of tools Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Special considerations for some types of tool 1 of 5
• Capturing test approach (linear scripting) )

ny
o Scripts are captured by recording actions of a manual tester r ma
Ge
o Control activities and data items are interlaced, which makesrl maintenance difficult, in (
especially if a large number of test scripts is used H, B e
o Scrips are likely to be unstable when unexpected G mbevents occur
s
v ic e
ser
lo gy
ec hno
t
en d ig
3 tr
0 2
©2

angajaților
• Data-driven testing approach

a ny)
o Scripts run program functions of the test object. The script looks e rmdata on an
for
( G
external file / spreadsheet / database in
o Testers wishing to execute new or changed test cases, , erl not need to write new
Bdo
scripts but rather adapt the external file m bH
c sG
ethe
o Changes in data or on the GUI might alter v i reaction of the test object, processing
s er
problems might occur y
n o lo g
• Other techniques employed t ech in data-driven techniques
ig data: Data is generated using algorithms based on
r e nd
o Instead of hard-coded
t
023 parameters at run time and supplied to the application
configurable
2
© may use an algorithm, which generates a random user-ID, and for
o A tool
repeatability in pattern, a seed is employed for controlling randomness

angajaților
• Keyword-driven / action word testing approach

a ny)
o Scripts are modularized down to atomic user interactions with the
e rmtest object.
n( G
Extremely flexible test sequences can be created without editing the scripts
rl i
Be
o Test data and invoked functions are saved externally.
b H, A control script evaluates
m data
these and invokes the particular functions with their
e sG
rvic for writing the script
o In the beginning a programmer is necessary
se
y
o l og without knowing the scripting language only using
o Testers will be able to define tests
the keywords hn ec
t external data will grow fast in complexity
d ig
o Problem: the necessary
en
3 tr
2 0 2
For both©techniques, the expected results for each test need to be stored for
later comparison

angajaților
• Performance test tools are used mostly on applications, which are distributed
and which communicate via networks a ny)
m r
Ge
in (
• In most cases, the test environment cannot be completely
, B e rl isolated and is
bH in detail at the time of
subject to the influence of factors that are not known
m
preparing and executing tests sG e
v ic
• The complexity of the environment ser
may make it impossible to repeat
l o gy
o
identical tests (results arenhardly comparable)
h
ec
• In many cases, detailed ig t
expert knowledge is needed to analyze the tool
r e nd
t
output correctly
23 and to draw the right conclusions
0
©2

angajaților
• Test management tools )

ny
o Information must be kept openly accessibly
er ma
( G
o A spreadsheet is the tool most commonly used by the testlmanager
e r in for evaluations
and reports , B
m bH
o The reports and evaluations should adapt to the
s G organization, not the other way
around! ic e
s e rv
y
o l og
n
t ech
ig
r e nd
t
2 0 23
©

angajaților

ny
Ge
rl in (
Be
b H,
m
e sG
v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

angajaților
Potential benefits and risks of tool support for testing 1 of 4
• The use of test tools causes cost and efforts )

ny
o selecting and / or providing the appropriate tool r ma
Ge
o developing the required tool skills
rl in (
Be
o installing the tool in the system environment H,
G mb
es
o possibly adjusting the tool or parameter settings
ic
v
ser
o ensuring system operations, administration efforts
lo
o changeover time preparingodifferent
gy
tests
hn
ec the tool
gt
o time and effort of operating
i
r e nd
t of using a tool must outweigh these costs
0 23
• The advantages
2
© / benefit analysis for a tool deployment must be done in advance
o a cost
o in some cases, the total benefit will only show for tool use in more than one/in all
projects

angajaților
• Potential benefits of using tools )

ny
o Repetitive work is reduced r ma
Ge
o Iterating identical activities
rl in (
Be
o Greater consistency and repeatability H,
G mb
es coverage)
o Objective assessment (e.g., static measure,
ic
rv or testing
setests
o Ease of access to information about
o gy
olenables
o Managing data with test tools a diversity of evaluations
e chn
ig tbetter information base to the management for decision
o In this way, providing
d
making re n
3t
2 02
©

angajaților
• Risks of using tools include )

ny
o Functionality of the test tool does not meet expectations r ma
Ge
o Usability of the test tool does not meet expectations in (
B e rl
H, significant and continuing
o Underestimating the time and effort needed to achieve
b
benefits from the tool Gm
es
rv ic
o Other quality requirements are notemet
ys
o Benefit was overestimated og
h nol
te c
o Costs of purchase, introduction
ig
or operation were underestimated
d
ren the effort required to maintain the test assets generates by the tool
o Underestimating
3t
20 2
o Over-reliance
©
on the tool (manual testing would be better)
o Neglecting version control of test assets within the tool

angajaților
• Wrong deployment of the tool

a ny)
o Neglecting relationships and interoperability issues between critical
e rm tools, such
G
r l in ( management
as requirements management tools, version control tools, incident
Be
tools, defect tracking tools and tools from multiple vendors
H,
mb the tool or selling the tool to a
o Risk of tool vendor going out of business, retiring
G
s
different vendor ic e rv
e
y ssupport, upgrades, and defect fixes
o Poor response from the vendor for
lo g
hno
o Risk of suspension of open-source / free tool project
ec
t
digwill solve all testing problems
o Expectation a tool
en
o A tool can tr
3 never replace a nonexistent process or compensate for a sloppy
2 0 2
procedure
©
o A new tool introducing during “hot” project phases
o Unforeseen, such as the inability to support a new platform
o “A fool with a tool is still a fool”

VI. Tool Support for Testing »
3. Introduction a Tool into an Organization Axa Prioritară: 3 – Locuri de muncă pentru toți
angajaților
Selecting a tool for an organization: 1 of 2
A demanding process that needs to be controlled and managed! )

ny
• Steps towards tool introduction r ma
Ge
o Assessment / opportunities: rl in (
, Be by tools (organization,
Identify the leaks where the test process can be supported
bH
weaknesses, strengths etc.) Gm s
e
o Requirement definition:
e r v ic
s
Demands on the tool are to be clearly
l o gy defined, weighted, prioritised and linked to
measurable criteria. Criteria nocan contain topics like tool chain compatibility and
ech tools, CI, etc.). Also consider requirements from other
integration (also with tbuild
ig
end purchasing)
departments (e.g.,
tr
0 23
o Evaluation:
2
©
Examine tools on short list. Test compliance with requested functionality. Assess
further quality criteria incl. licenses, vendors support, etc.
o Proof-of-concept:
Identify all changes needed to use the tool effectively like e.g., infrastructure,
processes. Testing the test-tool if it will bring the expected effect and support for test
process

angajaților
Selecting a tool for an organization: 2 of 2
• Steps towards tool introduction

a ny)
o Evaluation of the vendor / open-source tool:
e rm
G
r l i (
List all possible candidates with their key properties (support,n commercial, training,
Be
etc.), review the evaluation result and make the final decision
H,
o The use of the tool:
G mb
s
ice and mentoring
Identify the internal requirements for coaching
rv
e
o Evaluation of training:
g ys
o
l and related other potential users will be assessed
The skills of the current test
c h noteam
te
to identify needed training
ig
o Cost-benefitre nd
ratio:
t
023 business case will be the base for a cost-benefit estimation
A concrete
©2
Support introduction by coaching and training for tool usage. Ideally, set up a
pilot project to introduce the tool before start to roll out incrementally

angajaților
Advantages of a pilot project for tool introduction

• Getting to know the tool in detail with its strong and weak points )
ny
er
• Interfacing to other tools in use, adapting processes and workflows, ma
( G
in
preferably adjusting the tool e rl B
H,
b organizations, incl. metrics
• Defining reports according to the standardsGofmthe
es
to be collected by the tool v ic
y ser
• Assess if tool meets the expected g benefits
h n o lo
• Estimate whether the ec of deployment is within scope
tcost
i g
t r end
0 23
No roll-out without piloting: otherwise expect acceptance problems
© 2

angajaților
Success factors for a tool deployment within an organization:

• Step-by-step introduction and rollout incrementally and afterwards in) the
complete organization, not only in one project or department man
y
r
Ge
in (
e rl
• Make tool usage mandatory for the respective workflows/processes
B
b H,
• User guidelines are necessary before tool deployment
Gm
i ces
• Users must have access to adequate s ervtraining, quick support must be
y
available for the test team lo g
e c hno
t
• Experience gainedigfrom tool deployment should be made available for all
users t r end
2 0 23
©
• The actual use of the tool should be followed up, so that any necessary
interventions can be made to improve its acceptance
• Gathering lessons learned from all teams

angajaților
Summary
• There is a broad range of test tools available, covering many different
) tasks:
any
e rm
o Test management tools
o Test planning tools in (G
,B e rl
o Test specification tools H
mb
o Test execution tools e sG
o Tools for test object analysis e rv ic
g ys
o
o Tools supporting non-functional
c h nol tests
e
• Tool deployment shouldd ig t be carried out based on a cost-benefit analysis
n
3 t re
© 20 2
• The introduction of a new test tool must be prepared carefully in order to be
successful
• A step-by-step rollout starting with a pilot project is recommended
angajaților
Keywords
• data-driven testing )
ny
• keyword-driven testing r ma
Ge
• test automation rl in (
Be
• test execution tool b H,
m
e sG
• test management tool v ic
y ser
ol og
te chn
ig
tr end
3
© 20 2

angajaților
ny )
r ma
Ge
Thank you for your participation rl in (
Be
b H,
m
sG
and v ic e
ser
lo gy
e c hno
Good luck with
d ig tyour exam
n
3 t re
20 2
©
ISTQB® Certified Tester Foundation Level

angajaților
Condition testing and coverage

• The complexity of a condition that is made up of several atomic conditions is
taken into account a ny)
m er
G statements
in (
o An atomic condition cannot be divided further into smaller condition
rl
, Be
• This technique aims at finding defects resulting b H the implementation of
from
G m
complex or combined conditions
i c es
o Complex conditions are made up ofsatomicer v conditions, which are combined using
g y
logical operators like OR, AND,
ol o XOR, etc.
Example: ((a > 2) OR e < n6))
(bch
o Atomic conditions d g tnot contain logical operators but only relational operators and
ido
n
3 tre (=, > , <, etc.)
the NOT-operator
20 2
©
• There are three types of condition coverage
o simple condition coverage,
o multiple condition coverage
o modified condition decision coverage

angajaților
Simple condition testing and coverage

• Every atomic condition of a complex condition statement has to take at
) least
n y
once the logical values true and false ma r
Ge
rl to explain simple
• This example iseused in (
, B
H
condition coverage, testing a complex
G mb
es
condition
ic
Example s e rv
y • With only two test cases, a simple
consider the following condition:
no lo g condition coverage can be achieved
< 6t
a > 2 OR big
ech
d o Each atomic condition has taken on the
t r encondition value true and the value false
3
test cases for simple
02for
coverage 2are example:
© • However, the combined result is true in
a=3 (true) b=7 (false) a>2 OR b<6 (true) both cases, therefore no decision
a=1 (false) b=5 (true) a>2 OR b<6 (true)
coverage is achieved
o true OR false = true
o false OR true = true

angajaților
Multiple condition testing and coverage

• All combinations that can be created using permutation of the atomic )
conditions must be part of the tests m any
r
• This example is used i n (Gtoe explain
r l
, Be testing a complex
condition coverage
H
b
Gm expression
condition
ices
Example s e•rvWith four test cases, the multiple
y
consider the following condition: ol og condition coverage can be achieved
hn
te c
o All possible combinations of true and
a > 2 OR big
<6 false were created
end
3 tr condition
test cases for multiple
0 2 o All possible results of the multiple
coverage 2are for example: conditions were achieved
©
a=3 (true) b=7 (false) a>2 OR b<6 (true) • The number of test cases increases
a=3 (true) b=5 (true) a>2 OR b<6 (true) exponentially:
o n = number of atomic conditions
a=1 (false) b=7 (false) a>2 OR b<6 (false)
o 2n = number of test cases

angajaților
Modified condition decision coverage

• All combinations that can be created using the logical results of the) atomic
conditions must be part of the test, only if a change of the outcome
y one
anof
e r m
n (G
atomic condition changes the result of the combined condition
rl i
, B e
• This example
m bH is used to explain modified
condition G decision coverage using a
i c es
Example
v
ercomplex condition expression
ys
consider the following condition:
no lo g • For three out of the four test cases the
< 6t
a > 2 OR big
ech change of an atomic condition changes
end
tr condition decision
the overall result
2 3
test cases for modified
0 for example: o Only for case no. 2 (true OR true = true)
coverage 2are
© the change of an atomic condition will not
a=3 (true) b=7 (false) a>2 OR b<6 (true) result in a change of the overall result. This
a=3 (true) b=5 (true) a>2 OR b<6 (true) test case can be omitted!
• The number of test cases can be
a=1 (false) b=7 (false) a>2 OR b<6 (false) reduced to a value between n+1 and 2n

angajaților
Condition testing and coverage – general conclusions

• The simple condition coverage is a weak instrument for testing complex
a
conditions (can be improved by using condition-decision coverage) ny)
m r
Ge
in (
e rl
• The multiple condition coverage is a much better method
,B
H
mb
o It ensures statement and decision coverage
s G
cecases:
o However, it results in a high number of test 2n
i
e rv
ys
o Some combinations may not be possible to execute
g
oloatomic conditions cannot be false at the same time
§ e.g., for “x > 5 AND x < 10”nboth
h
ec
• The modified condition ig t
decision coverage is even better, because
nd e
o It reduces tr number of test cases (from 2n to n+1)
3the
0 2
©2
o Statement and decision coverage are achieved as well
o Takes into account the complexity of decision statements
All complex decisions must be tested – the modified condition decision

coverage is a suitable method to achieve this goal.

angajaților
Path testing and coverage 1 of 2
• Path coverage focuses on the execution of all possible paths through) a

ny
program ma er
(G
lin graph: an alternating
o A path is a combination of program segments (in a control flow
r
Be
sequence of nodes and edges)
b H,
m is sufficient. For path coverage,
o For decision coverage, a single path through aGloop
e s
there are additional test cases: v ic
ser
gy
§ One test case not entering the loop
lo
o number of loop executions
hnevery
§ One additional test case for
i g te c
nd to a very high number of test cases (infinite)
• This may easilyelead
02 3 tr
©2

angajaților
Path testing and coverage 2 of 2
• Focus of the coverage analysis is the control flow graph )

ny
o Statements are nodes r ma
Ge
o Control flow is represented by the edges in (
, B e rl
o Every path is a unique way from the beginning to the
m bHend of the control flow graph
• The aim of this test (test exit criteria) is ito c s G a defined path coverage
ereach
er v
percentage (although this is merely
g y s a theoretical concept):
h n o lo
tec of covered paths
number
path coverage =ndig × 100%
t r e total number of paths
2 0 23
©

angajaților
Path testing and coverage – example 1

• The control flow graph on the right represents the )
ny
program segment to be inspected. It contains three
r ma
Ge
in (
if-statements
B e rl
H,
• Two different test cases are needed to achieve full
b
m
statement coverage sG e
e r v ic
• Three different paths leading through s the graph of
l o gy
hno
this program segment achieve full decision coverage
e c
• However, five different
d ig t possible paths may be
e n
executed 23 tr
20 cases are required to achieve 100% path
© test
o Five
coverage
o two test cases needed for 100% C0-coverage
o three test cases needed for 100 % C1-coverage

angajaților
Path testing and coverage – example 2

• The control flow graph on the right represents the
program segment to be inspected. It contains two a ny)
if-statements and a loop inside the second if- ( G e rm
rl in
statement , Be
m bH
• Three different paths leading through thesgraph G
i c e
r v
of this program segment achieve fulledecision
coverage g ys
lo
hno
ec possible, if the loop is
• Four different paths
i g tare
executed twice nd
t re
2
• Every©increment
0 23
of the loop counter adds a
new test case

angajaților
Path testing and coverage – general conclusions

• 100% path coverage can only be achieved for very simple programs )
y a n
o A single loop can lead to a test case explosion because every possible
e rm number of
G
r l in (
loop executions constitutes a new test case. Alternatively in practice the loop is
executed 0 times (no loop), exactly 1 time and x times B e
, though the loop, where x is to
be determined at the tester’s own discretion bH
s Gm
iceis possible
o Theoretically an indefinite number of paths
v
y ser
g
• Path coverage is much more lcomprehensive than statement or decision
coverage h n oo
i g te c
e d
o Every possible npath through the program is executed
3 t r
• 100% path
2 02coverage includes 100% decision coverage, which again contains
©
100% statement coverage

CTFL 2018 en v3.1k Watermark EU Tender

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

CTFL 2018 en v3.1k Watermark EU Tender

Încărcat de

Drepturi de autor:

Formate disponibile

Axa Prioritară: 3 – Locuri de muncă pentru toți

Obiectivul specific: 3.12. Imbunătățirea nivelului de cunoștințe/competențe/aptitudini

trendig technology services GmbH 10787 Berlin I Germany

Certified Tester Foundation Level © trendig technology services GmbH 2

Certified Tester Foundation Level © trendig technology services GmbH 3

Certified Tester Foundation Level © trendig technology services GmbH 4

This training is designed according to the Certified Tester Foundation Level

Certified Tester Foundation Level © trendig technology services GmbH 5

Chapter 0. – Information / introduction

Chapter I. Fundamentals of testing ny )

Certified Tester Foundation Level © trendig technology services GmbH 6

Chapter III. Static testing

Certified Tester Foundation Level © trendig technology services GmbH 7

Chapter V. Test management

Certified Tester Foundation Level © trendig technology services GmbH 8

Qualifying program of the ISTQB®*

o in Germany: The German Testing Board (GTB®)

* ISTQBâ = International Software Testing Qualifications Board

Certified Tester Foundation Level © trendig technology services GmbH 9

ISTQB Agile Core Specialist

Certified Tester Foundation Level © trendig technology services GmbH 10

Certified Tester Foundation Level © trendig technology services GmbH 11

Main goals for the Certified Tester training

Certified Tester Foundation Level © trendig technology services GmbH 13

Requirements IREB® Certified Professional for Requirements Engineering

Certified Tester Foundation Level © trendig technology services GmbH 14

Innovation & Design Sprint

Holistic Testing: strategies for agile teams

Certified Tester Foundation Level © trendig technology services GmbH 15

ISTQB ISTQB IREB Design Sprint ISTQB Test

Certified Tester Foundation Level © trendig technology services GmbH 16

Certified Tester Foundation Level © trendig technology services GmbH 17

trendig technology services – instructions for use

Certified Tester Foundation Level © trendig technology services GmbH 18

Certified Tester Foundation Level © trendig technology services GmbH 19

Chapter I. Fundamentals of testing

Certified Tester Foundation Level © trendig technology services GmbH 20

Learning objectives for fundamentals of testing: 1 of 2

1.1 What is testing? )

Certified Tester Foundation Level © trendig technology services GmbH 21

Learning objectives for fundamentals of testing: 2 of 2

1.4 Test process )

Certified Tester Foundation Level © trendig technology services GmbH 22

Chapter I. Fundamentals of testing

Certified Tester Foundation Level © trendig technology services GmbH 23

The economic importance of software

Certified Tester Foundation Level © trendig technology services GmbH 24

Dynamic and static testing

Certified Tester Foundation Level © trendig technology services GmbH 25

Testing means more than running tests

Certified Tester Foundation Level © trendig technology services GmbH 26

• Prevent defects by evaluating work products )

Certified Tester Foundation Level © trendig technology services GmbH 27

• Find defects and failures, thereby reducing risks

Certified Tester Foundation Level © trendig technology services GmbH 28

Testing objectives – more

Certified Tester Foundation Level © trendig technology services GmbH 29

Testing and debugging 1 of 2

While test execution may show failures, debugging is the process of

Certified Tester Foundation Level © trendig technology services GmbH 30

Testing and debugging 2 of 2

Debugging activities (done by developers), targeting component ) and

Certified Tester Foundation Level © trendig technology services GmbH 31