25-10-2019

Fundamentals
Of
Software Testing

By
Shampa Dey, Project Engineer

Contents
• Objectives of Software Testing
• Software Testing – start and end
• Software Testing Principles
• Error, Fault and Failure
• Defect - Severity and Priority
• Verification and Validation
• Software Quality
• Importance of Software Quality
• Dimensions of Software Quality

10/25/2019 www.cdac.in 2

1
 25-10-2019

Objectives of Software Testing

Objectives of Software Testing are as follows –

 Finding as many defects as possible which may get created by the programmer while
developing the software
 Writing high quality test cases
 Prevent defects
 Confirm that system satisfies the business requirements
 To check if the application meets the system requirement specifications
 Gain customer confidence by providing them with a quality product
 Gaining confidence in the level of quality of the product

10/25/2019 www.cdac.in 3

Testing Fundamentals
Why to test?

Testing becomes absolutely essential to make sure the software works properly and
does the work that it is meant to perform.

What to test?

Any working product which forms part of the software application has to be
tested - Hardware, Software, Test Data.

10/25/2019 www.cdac.in 4

2
 25-10-2019

Testing Fundamentals
How often to test ?

When a program (source code) is modified or newly developed, it has to be

tested.

Who can test ?

Programmer, Tester and Customer

10/25/2019 www.cdac.in 5

Why start testing early ?

S
T E
A Planning Analysis Coding Testing N
R D
T
When project is planned
Planning, Analysis and Coding will take more
time then planned.

No project extension
The deadline is fixed
S E
T N
A Planning Analysis Coding Test
D
R
T Reality when project is executed

10/25/2019 www.cdac.in 6

3
 25-10-2019

Why start testing early ?

R
E
Q
U
I
R
E M
M
E
A Finding bugs in the requirement analysis phase
C I
N is going to be cost effective than finding the
O T T N
S E T same bug in testing phase.
T A C S E
N
O T N It will even be more cost effective than when
A D
D I A
L E
N you find the bug after going live.
Y S I N
S I N G C
I G G E
S N

Price of buggy code

10/25/2019 www.cdac.in 7

When to start testing

An early start to testing
 reduces the cost and time to rework.
 produce error-free software that is delivered to the client.

Testing can be started from the Requirements Gathering phase and continued till
the deployment of the software.

Tes ng is done in different forms at every phase of SDLC −

 During the requirement gathering phase, the analysis and verification of requirements.
 Reviewing the design in the design phase with the intent to improve the design.
 Testing performed by a developer on completion of the code.

10/25/2019 www.cdac.in 8

4
 25-10-2019

When to stop testing

It is difficult to determine when to stop testing.

Testing is a never-ending process and no one can claim that a software is

100% tested.

The following aspects are to be considered for stopping the testing process
Testing Deadlines
Completion of test case execution
Completion of functional and code coverage to a certain point
Bug rate falls below a certain level and no high-priority bugs are identified
Management decision

10/25/2019 www.cdac.in 9

SOFTWARE TESTING PRINCIPLES

10/25/2019 www.cdac.in 10

5
 25-10-2019

Software Testing Principles

Are you following the right strategy for testing ?

It is important to achieve optimum test results while conducting

software testing without deviating from the goal.

10/25/2019 www.cdac.in 11

Software Testing Principles

Example Problem:

Consider a scenario where you are moving a file from Folder A to Folder B. Think of all the
possible ways you can test this. Apart from the usual scenarios, you can also test the following
conditions –
 Trying to move the file when it is Open
 You do not have the security rights to paste the file in Folder B
 Folder B is on a shared drive and storage capacity is full.
 Folder B already has a file with the same name

Or suppose you have 15 input fields to test, each having 5 possible values, the number of
combinations to be tested would be 5^15 = 75

• If you were to test the entire possible combinations project EXECUTION TIME & COSTS would rise.
So we need certain principles and strategies to optimize the testing effort.

10/25/2019 www.cdac.in 12

6
 25-10-2019

Seven Principles of Software Testing

Here are the common seven testing principles that are widely practiced in the software industry.

Principle 1 Exhaustive testing is impossible

Principle 2 Defect Clustering

Principle 3 Pesticide Paradox

Principle 4 Presence of defects

Principle 5 Absence of errors - fallacy

Principle 6 Early Testing

Principle 7 Context dependent

10/25/2019 www.cdac.in 13

Software Testing Principles

1) Exhaustive testing is not possible

Yes! Exhaustive testing is not possible.

Instead, we need the optimal amount of testing based on the risk assessment
of the application.

10/25/2019 www.cdac.in 14

7
 25-10-2019

Software Testing Principles

2) Defect Clustering

Approximately 80% of the problems are found in 20% of the modules.

A small number of modules contain most of the defects detected.

By experience, you can identify such risky modules. But this approach has its
own limitations

 If the same tests are repeated over and over again, eventually the same test cases will no
longer find new bugs.

10/25/2019 www.cdac.in 15

Software Testing Principles

3) Pesticide Paradox

Repetitive use of the same pesticide mix to eradicate insects during farming will over
time lead to the insects developing resistance to the pesticide.
If the same set of repetitive tests are conducted, the method will be useless for
discovering new defects.

To overcome this -
 Test cases need to be regularly reviewed & revised, adding new & different test cases to help find
more defects.
 Testers cannot simply depend on existing test techniques. He must look out continually to improve
the existing methods to make testing more effective.

But even after all this hard work in testing, you can never claim your product is bug-free.

10/25/2019 www.cdac.in 16

8
 25-10-2019

Software Testing Principles

4) Presence of defects

Testing talks about the presence of defects and don’t talk about the absence of
defects.
Software Testing reduces the probability of undiscovered defects remaining in the
software.
Even if no defects are found, it is not a proof of correctness.

But what if, you work extra hard, taking all precautions & make your
software product 99% bug-free. And the software does not meet the needs
& requirements of the clients.
This leads us to our next principle - Absence of Error

10/25/2019 www.cdac.in 17

Software Testing Principles

5) Absence of Error – fallacy
It is possible that software which is 99% bug-free is still unusable.

How ?
System is tested thoroughly for the wrong requirements.

Software testing is not merely finding defects, but also to check that software
addresses the business needs.

Finding and fixing defects does not help if the system build is unusable and does not
fulfill the user's needs & requirements.

To solve this problem, the next principle of testing - Early Testing

10/25/2019 www.cdac.in 18

9
 25-10-2019

Software Testing Principles

6) Early Testing

Testing should start as early as possible in the Software Development Life

Cycle.
Defects in the requirements or design phase will be captured in early stages.
Cost effective to fix a defect in the early stages of testing.

But how early one should start testing?

Start finding the bug the moment the requirements are defined.

10/25/2019 www.cdac.in 19

Software Testing Principles

7) Context dependent

All the developed software’s are not identical.

Testing is context dependent.
You might use a different approach, methodologies, techniques, and types of
testing depending upon the application type.
Testing any POS system at a retail store will be different than testing an ATM
machine.

10/25/2019 www.cdac.in 20

10
 25-10-2019

Myth: "Principles are just for reference.

I will not use them in practice .”
• Testing Principles will help you create an effective Test Strategy.

• Learning testing principles is just like learning to drive for the first time.
Initially, you pay attention to each and everything like gear shifts, speed, clutch
handling, etc.
But with experience, you just focus on driving the rest comes naturally.
You even hold conversations with other passengers in the car.

• Experienced testers have internalized these principles to a level that they

apply them even without thinking.

10/25/2019 www.cdac.in 21

TERMS & DEFINITIONS

10/25/2019 www.cdac.in 22

11
 25-10-2019

Error, Fault and Failure

Error – a human action that produces an incorrect result. This is also
sometimes referred to as Mistake.

Fault – a manifestation of an error in software, also known as a Bug.

Failure – a deviation of the software from its expected delivery or service.

10/25/2019 www.cdac.in 23

Error, Fault and Failure

Fault in Software
System Failure
Human makes error

10/25/2019 www.cdac.in 24

12
 25-10-2019

Defect
Defect – non-conformance to requirements or functional specification.
It is something that does not correspond to valid Customer’s expectations that are
assumed but may be not described in product requirements.

Expected Result – the behavior of the system (software product) we expect to see in
result of our actions (inputs). In other words – this is correct behavior of the software
product.

If the actual behavior of the program we see does not correspond to that one which we
expect (not the same), such behavior may be considered as wrong and we mat say that
we have found the defect.

10/25/2019 www.cdac.in 25

Severity of a Defect
Severity is defined as the

 Degree of impact a defect has on the operation of a software

 Severity of the consequences
 Higher effect on the system functionality will lead to higher severity.

Severity can be critical or non-critical

10/25/2019 www.cdac.in 26

13
 25-10-2019

Priority of a Defect
Priority is defined as the

 Order in which a defect should be fixed.

 Higher the priority the sooner the defect should be resolved.
 Defects that leave the software system unusable are given higher priority
 Prioritization is typically a managerial decision
 Prioritization is usually based on severity

Priority can be high or low

10/25/2019 www.cdac.in 27

Severity and Priority

Severity
Critical Non-critical
High

Key features Color of Logo is

Priority

does not work incorrect

Low

Rarely used features Caption of Image is

does not work in the wrong font

10/25/2019 www.cdac.in 28

14
 25-10-2019

VERIFICATION & VALIDATION

10/25/2019 www.cdac.in 29

Verification
The verifying process includes checking documents, design, code, and
program.

Ensures the product is designed to deliver all functionality to the customer

Involves reviews and meetings to evaluate documents, plans, code,
requirements and specifications
Verification is done with checklists, issues lists, walkthroughs and inspection
meetings.

10/25/2019 www.cdac.in 30

15
 25-10-2019

Validation
Validation is a dynamic mechanism of software testing and validates
the actual product.

Validation is the process to make sure the product satisfies the specified
requirements at the end of the development phase
Validation involves actual testing
Validation takes place after verifications are completed

10/25/2019 www.cdac.in 31

Example – Verification / Validation

SRS : A clickable button with name Submet SRS : A clickable button with name Submit

Verification checks SRS, design doc Validation tests the software and
and corrects spelling finds button not clickable

Developer modifies code

Submit and makes the button
SRS : A clickable clickable
button with name
Submit

Submit
Submet Submit

10/25/2019 www.cdac.in 32

16
 25-10-2019

Comparison - Verification vs
Validation
VERIFICATION VALIDATION

Verification is the process of evaluating product of a
development phase to find out whether they meet the
specified requirements.
It answers the question “Are we building the system
right?”
Inspections, Reviews, Walkthroughs are methods of
verification.
Validation is the process of evaluating software at the
end of the development process to determine whether
software meets the customer expectations and
requirements.
It answers the question “Are we building the right
system?”
Testing the software application is the method of
validation.

Developer does verification. Tester does validation.

10/25/2019 www.cdac.in 33

SOFTWARE QUALITY

10/25/2019 www.cdac.in 34

17
 25-10-2019

Software Quality
As with any definition, the definition of ‘software quality’ is also
varied and debatable.
 Some say that ‘quality’ cannot be defined.
Can be defined but only in a particular context.
‘quality is simply a lack of bugs’.

High quality is something we all aspire to.

10/25/2019 www.cdac.in 35

Software Quality
SOFTWARE QUALITY is the degree of conformance to explicit or
implicit requirements and expectations.

Explanation
Explicit: clearly defined and documented
Implicit: not clearly defined and documented but indirectly suggested
Requirements: business/product/software requirements
Expectations: mainly end-user expectations

10/25/2019 www.cdac.in 36

18
 25-10-2019

Software Quality
Definition by IEEE (Institute of Electrical and Electronics Engineers)

The degree to which a system, component, or process meets specified

requirements.

The degree to which a system, component, or process meets customer or user

needs or expectations.

10/25/2019 www.cdac.in 37

Software Quality
Definition by
ISTQB (International Software Testing Qualifications Board)

quality: The degree to which a component, system or process meets specified

requirements and/or user/customer needs and expectations.

software quality: The totality of functionality and features of a software

product that bear on its ability to satisfy stated or implied needs.

10/25/2019 www.cdac.in 38

19
 25-10-2019

Importance of Software Quality

saves good amount of time and money
software will have fewer defects
saves time during testing and maintenance phases
greater reliability
increase in customer satisfaction
lower maintenance costs

10/25/2019 www.cdac.in 39

Dimensions of Software Quality

“This software is of a very high quality.”

“In which dimension of quality?”

SOFTWARE QUALITY DIMENSIONS – Which software quality

dimension is more important than the other is subjective and
depends on what dimension you value the most in the particular
situation.

10/25/2019 www.cdac.in 40

20
 25-10-2019

Dimensions of Software Quality

Accessibility:
 used comfortably by a wide variety of people
 users who require assistive technologies like screen magnifiers or voice recognition.

Compatibility:
 suitable in different environments
 different devices, operating systems and browsers

Concurrency:
 Service multiple requests to the same resources at the same time.

10/25/2019 www.cdac.in 41

Dimensions of Software Quality

Functionality:
 Ability of software to carry out the functions as specified or desired.

Installability:
 The ability of software to be installed in a specified environment.

Localizability:
 The ability of software to be used in different languages, time zones, etc.

10/25/2019 www.cdac.in 42

21
 25-10-2019

Dimensions of Software Quality

Performance:
 The speed at which software performs under a particular load.

Reliability:
 The ability of software to perform a required function under stated conditions for the stated
period of time without any errors

Scalability:
 Ability to increase or decrease in performance in response to changes in software’s
processing demands.

10/25/2019 www.cdac.in 43

Dimensions of Software Quality

Security:
 Extent of protection of software against unauthorized access
 Invasion of privacy, theft, loss of data, etc.

Testability:
 The ability of software to be easily tested.

Usability:
 The degree of software’s ease of use.

10/25/2019 www.cdac.in 44

22
 25-10-2019

Dimensions of Software Quality

Efficiency:
 perform well or achieve a result without wasted energy, resources, effort, time or money

Maintainability:
 The ease with which software can be modified
 adding features, enhancing features, fixing bugs, etc.

Portability:
 The ability of software to be transferred easily from one location to another.

10/25/2019 www.cdac.in 45

Thank you

10/25/2019 www.cdac.in 46

23