ST Lecture 1 - Fundamentals of ST - Part I

Fundamentals of Software Testing
Fundamentals of Software Testing
Sept Dec, 2012
Outline
What is Software Testing? Software Quality Why Test Software? The Testing Spectrum Benefits of Software Testing What should be Tested?
Who does the Testing

Key Players in Software Testing and their Roles
Software Testing
12/31/2012
What is Software Testing?

According to John E Bentley et al, Software Testing is the process of
verifying and validating that a software application or program:

o Meets the business and technical requirements that guided its
design and development

o Works as expected Software Testing also identifies important defects, flaws, or errors in
the application code that must be fixed

According to Lu Luo, Software Testing is a means of assessing the
software to determine its quality

o What is Software Quality?
Software Testing
12/31/2012
What is Software Testing?...

A common perception of testing is that it only consists of running tests,
i.e. executing the software. This is part of testing, but not all of the testing activities
Test activities exist before and after test execution. These activities
include:
o Planning and control o Choosing test conditions o Designing and executing test cases o Checking results o Evaluating exit criteria
o Reporting on the testing process and system under test

o Finalizing or completing closure activities after a test phase has
4 Software Testing 12/31/2012
been completed

Testing also includes reviewing documents (including source code)
and conducting static analysis

Both dynamic testing and static testing can be used as a means for
achieving similar objectives, and will provide information that can be used to improve both the system being tested and the development and testing processes
Testing can have the following objectives: o Finding defects o Gaining confidence about the level of quality o Providing information for decision-making o Preventing defects

The thought process and activities involved in designing tests early in
the life cycle can help to prevent defects from being introduced into code. Reviews of documents (e.g., requirements) and the identification and resolution of issues also help to prevent defects appearing in the code
Different viewpoints in testing take different objectives into account.
For example, in development testing (e.g., component, integration and
system testing), the main objective may be to cause as many failures

as possible so that defects in the software are identified and can be fixed
In acceptance testing, the main objective may be to confirm that the
system works as expected, to gain confidence that it has met the

requirements

In some cases the main objective of testing may be to assess the quality
of the software (with no intention of fixing defects), to give information to stakeholders of the risk of releasing the system at a given time
Maintenance testing often includes testing that no new defects have been
introduced during development of the changes

During operational testing, the main objective may be to assess system
characteristics such as reliability or availability

Debugging and testing are different. Dynamic testing can show failures
that are caused by defects. Debugging is the development activity that finds, analyses and removes the cause of the failure
Subsequent re-testing by a tester ensures that the fix does indeed resolve
the failure. The responsibility for each these activities is usually testers test and developers debug
Software Quality
Garvin D in his article What does product quality mean? describes
quality from five different perspectives:

The Transcendental View: Quality is something we can recognize
but not define

The User View: Quality is fitness for purpose Manufacturing View: Quality is conformance to specification Product View: Quality is tied to inherent product characteristics Value-based View: Quality depends on the amount the customer
is willing to pay for it.

Software Quality
The manufacturing view examines whether the product was built right
the first time, avoiding costly rework to fix delivered faults. Thus the manufacturing view is a process view, advocating conformance to good process
Everybody has a different view of what characteristics of a software
contribute to its overall quality. This is because the importance of the characteristics depend on who is analyzing the software. For example:
Does it do what they want?
Is it easy to learn? How does it compare with other products? What are the number of failures and the type of failures?
Software Quality
With the help of testing, it is possible to measure the quality of software in
terms of defects found, for both functional and non-functional software requirements and characteristics
Testing can give confidence in the quality of the software if it finds few or
no defects
A properly designed test that passes reduces the overall level of risk in a
system
When testing does find defects, the quality of the software system
increases when those defects are fixed

Lessons should be learned from previous projects. By understanding the
root causes of defects found in other projects, processes can be improved, which in turn should prevent those defects from reoccurring and, as a consequence, improve the quality of future systems. This is an aspect of quality assurance
Testing should be integrated as one of the quality assurance activities (i.e.
Process Quality
The quality of the development and maintenance process is as
important as product quality
The process should be improved by examining: Where and when you are likely to find a particular kind of fault How can we find faults earlier in the development process? How can we design secure, high-quality systems? Are there alternative activities that can make our process more
effective or efficient at ensuring quality?
11
Software Testing
12/31/2012
Quality Attributes
Measures of software quality can be divided into: o Static quality attributes o Dynamic quality attributes
Static
quality attributes refer to the actual code and related
documentation. It focuses on the range of methods that are used to determine or estimate software quality without reference to actual executions. Techniques in this area include code inspection, program analysis, symbolic analysis, and model checking
Static quality attributes include structured, maintainable, and testable
code as well as the availability of correct and complete documentation

Quality Attributes
Dynamic quality attributes relate to the behavior of the application
while in use. It deals with specific methods for ascertaining and/or approximating software quality through actual executions, i.e., with real data and under real (or simulated) circumstances.
Techniques in this area include synthesis of inputs, the use of
structurally dictated testing procedures, and the automation of testing environment generation
Dynamic quality attributes include software reliability, correctness,
completeness, consistency, usability, and performance.

Quality Attributes
Reliability: the probability of a failure-free operation Correctness: the correct operation of an application and is always with
reference to some artifact

Completeness: the availability of all the features listed in the
requirements or in the user manual

Consistency: refers to adherence to a common set of conventions and
assumptions e.g all buttons in the user interface might follow a common color-coding convention
Usability: the ease with which an application can be used Performance: refers to the time the application takes to perform a
requested task
Why Test Software?

Software
systems are an integral part of life, from business
applications to consumer products

Software that does not work correctly can lead to many problems,
including loss of money, time or business reputation, and could even cause injury or death
Software defects can be caused by human beings. An error may
produce a defect in the program code, or in a document

If a defect in code is executed, the system may fail to do what it should
do (or do something it shouldnt), causing a failure

Defects in software, systems or documents may result in failures, but
not all defects do so

Why Test Software?

Software testing answers questions that development testing and code
reviews cannot:
o Does it really work as expected? o Does it meet the users requirements? o Is it what the users expect? o Do the users like it? o Is it compatible with our other systems? o How does it perform? o How does it scale when more users are added? o Which areas need more work? o Is it ready for release?
Why Test Software?...

The general aim of testing is to affirm the quality of software systems
by systematically exercising the software in carefully controlled circumstances

Software Testing has three main purposes: o Verification o Validation o Defect finding Verification: Are we building the product right? Validation: Are we building the right product?
17
Software Testing
12/31/2012

The verification process confirms that the software meets its technical
specifications. A specification is a description of a function in terms of a measurable output value given a specific input value under specific preconditions
A simple specification may be along the line of a SQL query retrieving data
for a single account against the multi-month account-summary table must return these eight fields <list> ordered by month within 3 seconds of submission.
The validation process confirms that the software meets the business
requirements
A simple example of a business requirement is After choosing a branch
office name, information about the branchs customer account managers will
appear in a new window. The window will present manager identification and
18
summary Testing Software information about each managers customer base: <list of data 12/31/2012 elements>. Other requirements provide details on how the data will be

A defect is a variance between the expected and actual result. The
defects ultimate source may be traced to a fault introduced in the specification, design, or development (coding) phases
A good test is one that has a high probability of finding an as yet
undiscovered error, and a successful test is one that uncovers an as yet undiscovered error
Defects occur due to: o Errors made by human beings o Time pressure o Complexity in code o Complexity of infrastructure o Changing technologies o Many system interactions

Failures can be caused by environmental conditions as well. For
example, radiation, magnetism, electronic fields, and pollution can cause faults in firmware or influence the execution of software by changing the hardware conditions
Testing plays a major role in software development, maintenance and
operations:
o Rigorous testing of systems and documentation can help to
reduce the risk of problems occurring during operation and contribute to the quality of the software system, if the defects found are corrected before the system is released for operational
use
20 Software Testing
o Software testing may also be required to meet contractual or legal

12/31/2012
requirements, or industry-specific standards

Testing also plays a role in the quality of the resulting software/product: o With the help of testing, it is possible to measure the quality of
software in terms of defects found, for both functional and nonfunctional software requirements and characteristics
o Testing can give confidence in the quality of the software if it finds
few or no defects
o A properly designed test that passes reduces the overall level of risk
in a system.
o When testing does find defects, the quality of the software system
increases when those defects are fixed

o By understanding the root causes of defects found in other projects,
21
processes Software Testing can
be improved, which in turn should prevent those 12/31/2012
defects from reoccurring and, as a consequence, improve the quality
How Much Testing is Enough?

Deciding how much testing is enough should take account of: o The level of risk, including technical, safety, and business risks o The project constraints such as time and budget
Testing should provide sufficient information to stakeholders to make
informed decisions about the release of the software or system being tested, for the next development step or handover to customers
22
Software Testing
12/31/2012
The Testing Spectrum

Testing is involved in every stage of software life cycle, but the testing
done at each level of software development is different in nature and has different objectives
Unit Testing is done at the lowest level. It tests the basic unit of
software, which is the smallest testable piece of software, and is often called unit, module, or component interchangeably.
Integration Testing is performed when two or more tested units are
combined into a larger structure. The test is often done on both the
interfaces between the components and the larger structure being

constructed, if its quality property cannot be assessed from its
23 Software Testing components. 12/31/2012
The Testing Spectrum

System Testing tends to affirm the end-to-end quality of the entire
system. System test is often based on the functional/requirement specification of the system. Non-functional quality attributes, such as reliability, security, and maintainability, are also checked.
Acceptance Testing is done when the completed system is handed
over from the developers to the customers or users. The purpose of acceptance testing is rather to give confidence that the system is working than to find errors.
24
Software Testing
12/31/2012
Benefits of Software Testing

Save time and money by identifying defects early Avoid or reduce development downtime Provide better customer service by building a better application Know that weve satisfied our users requirements Build a list of desired modifications and enhancements for later
versions
Identify and catalog reusable modules and components
Identify areas where programmers and developers need training

What Should be Tested?

First, test whats important
Focus on the core functionalitythe parts that are critical or popular
before looking at the nice to have features
Concentrate on the applications capabilities in common usage
situations before going on to unlikely situations. For example, if the application retrieves data and performance is important, test reasonable queries with a normal load on the server before going on to unlikely ones at peak usage times
26
Software Testing
12/31/2012
What Should be Tested?...

The value of software testing is that it goes far beyond testing the
underlying code. It also examines the functional behavior of the application. Behavior is a function of the code, but it doesnt always follow that if the behavior is bad then the code is bad. Its entirely possible that the code is solid but the requirements were inaccurately or incompletely collected and communicated
A comprehensive testing regime examines all components associated
with the application

Testing provides an opportunity to validate and verify things like the
assumptions that went into the requirements, the appropriateness of
the systems that the application is to run on, and the manuals and
documentation that accompany the application
What Factors Does Testing Involve?

Testing can involve some or all of the following factors. The more, the
better.
o Business requirements
o Functional design requirements

o Technical design requirements o Regulatory requirements o Programmer code o Systems administration standards and restrictions o Corporate standards o Professional or trade association best practices
Software Testing
o Hardware configuration
12/31/2012
28
o Cultural issues and language differences
Who Does the Testing?

Software Testing is not a one person job. It takes a team, but the team
may be larger or smaller depending on the size and complexity of the application being tested
The programmer(s) who wrote the application should have a reduced
role in the testing if possible. The concern here is that theyre already so intimately involved with the product and know that it works that they may not be able to take an unbiased look at the results of their
labors
Testers must be cautious, curious, critical but non-judgmental, and
good communicators. One part of their job is to ask questions that the
developers might find not be able to ask themselves or are awkward,

irritating, insulting or even threatening to the developers.
Who Does the Testing?...

A good developer does not necessarily make a good tester and vice
versa, but testers and developers do share at least one major traitthey itch to get their hands on the keyboard
Testers are the only IT people who will use the system as heavily an
expert user on the business side

User testing almost invariably recruits too many novice business users
because theyre available and the application must be usable by them.
The problem is that novices dont have the business experience that the
expert users have and might not recognize that something is wrong
Testers from IT must find the defects that only the expert users will find
because the experts may not report problems if theyve learned that it's
not worth their time or trouble

Business Sponsor(s) and Partners o Provides funding o Specifies requirements and deliverables o Approves changes and some test results
Software Developer(s)
o Designs, codes, and builds the application

o Participates in code reviews and testing o Fixes bugs, defects, and shortcomings
31
Software Testing
12/31/2012

Project Manager o Plans and manages the project
Testing Coordinator(s) o Creates
test plans and test specifications based on the
requirements and functional, and technical documents
Tester(s) o Executes the tests and documents results
32
Software Testing
12/31/2012
END
33
Software Testing
12/31/2012

ST Lecture 1 - Fundamentals of ST - Part I

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

ST Lecture 1 - Fundamentals of ST - Part I

Încărcat de

Drepturi de autor:

Formate disponibile

Fundamentals of Software Testing

Fundamentals of Software Testing

Sept Dec, 2012

Who does the Testing

What is Software Testing?

verifying and validating that a software application or program:

design and development

the application code that must be fixed

software to determine its quality

What is Software Testing?...

o Reporting on the testing process and system under test

What is Software Testing?...

and conducting static analysis

What is Software Testing?...

For example, in development testing (e.g., component, integration and

system testing), the main objective may be to cause as many failures

system works as expected, to gain confidence that it has met the

What is Software Testing?...

introduced during development of the changes

characteristics such as reliability or availability

quality from five different perspectives:

but not define

is willing to pay for it.

increases when those defects are fixed

Testing should be integrated as one of the quality assurance activities (i.e.

important as product quality

effective or efficient at ensuring quality?

quality attributes refer to the actual code and related

code as well as the availability of correct and complete documentation

Techniques in this area include synthesis of inputs, the use of

Dynamic quality attributes include software reliability, correctness,

completeness, consistency, usability, and performance.

reference to some artifact

requirements or in the user manual

Why Test Software?

systems are an integral part of life, from business

applications to consumer products

produce a defect in the program code, or in a document

do (or do something it shouldnt), causing a failure

not all defects do so

Why Test Software?

Why Test Software?...

by systematically exercising the software in carefully controlled circumstances

Why Test Software?...

Why Test Software?...

Why Test Software?...

o Software testing may also be required to meet contractual or legal

requirements, or industry-specific standards

Why Test Software?...

increases when those defects are fixed

processes Software Testing can

be improved, which in turn should prevent those 12/31/2012

defects from reoccurring and, as a consequence, improve the quality

How Much Testing is Enough?

Testing should provide sufficient information to stakeholders to make

The Testing Spectrum

Integration Testing is performed when two or more tested units are

interfaces between the components and the larger structure being

The Testing Spectrum

Acceptance Testing is done when the completed system is handed

Benefits of Software Testing

Identify areas where programmers and developers need training

What Should be Tested?