Software Testing & Quality Assurance

reff. Kostas Kontogiannis

Welcome
Welcome to the Software Testing and Quality Assurance course Demanding, challenging and, rewarding course A glimpse of what software testing in theory and practice is all about

Software Testing

CHAPTER 1
Unit 1

Outline for today

Introduction

Course description Software Testing, Quality Assurance, and Evolution basics

Course website
http://www.swen.uwaterloo.ca/~kostas/ECE453-06/ schedule lecture notes & slides recommended book
Software Testing A Craftsman's Approach 2nd edition, Paul C. Jorgensen, CRC Press

past exams project description grade allocation course news system

5

Goals of this course

Become familiar with software testing concepts Learn how to design and plan software testing activities Experience software testing in a project that uses several components Note:
Scratching the surface of software testing and QA Fitting you to become an amateur software tester
6

Course outline
Unit 1: Overview of Software Testing (Chapter 1-2) Unit 2: Mathematics for S/W Testers (Chapters 3-4) Unit 3: Black Box / Functional Testing (Chapters 5-8) Unit 4: White Box / Structural Testing (Chapters 9-11) Unit 5: Integration and System Testing (Chapter 12-13) Unit 6: Software Reliability and Quality Assurance Basics Unit 7: Software Reengineering and Evolution Basics Unit 8: Project Planning and Management

Outline for today

Introduction Course description Software testing basics

Software Lifecycle

Requirements phase Analysis phase Design phase (System and Object) Implementation phase Testing phase Integration phase Maintenance phase

Software Lifecycle Activities

...and their models
Requirements Elicitation Analysis System Design Object Design Implementation Testing

Expressed in Terms Of

Structured By

Implemented By Realized By Verified By

class... class... class...

Use Case Model

Application Subsystems Domain Objects

Solution Domain Objects

? class.... ? Test Cases

10

Source Code

Software Testing : Definition

The process of devising a set of inputs to a given piece of software that will cause the software to exercise some portion of its code. The developer of the software can then check that the results produced by the software are in accord with his or her expectations.

www.reactive-systems.com/esda-glossary.msp
11

Software Testing : Definition

Software testing is a process used to identify the correctness, completeness and quality of developed computer software.
Actually, testing can never establish the correctness of computer software, as this can only be done by formal verification (and only when there is no mistake in the formal verification process). It can only find defects, not prove that there are none. en.wikipedia.org/wiki/Software_testing
12

A Testing Cycle
Although testing varies between organizations, there is a cycle to testing:
1. Requirements Analysis: Testing should begin in the requirements phase of the software life cycle (SDLC). 2. Design Analysis: During the design phase, testers work with developers in determining what aspects of a design are testable and under what parameter those testers work. 3. Test Planning: Test Strategy, Test Plan(s), Test Bed creation. 4. Test Development: Test Procedures, Test Scenarios, Test Cases, Test Scripts to use in testing software. 5. Test Execution: Testers execute the software based on the plans and tests and report any errors found to the development team. 6. Test Reporting: Once testing is completed, testers generate metrics and make final reports on their test effort and whether or not the software tested is ready for release. 7. Retesting the Defects

13

Some Software Testing Objectives

Find as many defects as possible. Find important problems fast. Assess perceived quality risks. Advise about perceived project risks. Advise about perceived quality. Certify to a given standard. Assess conformance to a specification (requirements, design, or product claims).
14

Software Testing in RUP

Phases Process Workflows
Business Modeling Requirements
Inception Elaboration Construction Transition

Architecture & Design

Implementation

Test
Deployment

Supporting Workflows
Configuration Mgmt Management Environment
Preliminary Iteration(s) Iter. #1 Iter. #2 Iter. #n Iter. Iter. #n+1 #n+2 Iter. #m Iter. #m+1

Iterations

15

Software Testing in RUP

RUP Testing Philosophy
Iterative development. Low up-front documentation Holistic approach. Automation Test Manager Test Analyst Test Designer Tester
16

Actors and Roles

Software Testing in XP
The biggest difference between XP projects and most traditional software development projects is the concept of test-driven development.
With XP, every chunk of code is covered by unit tests, which must all pass all the time. The absence of unit-level and regression bugs means that testers actually get to focus on their job: making sure the code does what the customer wanted. The acceptance tests define the level of quality the customer has specified (and paid for!)
17

Software Testing in XP
Characteristics
Planning Every 2-3 weeks

Write tests

Release

Pair Programming + Refactoring

Test Min. daily

Integration

Evolutionary development Collection of 12 Best Practices Focus on working code that implements customer needs (rather than documents) Testing is a crucial element of the process Focus on flexibility and efficiency of the process Designed for small teams (<10)
18

Software Testing in XP
Activities testers perform on XP teams.
Negotiate quality with the customer (its not YOUR standard of quality, its what the customer desires and is willing to pay for!) Clarify stories, flush out hidden assumptions Enable accurate estimates for both programming and testing tasks Make sure the acceptance tests verify the quality specified by the customer Help the team automate tests Help the team produce testable code Form an integral part of the continuous feedback loop that keeps the team on track

19

Software Quality Assurance : Definition

The function of software quality that assures that the standards, processes, and procedures are appropriate for the project and are correctly implemented.
www.hq.nasa.gov/office/codeq/software/umbrella_defs.htm

20

Software process improvement initiatives

Capability maturity model (CMM)
http://www.sei.cmu.edu/cmm/cmms/cmms.html

ISO 9000-series ISO/IEC 15504 Standard for Software Process Assessment (SPICE)
http://www-sqi.cit.gu.edu.au/spice/

21
Steven Schach 2002 [modified]

Capability Maturity Model

A set of strategies for improving the software process
SWCMM for software PCMM for human resources (people) SECMM for systems engineering IPDCMM for integrated product development SACMM for software acquisition

These strategies are being unified into CMMI (capability maturity model integration) Developed by Software Engineering Institute (SEI)
Steven Schach 2002 [modified]

22

SWCMM
A strategy for improving the software process Put forward in 1986 by the SEI Fundamental ideas:
Improving the software process leads to
Improved software quality Delivery on time, within budget

Improved management leads to

Improved techniques

Five levels of maturity are defined Organization advances stepwise from level to level
Steven Schach 2002

23

SWCMM Summary

24
Steven Schach 2002

Software Evolution : Definition

A continuous change from a lesser, simpler, or worse state to a higher or better state [for a software system]
www.cc.gatech.edu/reverse/glossary.htm

25

Software Maintenance : Definition

Software maintenance is one of the activities in software engineering, and is the process of enhancing, porting, and optimizing deployed software, as well as remedying defects
en.wikipedia.org/wiki/Software_maintenance

26

Average cost distribution

(19761981 data)

Object-Oriented and Classical Software Engineer 5th Edition, Schach (2002)

27

Summary
Testing is an important part of the Software Lifecycle Highly technical and challenging Is affected by the selected process Quality Assurance is paramount both for mission critical and non-critical systems Software Evolution aims to keep systems operational when environment changes occur
28