Topics

Attribute of good software Introduction of non functional testing Functional & non functional requirement

Non-Functional testing techniques

Introduction of performance testing

Nishant Worah 2009

Attribute of good software

The ISO 9126 international standard lists six quality attributes for systems. these are then broken down further sub-attributes.

Functionality Interoperability Security Reliability-Availability Recoverability (Dependability) Maintainability Usability Efficiency - Performance Portability - Installability

Nishant Worah 2009

Attribute of good software

cont

Nishant Worah 2009

Introduction of Non Functional

Testing

Functional Testing
Functional testing demonstrates WHAT the product does

Non Functional Testing

Non-functional testing demonstrates HOW WELL the product behaves

***Peoples desire to purchase/use a product may be more affected by

non-functional qualities than they are by functionality
Nishant Worah 2009
4

Functional & non functional requirements

Functional Requirement
A requirement that specifies a function that a system or system component must be able to perform.

Non Functional Requirement

Building systems with only functional requirements is relatively easy, but ultimately pointless. No-one wants a slow, unreliable system that costs a fortune to modify.

Non-functional requirements is that they specify all the remaining

requirements not covered by the functional requirements.

Nonfunctional requirements specify the systems quality characteristics or quality attributes.

Nishant Worah 2009

Functional & non functional requirements

cont..
Qualitative requirements

Dont ask for a system that is:

Easy to use Faster than a cheetah on speed As reliable as possible

The difficulty with purely qualitative requirements is that they are neither measurable nor testable
Nishant Worah 2009
6

Functional & non functional requirements

cont..
Qualitative requirements

Ideally all nonfunctional requirements need to be specified in a way that makes them understandable by all of the stakeholders. One way of achieving this is by using what are known as SMART requirements.

S-M-A-R-T

Nishant Worah 2009

Functional & non functional requirements

cont..

Specific
We want requirements that are both precise and thorough

Measurable
while measurable should encourage the use of quantitative testable requirements that have been assigned an actual value to both achieve and test against.

Acceptable
Acceptable requirements should satisfy the needs of the customers and users.

Realisable
The main point of realisable requirements is that they should be realistic and achievable given the constraints of the project

Traceable
back to the user requirements and

while traceable requirements should allow traceability both forwards to the subsequent implementation and tests.

Nishant Worah 2009

Non-Functional testing techniques

Memory Management Reliability Usability Maintainability Configuration Portability Recovery Disaster Recovery

Nishant Worah 2009

Non-Functional testing techniques

cont

Performance Stress Security Procedure Interoperability Compatibility Installability Conversion

Nishant Worah 2009
10

Reliability

IEEE 610.12:1990 definition - Ability of a system or component to perform its required functions under stated conditions for a specified period of time Ability of software to perform without failure under specified condition for either a specified period of time or for a specified number of transaction The major difference between reliability testing and functional (defect) testing is that with reliability testing the test inputs aim to mimic real life (i.e. the operational profile) rather than being aimed at exercising specific functions or finding specific types of fault. Metrics required to measure the reliability - for continuously-running systems metric such as the mean time to failure (MTTF) and probability of failure on demand (POFOD) are generally used.
Nishant Worah 2009
11

Maintainability

IEEE 610.12:1990 the ease which a software systems or component can be modified to correct faults, improve performance, or other attributes, or adapt to a changed environment fault removal is typically only a small part of the overall maintenance burden. The majority of maintenance tasks are concerned with implementing new requirements and adapting to new environments, such as a new operating system. Measure of software maintainability are:

MTTR (Mean time to repair) and MTTM (Mean time to modify)

Nishant Worah 2009
12

Maintainability
cont..

Indirect measure of software maintainability

Software design Language and operative system Design and coding standards

Modularity Documentation Structure coding techniques Code reliability

Test case suite and regression testing

Nishant Worah 2009
13

Availability

IEEE 610.12:1990 The degree to which a system or component is operational and accessible when required for use The probability that the system will be functioning correctly at any given time Example Service Help desk must be available 24/7 Phone system must:

be available at least 98% of the time Unavailable no more than 2% of the time

There is a close relationship between reliability, availability and maintainability. A (mythical) perfectly reliable system never fails and so is always available, and also requires no maintenance to fix faults
Nishant Worah 2009
14

Recovery

IEEE 610.12:1990-The restoration of system, program, database, or other system resource to prior state following a failure or externally caused disaster

Nishant Worah 2009

15

Usability

ISO/IEC 9126-1 (2000) - Usability: the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions. Comparing usability testing with Functional Testing concentrates on whether the software performs the task correctly, it does not test how well the user is able to use software to perform the tasks.
Nishant Worah 2009
16

Configuration

Configuration testing is testing the product under a different number of hardware or software configurations.

Hardware and software can sometimes have an unlimited number of configurable items and this would be too much to test, so it is advisable that the product be tested with, at least, the minimum, the maximum configuration and a few different configurations in between.

Nishant Worah 2009

17

Security

BS7925-1:1998 Testing whether the system meets its specified security objectives So what are the security objectives ?

Confidentiality e.g. Information leakage Integrity e.g. using spoofing to fool software to accept input it should reject, or by corrupting data held in the system Availability e.g. by denying access to authorised users

There are various security mechanisms that are used to implement these areas, such as:

Authentication Authorisation Auditing

Nishant Worah 2009
18

Performance
Performance Requirement - IEEE 610.12:1990

A requirement that imposes conditions on a functional requirement; for example a requirements that specifies the speed, accuracy, or memory usage with which a given function must be performed
Performance -IEEE 610.12:1990

The degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage.
Performance Testing IEEE 610.12:1990, BS-7925-1:1998

Testing conducted to evaluate the compliance of a system or component with specified performance requirements.
Nishant Worah 2009
19

Performance Testing
Performance testing helps to determine whether software will meet the performance requirements such as

Response time Concurrency Resource consumption and Volumetrics

Performance testing touches on many areas including:

Load testing Stress testing Volume testing Soak testing Scalability

Nishant Worah 2009
20

Performance Testing
cont..
1) Done correctly Time (Response Time) Rate (Throughput) Resource (Utilisation) 2) Done incorrectly Reliability

Request for service

System
3) Can not do
Nishant Worah 2009

Availability

21

Thank You !!!

Nishant Worah 2009

22