10 1 1 27

University Of Strathclyde Department Of Computer Science Research Report/94/171
RELATIONSHIP BETWEEN THE ACTIVITIES OF

THE SOFTWARE PROCESS AND THE QUALITY OF
THE SOFTWARE PRODUCT
by
Ian H G Woodman
Software Quality Research Group
Department of Computer Science
University Of Strathclyde
Livingstone Tower
26 Richmond Street
GLASGOW G1 1XH
e-mail: ihgw@cs.strath.ac.uk
Research Report/94/171 December 1994
Abstract
The main justification for process improvement is that the quality of the
software product will improve as a result. This seems intuitive but
unfortunately there is no empirical evidence to prove its validity yet.
The purpose of this paper is to present a possible mapping between the activities
of the software process and the quality of the software product. It is argued that
the product quality characteristics should be one of the prime drivers when
assessing and improving the quality of an organisation's software process as it
is the quality of the product about which we are increasingly concerned.
Page 1
1. Introduction
1.1. Software Product Quality
The term "Software Product Quality" is a rather general concept that is

influenced by many different variables, some of which are easier to quantify
than others. To determine if one piece of software is of higher quality than
another is a difficult task at present; it depends on our idea of quality.
The ISO 9000 series of standards (known as BS5750 in the UK) defines quality
as:-
"The totality of features and characteristics of a product or service that bear on

its ability to satisfy stated or implied needs" [1]
This is a very general view of quality, intended to encompass all products, not
just software.
Fenton [2] sites three common definitions of software quality with very
different connotations:-
"Fitness for purpose" "Conformance to specification" "Degree of excellence"
The term "software quality" has different meanings for different people in a
software organisation. A marketing managers view of the quality of a product
will not be the same as a software engineers view. These views are often
conflicting and the term quality cannot be defined in such a way as to
encompass these.
1.2. Software Quality Characteristics
In the ISO definition given above, "totality of features and characteristics"

implies that quality is comprised of many components.
The idea of decomposing the intangible concept of software quality into a
number of more discernible factors is not a new one and many attempts have
been made to find a collection of measurable "-ilities" to achieve this goal. Any
textbook on Software Engineering will lay testimony to this. By decomposing
quality into a number of factors, we are essentially providing a definition of it.
The first work in providing a software quality framework was conducted by

McCall, Richards and Walker (1977) [3] after their well-known work at the
Rome Air Development Centre in New York State. They produced a list of
eleven factors or characteristics which they believed encompassed the range of
aspects of software quality.
Around the same time, Boehm et al (1978) [4] published an independent list of
seven factors. Years later, Bowen, Wigle and Tsai (1985) [5] produced a
longer list (thirteen factors) again from work at RADC. Later attempts included
Grady and Caswell (1987) [6], Deutsch and Willis (1988) [7], Forse (1989) [8]
and von Maryhauser (1990) [9].
Page 2
Boehm et al McCall et al Bowen et al

Efficiency Efficiency Efficiency
Reliability Reliability Reliability
Human Engineering Usability Usability
Modifiability Flexibility Expandability
Portability Portability Portability
Testability Testability Verifiability
Understandability Reusability Reusability
Maintainability Maintainability
Interoperability Interoperability
Correctness Correctness
Integrity Integrity
Flexibility
Survivability
Figure 1.1 - Software Quality Factors
Although the lists are similar and appear to contain some identical factors, these
are defined differently in each model. This multitude of proposals with many
different definitions of characteristics resulted in a lot of confusion as to what
software quality actually meant. It is for this reason that the ISO 9126 [10]
model was developed, so that international agreement could be reached on what
the software quality characteristics should be.
The different models did have one thing in common; each was based on a
Factor - Criteria - Metric paradigm whereby each quality Factor is
composed of a number of Criteria which in turn can be assessed according to a
number of defined Metrics. These metrics are actual measurements taken from
components of the software product.
It is interesting to note that further work has emerged since the publication of
ISO 9126 (1991), for example Gillies (1993) [11].
1.3. ISO 9126
ISO 9126 is an international standard which defines six quality characteristics of

software and presents guidelines for their use. It was the result of some five
years discussion and is an attempt to consolidate the many different views of
software product quality that have been proposed in the past.
The standard definitions:-
• provide complete coverage of all aspects and views of software

quality
• are minimally overlapping
• are applicable to every kind of software
• apply to software throughout its lifetime
Page 3
In an appendix of the standard, each quality characteristic is broken down into a

number of sub-characteristics to help guide their assessment. This
decomposition is not part of the standard yet as the field is not mature enough,
but for the purposes of this paper, the sub-characteristics have been used as they
are necessary to give a more detailed understanding of the coverage of each
definition.
Characteristics Sub-Characteristics
Suitability
Accuracy
Interoperability
Compliance
Security
Maturity
Fault Tolerance
Recoverability
FUNCTIONALITY
Understandability
RELIABILITY Learnability
USABILITY Operability
EFFICIENCY Time Behaviour

Resource Behaviour
MAINTAINABILITY
Analysability
PORTABILITY
Changeability
Stability
Testability
Adaptability
Installability
Replaceability
Conformance
Figure 1.2 - The ISO 9126 Model
1.4. Process and Product
Although ISO 9126 relates to the characteristics of the software product, there is
an intuitive idea that the quality of the process used to produce the software will
have an effect on the quality of the end product. This is an assumption of all
software process improvement work as stated in the Software Engineering
Institute's (SEI) Capability Maturity Model (CMM), one of the most successful
of previous improvement frameworks:-
"The quality of a software system is governed by the quality of the process used
to develop and maintain it" Watts Humphrey [12]
This is generally accepted within the software engineering community with a

few exceptions, the opposing argument being that there is no solid evidence to
substantiate the claim. However, without this assumption, the field of software
process improvement is without foundation and, as with all areas of scientific
discovery, assumptions must be made for progress to take place. (The SEI is
Page 4
now beginning to collect evidence from its assessments to support the CMM
and this will hopefully prove the validity of this premise).
In the past, most work in the field has concentrated on either the process
approach or the product approach but not both, and the relationship between the
two is not well understood.
The purpose of this paper is to present a possible mapping between the activities
of the software process and the quality of the software product. It is argued that
the product quality characteristics should be one of the prime drivers when
assessing and improving the quality of an organisation's software process as it
is the quality of the product about which we are increasingly concerned.
2. The SPICE Project

This work was originally carried out as part of the Software Process
Improvement and Capability dEtermination (SPICE) project [13]. This project
was the result of an ISO study in 1991 to investigate the need for an
international standard for software process assessment which can be used by
software organisations to assess and improve the capability of their processes
and to assess the capability of potential suppliers of software.
The project aims to build on the best features of existing assessment methods
that have been used extensively in the past, including:-
• The Capability Maturity Model (Paulk et al, 1993) developed by

the Software Engineering Institute (SEI) in the United States
• TRILLIUM (Bell Canada, 1992): Telecom Software Product

Development Capability Assessment Model developed by Bell
Canada, Bell Northern Research and Northern Telecom
• The Software Technology Diagnostic (Compita, 1993)

developed by the Scottish Development Agency and enhanced by
Compita in the UK
• The Bootstrap (CEC, 1990-92) method developed as part of a

European ESPRIT project
At the heart of SPICE is the Baseline Practices Guide (BPG) [14] which defines
at a high level the set of activities that are considered to be fundamental to good
software engineering. These are the practices that should be implemented to
establish and improve an organisations software development, maintenance,
operation and support capabilities. SPICE uses the BPG as a basis to assess the
capability of an organisations software process.
The BPG consists of five cohesive process categories:-

Engineering - Processes that directly specify, implement or maintain a system
and software product and its user documentation. These are the familiar
activities of the classic Waterfall model but are intended to be applicable to any
lifecycle model
Page 5
Customer-Supplier - Processes that directly impact the customer, supporting

development and transition of the software to the customer and provide for its
correct operation and use
Project - Processes which establish the project, and coordinate and manage its
resources to produce a product or provide services which satisfy the customer
Support - Processes which may be employed by any of the other processes

(including other supporting processes). The supporting processes can be
employed at various lifecycle points and may be performed by the organisation
employing them, the customer, or by an independent organisation
Organisation - Processes which establish the business goals of the
organisation and develop process, product and resource assets which, when
used by the projects in the organisation, will help the organisation achieve its
business goals
Each of these categories consists of a set of processes which in turn are made up
of a number of base practices. A base practice is a software engineering or
management activity that addresses the purpose of a particular process.
The base practices represent the lowest level of the architecture and describe
individual activities but do not place any restrictions on how they are
implemented.
3. Approach
The analysis was essentially product based, taking each of the ISO Product
Characteristics in turn and deciding to what extent each of the base practices
affect its attainment.
The degree to which a characteristic is affected by a particular base practice was

measured/illustrated on a 4-point (ordinal) scale. This is similar to the way an
organisations base practices are assessed during a SPICE process assessment
and gave a simple but rich way of rating the practices.
Not Supportive The base practice does not to any degree

contribute to providing / improving / assuring the
quality characteristic.
Partially Supportive The base practice does a little to contribute to

providing / improving / assuring the quality
characteristic.
Largely Supportive The base practice largely contributes to providing

/ improving / assuring the quality characteristic.
Fully Supportive The base practice critically / fully contributes to

providing / improving / assuring the quality
characteristic.
Those practices that are fully supportive will have a direct effect on the degree of
the characteristic in the final product.
Page 6
A matrix was used to record the results of the analysis and the most important
practices in each process category were identified. These are discussed in the
following sections. As SPICE is not yet in the public domain, the actual
matrices cannot be reproduced here as they detail the BPG architecture.
Much of the analysis is subjective and the mapping is by no means definitive. It

is hoped however that a consensus of expert opinion would produce broadly
similar results. This has been satisfied in part from further reading (eg. Glass
[15]).
The Organisation process category was found to be of a different nature to the

rest of the process categories as it is at a much higher level. It is however
thought that most of the processes within it will be partially supportive of the
quality characteristics as they will improve the performance of all the other
practices. It is not examined in detail here.
4. Universally Applicable Base Practices

This is a description of those base practices that are considered to be equally
supportive of all six product characteristics. In the sections that follow
however, specific examples may be given to show the relevance of the practice
to an individual quality characteristic.
4.1 Universal Fully Supportive Practices
These practices fully support all of the product characteristics. During analysis,
it became apparent that they were all essential to the production of quality
software.
4.1.1. Engineering Process Category
Specify System Requirements This will include requirements relating to

all of the product quality characteristics. Proper specification will lead to greater
understanding and correct implementation of requirements.
Determine Software Requirements Capture and documentation of the

software requirements is the starting point for building quality software. Effort
spent at this stage will help to ensure that the characteristics are built in to the
software right from the beginning.
Analyse Software Requirements

Evaluate Requirements with Customer Analysis and discussion of the
requirements will ensure that they are fully understood before development
commences and this will greatly increase the probability that the product will be
built correctly.
Verify the Software Units

Integrate and test software
Integrate and test system Verification and validation activities are where
the presence and degree of attainment of each of the quality characteristics will
be evaluated. Defects that are detected and removed may relate to any of the six
characteristics.
Page 7
Maintain System and Software The maintenance of the system and

software will directly affect each of the quality characteristics in that each can be
improved by the detection and correction of defects in the software.
4.1.2. Customer-Supplier Process Category
Define the Requirements This covers the requirements practices described

above.
Negotiate Contract This practice includes establishing agreed procedures

for handling changes in requirements, product quality monitoring and standards
to be used amongst other activities and so is very important to the management
of requirements.
Obtain Customer Requirements and Requests

Understand Customer Expectations Again, these involve gaining a better
understanding of what the customer's requirements are.
Support Customer Acceptance Review This is where the customer

will assess if the software adheres to his/her requirements and expectations.
Install Perfective Upgrades This ties to "Maintain System and

Software" above. It encompasses the planning, testing and installation of
upgrades to correct defects or improve performance which can apply to any of
the six characteristics.
4.1.3. Project Process Category
Manage Requirements Just as the determination, discussion and

specification of requirements is critical to all characteristics, so the management
of these is as well. Agreement on requirements ensures synchronised
development and management of changes is very necessary to communicate
these to the right teams.
Manage Quality "Managing quality involves identifying the required

quality characteristics of the software, working to achieve this quality and
demonstrating that this quality was achieved." [14]
Each of these base practices has a fully supportive effect on the quality of the
end product. They help to track the quality throughout the lifecycle so that
problems are rectified early.
Conduct Technical Reviews Technical Reviews are a fundamental quality

technique. They include inspections and code walkthroughs.
Manage Commitments Any problems identified during reviewing will be

dealt with here and action taken to resolve them.
The following three practices are from the Manage Subcontractors process
and are only applicable if subcontracting occurs:-
Establish Statement of Work This is analogous to the determination and

specification of correct requirements.
Page 8
Assess Compliance Compliance to the agreed standards will help ensure

the quality of the software.
Assess Subcontractor Quality This will encompass assessment of the

software for the quality characteristics specified in the contract.
4.1.4. Support Process Category
Perform Quality Assurance Quality assurance is the process by which

each of the characteristics will be assessed and is thus fundamental.
Perform Problem Resolution The purpose of this is "to ensure that all
discovered problems are analysed and removed... whatever their nature or
source." This will involve problems and defects that will affect each of the
characteristics as well as deviations from the quality requirements themselves.
Perform Peer Reviews Peer reviewing is one of the most effective

techniques for assuring quality and finding certain types of faults in software.
Reviews of requirements, designs, code, changes, tests and post-delivery
reviews are all essential to ensure that the characteristics are present.
4.2. Universal Largely Supportive Practices
These are base practices that are considered to have a substantial effect on all of
the product quality characteristics but do not impact them as directly as the fully
supportive practices.
Prepare Acquisition Strategy The use of existing products can have

different effects on each of the product quality characteristics, as discussed later.
Prepare a Request Proposal

Select Software Product Supplier These two practices involve the
determination of project requirements and evaluation of suppliers to determine
which ones are most qualified to do the work.
Review Contract Before Finalisation Reviewing the contract will help

identify areas that are not properly understood before development begins.
Prepare for Customer Audits and Reviews Proper preparation before

reviewing makes them more valuable and effective.
Evaluate Options for Product Development Similar to Prepare

Acquisition Strategy above.
Page 9
Again, the following practices are only applicable if subcontracting occurs:-
Qualify Potential Subcontractors

Select Subcontractor The selection of the subcontractor(s) according to
their capability to do the work will have a substantial effect on the quality of the
product.
Establish and Manage Commitments If the subcontractor understands

the details of the project then it is more likely that it will run according to plan.
Maintain Communications Understanding is maintained throughout the

project by performing this practice.
4.3. Universal Partially Supportive Practices
These are base practices that are considered to affect all of the quality
characteristics but in an indirect way. That is, the characteristics are not
drastically affected by their performance but they will benefit from them. Most
of these are project management practices which improve the practices that
directly affect the characteristics.
Determine Interfaces to Independent Agents

Determine Interfaces to Subcontractors
Perform Joint Process Assessment
Assess Customer Satisfaction By determining customer satisfaction and
communicating this back throughout the organisation, the morale of the
employees should improve and have an effect on future work.
Select Software Lifecycle The effect of different lifecycles on the quality

of the software product is not well known but an iterative model would seem to
lead to greater understanding of the product being developed, particularly where
large systems are being built. This practice may not be applicable if the
customer specifies a particular model.
Describe Activities and Tasks

Establish Task Sequence
Document Activities
Develop Work Breakdown Structure No matter which lifecycle is used,
a description of the software engineering techniques and their associated tasks
will lead to a greater understanding of the software development and better
management.
Identify Initial Project Risks

Manage Risks Identification and mitigation of risks to the project will
reduce the chance of key practices not being performed properly because of
delays or failures.
Page 10
Develop Project Estimates

Identify Project Measures
Establish Project Schedule
Establish Project Commitments
Document Project Plans
Track Progress Establishing commitments to and tracking the project plans
increases the chance of them being understood and carried out.
Build Project Teams The careful selection of team members and their
support and management will have a positive effect on the performance of
software engineering practices.
5. Functionality
Functionality - a set of attributes that bear on the existence of a set of
functions and their specified properties. The functions are those that satisfy
stated or implied needs
Sub-characteristics
Suitability - Attributes of software that bear on the presence and

appropriateness of a set of functions for specified tasks
Accurateness - Attributes of software that bear on the provision of right or

agreed results or effects
Interoperability - Attributes of software that bear on its ability to interact

with specified systems
Compliance - Attributes of software that make the software adhere to

application related standards or conventions or regulations in laws and similar
prescriptions
Security - Attributes of software that bear on its ability to prevent

unauthorised access, whether accidental or deliberate, to programs and data
From these definitions, functionality can be seen as characterising the degree to

which software satisfies its functional and related requirements (such as
accuracy of results). Other non-functional requirements (such as how quickly
or how reliably it does this) are addressed by the other product quality
characteristics and it is independent of these.
Clearly, functionality is relevant to all software and may to some be the most
important of the six characteristics.
(In a survey of software producers carried out as part of the SCOPE project,
functionality and reliability were seen to be the two most important [16]).
The majority of the base practices can potentially affect functionality. However,
some can be identified as being critical to ensuring that the software actually
does what it is supposed to do and to as great an extent as possible.
Page 11
4.1. Engineering Process Category
System and Software Requirements Discussion and clear specification

of these will help to ensure that development of the right functions occurs. The
specification drives the design which drives the code. This is then tested against
the requirements so early, clear definition is essential.
Traceability at the Design Stage This is a form of verification used to

ensure that the design fulfils all functionality requirements before
implementation begins.
Verification and Validation are the best techniques for assessing the
functionality of software, (particularly black-box testing).
Maintenance is also essential if the level of functionality assessed is not

acceptable or if the functional requirements change during the lifetime of the
software.
4.2. Customer-Supplier Process Category
Performing Joint Audits and Reviews involves assurance that the

software is being developed to the customer's satisfaction.
Support Operation of Software is where previously undetected problems

with functionality will be identified and resolved.
Installation of Perfective Upgrades will involve correction of defects in

functionality possibly as a result of these identified problems.
4.3. Project Process Category
Different Options for Product Development may have different

implications for functionality. For example, an off-the-shelf product may have
all the required functionality already and this will have been thoroughly
validated already.
Identify Applicable Standards One of the sub-characteristics of

functionality concerns compliance to application-related standards and so it is
essential to identify these at an early stage.
Regular Management Reviews ensure that the software is complying with

the identified standards.
Technical Reviews These check functionality at all stages of the lifecycle,

thus increasing the chance of a high degree at the end of development. Action
taken to rectify lack of functionality identified by these reviews is of course just
as important.
4.4. Support Process Category
Peer reviews As mentioned previously, the processes in this category are

relevant to all of the product characteristics but in terms of functionality, peer
Page 12
reviews are especially important as they ensure adherence to functional

requirements at all stages of the lifecycle.
6. Reliability
Reliability - a set of attributes that bear on the capability of software to
maintain its level of performance under stated conditions for a stated period of
time
Sub-characteristics
Maturity - Attributes of software that bear on the frequency of failure by faults

in the software
Fault Tolerance - Attributes of software that bear on its ability to maintain a

specified level of performance in cases of software faults or of infringement of
its specified interface
Recoverability - Attributes of software that bear on the capability to re-

establish its level of performance and recover the data directly affected in case of
a failure and on the time and effort needed for it
Note : Most centres of expertise in software reliability use the following

definition:-
"The reliability of a piece of software is the probability that it will execute

without failure for a given period of time in a given environment" [2]
In the definition of Maturity above, "the frequency of failure by faults in the

software" is essentially the same as this definition.
The following statement appears in the ISO 9126 document itself:-
"Limitations in reliability are due to faults in requirements, design and

implementation"
Therefore, base practices that are supportive of these engineering practices will
help lead to a more reliable product.
Reliability is evaluated by random testing according to an operational profile - a
profile of the predicted usage of the software- in a simulation of the environment
in which it will be used. This will not usually take place until the software has
been integrated but testing of aggregates is also important as this may reveal
further faults in the software.
System Requirements explicitly include "performance of the system" which

will encompass the desired level of performance of the software. Therefore, the
specification of these requirements and the resulting "product configuration"
which transfers them to the relevant parts of the system are all important to
ensure that the desired level is worked towards.
Page 13
Software Requirements Correct specification and evaluation of these is

equally important for the same reasons.
Select Software Lifecycle In the case of an iterative lifecycle, the

reliability may be assessed at the end of each iteration and the feedback used to
modify other requirements that affect reliability (eg. system testing effort).
Verification and Validation activities may identify faults in the software
that could lead to failures and thus compromise reliability.
Testing the Integrated System is the most important activity for assessing
reliability, especially Develop Tests For System, where the operational
profile will be used to develop and guide test plans. The software will be tested
according to its intended use and in its intended environment and failure data
will be captured against which reliability is estimated. This can be evaluated
against the stated reliability requirements.
Maintenance is also very important to reliability as failures may occur after the
product is released, particularly at the beginning of its employment.
The Customer-Supplier base practices that are considered to affect reliability are
similar to those that affect the functionality of the software.
Definition of Requirements covers both system and software requirements

as highlighted above.
Discussion and Agreement on Contracts between both parties will

ensure that the integrity of requirements is preserved.
Prepare Acquisition Strategy Just as an off-the-shelf product may

provide all the functionality required, the reliability of it may be expected to be
higher, especially if it is widely used. This therefore has a positive effect on the
reliability of the final product. However, a different usage of software in a new
environment that it was not built for will affect the perceived reliability so
existing products must be evaluated carefully.
Obtain and Review Requirements and Requests Direct solicitation of

the customer to obtain these is most important as this is the place where the
information needed to construct the operational profile is gathered. This can
include proposed market, profile of users and operators and the customers
expectations of the software.
Customer Acceptance Review is important as this is where the customer

will decide if the level of reliability is acceptable, according to the original
requirements.
Support Operation Of Software Failure-free performance of the

software is what reliability is specifically concerned with and this will be
initially tested, monitored and improved, if need be by the base practices of this
process. Faults undetected by testing (eg. in little used functions) may cause
failures at this stage and these must be fixed to restore reliability. (N.B. - Only
these failure producing faults will have any major effect on overall reliability).
Page 14
Monitoring the Performance and

Installing Perfective Upgrades to improve performance will directly
increase reliability.
Evaluate Options for Product Development

Determine Reuse Strategy As stated above, the use of an already
established product can improve reliability. The evaluation of these products is
performed here and encapsulated by the above practice. This is also addressed
by the reuse of software components.
Identify Project Standards Standards for software reliability exist (eg.

BS5760 Part X : Assessment of Reliability of Systems Containing Software)
and if a high degree of reliability is desired, then these should be identified and
used as a guide.
Management Reviews include regularly checking for adherence to these

standards.
Technical reviews Although reliability cannot be guaged until the end of

system testing, quality techniques such as reviewing can help to 'build in'
reliability by identifying certain types of faults (see below).
Peer Reviews Apart from the universal supportive processes in this area,
peer reviews are especially supportive of reliability. These will help to find
defects in all work products - requirements, designs, code and test plans - that
affect the reliability of the software.
7. Portability
Portability - a set of attributes that bear on the ability of the software to be
transferred from one environment to another
Sub-Characteristics
Adaptability - Attributes of software that bear on the opportunity for its

adaption to different specified environments without applying other actions or
means than those provided for this purpose for the software considered
Installability - Attributes of software that bear on the effort needed to install

the software in a specified environment
Conformance - Attributes of software that make the software adhere to

standards or conventions relating to portability
Replaceability - Attributes of software that bear on opportunity and effort of

using it in the place of specified other software in the environment of that
software
Page 15
Portability can be a very important or very trivial product characteristic,

depending on the product being developed. For highly specialised bespoke
software which is only intended to run on specific hardware it may not matter,
but for more flexible products, the software may have to run on different
platforms and will usually outlive the hardware it is initially installed on.
Modern word-processors are good examples of this.
Portability is influenced mainly by the features and flexibility of the language

used to implement the software but also by the documentation supplied with the
software to ease its transfer to a different environment.
System and Software Requirements Once again, this will be the place
where the need for portability will be established and allocated to the appropriate
parts of the software architecture.
Designing the Software Careful consideration of these requirements

during design will make portability much easier to achieve. For example,
functionally equivalent units that are targeted for different environments may
need to be designed from the specification.
Establish Traceability This will help to ensure that the portability

requirements have been accounted for in the design.
Verification Once the code has been written, verification eg. by checking
for anomalies such as non-portable language features, will be one of the main
methods for assuring a portable product.
Determine Regression Test Strategy If portability is important, small

changes to code may introduce non-portable features and so regression testing is
important.
Testing of the Software Units is also essential as the target environment

will influence these tests and so the software should be tested for portability to
each of its specified environments.
Maintenance of the software is all important if it is to be ported to an

environment that it wasn't intended for and with which it is not compatible.
Prepare Acquisition Strategy may also affect portability as, for example,
off-the-shelf or customer-furnished products may only be specifically tailored to
one environment.
Technical Reviews Ensuring that development of the product fulfils

portability requirements is part of technical reviewing.
Management Reviews will seek to ensure adherence to portability standards.
Package, Deliver and Install the Software This is one Customer-

Supplier process that is essential for portability but does not relate to any of the
Page 16
other product characteristics. The sub-characteristic Installability will be fully

supported by most of the base practices of this process.
Once the software is operating in its specified environment, portability is not

relevant as the software need not be moved.
Install Perfective Upgrades This may be needed if the software is to be

ported to an environment it was not intended for (Replaceability).
7.3 Project Process Category

Determine Reuse Strategy Careful evaluation of existing products and
reuse of existing components may have an effect because existing products may
be tailored only to specific environments.
Identify Project Standards Use of standard language features,

environments and operating systems during development is very important for
the portability of software and so early identification and conformance (a sub-
characteristic) are essential.
Management Reviews
Manage Commitments Adherence to standards is again checked by regular
management reviews and deviations are handled by taking action with the
appropriate groups.
Identify Specialised Facilities If the software is to be able to be operated

on target different hardware platforms, these must be acquired at an early stage
so that portability can be tested as early as possible.
The lifecycle model selected should not really affect portability.
This is a very important category for portability because as well as the universal
practices, Develop Documentation and Configuration Management are
fully supportive of it.
Develop Documentation Good documentation decreases the effort needed

to install the software and port it to a different specified environment.
Configuration management can also be extremely important as highly

portable software may exist as a library of many units with different
configurations being constructed for different environments. A good example
of this would be a word-processing package where the same functionality is
needed but for entirely different computer systems and where new releases and
upgrades need to replace their predecessors (Replaceability).
Peer Reviews As mentioned before, these are an effective method of

detecting anomalies such as non-portable language features.
Page 17
8. Efficiency
Efficiency - a set of attributes that bear on the relationship between the level
of performance of the software and the amount of resources used, under stated
conditions
Sub-characteristics
Time Behaviour - Attributes of software that bear on its response and

processing times and on throughput rates in performing its function
Resource Behaviour - Attributes of software that bear on the amount of

resources and the duration of such use in performing its function
Efficiency is the measure of the timing and sizing characteristics of software. It

should be addressed at all stages of the lifecycle, from determining performance
requirements through to maintenance if requirements or system throughput
change.
System and Software Requirements The performance characteristics of

a piece of software are stated explicitly in the requirements and as usual,
specification and analysis of these is wholly supportive.
Software Design Careful design, for example to minimise Input/Output

transactions and employ efficient data structures (both in terms of time and
space) as well as choice of algorithm, language and compiler are all critical to
efficiency as is the traceability of these to meet requirements.
Testing of Individual Units and Aggregates of Units Performance

testing to assess the efficiency of components against their requirements and
identify areas for optimisation is important.
Testing of the Integrated Software is also crucial because interfaces

between components can introduce inefficiencies.
Determine Regression Test Strategy Slight changes in code can reduce

efficiency so this is also important.
Integration and Testing of the System is the place where the efficiency
can be analysed in the actual environment and further bottlenecks identified.
Modification of other system elements such as hardware can improve software
performance.
Maintain System and Software All modifications are handled by this

process and it is critical if the requirements should change or the throughput of
the system increases.
Page 18
Prepare Acquisition Strategy Careful evaluation of existing software is

important if efficiency is a priority.
Support Operation of Software The operation of the released software is

where the efficiency will be tested in the real environment that it was intended
for and so this is a very significant area. Important base practices are:-
Identify Risks to System Functionality Failure of for example hardware

or peripherals can seriously affect the efficiency of software.
Operational Testing
Resolution of Problems Testing the efficiency under real conditions can
uncover problems not identified during development.
Handle User Requests Although the operator is more likely to be aware of

efficiency than the user, performance requests may come from here as well.
Temporary Work-Arounds may help maintain the level of performance.
Monitoring of System Capacity and Service is very supportive as this

will include time and space problems.
Monitor Performance
Install Perfective Upgrades In the same vein, these two practices are
vital in providing the required level of efficiency.
Joint Technical Reviews and the

Customer Acceptance Review are important to make sure that the
efficiency meets the customer's expectations.
Determine Reuse Strategy Reuse of known efficient algorithms is

beneficial but previously produced software may or may not have a positive
effect on efficiency so careful evaluation is critical.
Identify Specialised Facilities Analysis of efficiency may require

specialised test equipment such as performance analysis tools and requirements
must be defined early on.
Technical reviews, especially in the form of code inspections are another

method for identifying efficiency bottlenecks and optimisation of these areas is
critical.
This category is not as directly relevant to efficiency as it is to the other quality

characteristics but the three universally supportive processes are just as relevant.
The importance of Problem Resolution relating to performance (identified
by 'Monitor Performance') and Peer Reviews has been highlighted earlier.
Page 19
9. Usability
Usability - a set of attributes that bear on the effort needed for use and on the
individual assessment of such use by a stated or implied set of users
Sub-characteristics
Understandability - Attributes of software that bear on the users' effort for

recognising the logical concept and its applicability
Learnability - Attributes of software that bear on the users' effort for learning
its application (eg. operation control, input, output)
Operability - Attributes of software that bear on the users' effort for operation
and operation control
"It is generally agreed that well structured manuals, informative error messages,
help functions and consistent interfaces are the means to achieve usability"
Fenton [2]
It seems intuitive that all these factors will make software easier to use but
during development, interfacing with the user to assess the interface and
education once the product is released are highly supportive as well.
System and Software Requirements Definition and analysis of the

interface requirements through "human engineering" and "interface (to user) and
its characteristics" explicitly involve the usability characteristics desired in the
final product.
Updating the Requirements Prototyping is often used to evaluate user

interfaces and so review after each iteration is important (if applicable).
Design of the [User] Interface is probably the most important aspect of

usability.
Unit Testing and

Testing of Aggregates will involve interface testing for response times and
error reports for example.
Testing of the Integrated Software will involve verification such as

consistency checks.
Testing of the Integrated System Equally, combining the software with

manual system elements and testing will assess how usable the software is in
the context of the wider system.
Maintain System and Software User requests for enhancements (from

'Handle User Requests') must be addressed to maintain usability and this is
handled by the maintenance process.
Page 20
Upgrading of the System will include additional user training which will
ensure that modifications do not compromise usability.
Evaluation of Existing Products may include the user interface and it will
be important to decide which choice is best depending on factors such as current
trends and target market.
Understand Customer Expectations Prototyping or demonstration of

existing interfaces may be needed to ascertain the customers desires.
Joint Management Reviews

Joint Technical Reviews
Customer Acceptance Review The development of the interface will be
controlled, monitored and evaluated by the customer via these three practices.
All of the base practices of the Support Operation of Software process are
irrelevant to usability except:-
Handle User Requests which is highly relevant. This is where user

problems and requests are managed and this has a direct effect on the usability.
A distinction is made here between the operator of the software and the end-user
as usability would only be relevant to the latter.
Training the Customer to use the software and providing user

documentation can be seen as one of the most effective ways to attain usability
as can Product Support facilities such as helplines. These practices are
carried out after development and are not really attributes of the software (as per
ISO 9126) but improve usability in general.
Monitor Performance Performance of the software can have an effect on

its usability and so monitoring this is beneficial.
Installing Perfective Upgrades in response to user problems or

performance problems is fully supportive of usability.
Evaluation of Existing Interfaces As already discussed above, these

practices are important as most applications adopt a standard interface and often
reuse (parts of) interfaces developed for other applications.
Select Software Lifecycle An iterative lifecycle can improve usability by

evaluating the interface after each iteration and using the feedback to improve it.
Management Reviews
Technical Reviews Review of the software by management and more
importantly by peers gives a valuable subjective evaluation of the usability.
Page 21
Development of Documentation is one of the most important processes as

far as usability is concerned because well-written, well-structured user manuals
and training documentation make the software significantly easier to use and
learn.
Peer Reviews are also important when evaluating the usability of an interface
as discussed above.
10. Maintainability
Maintainability - a set of attributes that bear on the effort needed to make
specified modifications
Sub- characteristics
Analysability - Attributes of software that bear on the effort needed for

diagnosis of deficiencies or causes of failure, or for identification of parts to be
modified
Changeability - Attributes of software that bear on the effort needed for

modification, fault removal or for environmental change
Stability - Attributes of software that bear on the risk of unexpected effects of

modifications
Testability - Attributes of software that bear on the effort needed for

validating the modified software
Ease of maintenance is desirable for all software, as even the smallest systems
will undergo change in their lifetime. As well as easy understanding and
analysis of the code, proper supporting documentation is critical.
1 0 . 1 . Engineering Process Category
System and Software Requirements explicitly state "maintenance

requirements" and these will have a direct effect on the level of maintainability
required of the product. Testable requirements are highly desirable to improve
the testability of the actual software and their ease of understanding is also
important.
Develop Software Design Good design principles such as modularity low

coupling between modules make code much easier to understand and follow,
and complexity will also have a substantial effect.
Requirements Traceability can also be particularly useful here; if a

requirement changes, the resulting change to design can be followed through
and this will be true later for coding and testing. This reduces the effort needed
to make modifications.
Page 22
Development and Documentation of all Test Procedures is essential

to maintainability as these will be used for re-validating the software when
future modifications are made.
Regression Test Strategy is very important as part of this development.
Maintain System and Software is one of the most important processes for
maintainability as this is where maintenance is analysed, performed and
documented using the previous engineering practices.
1 0 . 2 . Customer-Supplier Process Category
Prepare Acquisition Strategy Already developed software may not come

with maintenance documentation and may or may not be easily understood.
Also, the maintenance team any not be able to communicate with the original
developers, so careful evaluation is necessary here if a high level of maintenance
is anticipated.
Review Contract Before Finalisation

Negotiate Contract Within these practices, special attention is given to the
"procedures for handling changes in requirements and customer detected
problems". These procedures will clearly be important with a view to
improving maintainability by easing the transition from the initial report to actual
modification of the software.
Joint Technical Reviews with the customer are similar to those in the
Project Process Category.
Customer Acceptance Review will include checking for maintenance

documentation.
Perform Operational Testing

Resolve Operational Problems
Handle User Requests As mentioned in the Engineering Process Category,
testing will help locate parts of the software that need modification and this
applies also to operational testing. Recording the results of these practices for
later reference is crucial.
Planning, Testing and Installation of Perfective Upgrades to the

software and especially documentation makes for more maintainability.
1 0 . 3 . Project Process Category

Determine Reuse Strategy Maintainabilty of existing software is discussed
above.
Build Project Teams The management of project teams can be seen as

being more relevant here than to the other quality characteristics. Modifications
to the software may involve more than one team and so conflict resolution and
interaction methods are important. (Communication with the teams after the
software has been released may be required).
Page 23
Maintain Traceability is especially supportive as if modifications to

requirements are made, these must be able to be traced through design and code
to make modification of the software easier.
Technical Reviews are supportive in that low maintainability, for example

through poor readability and high complexity of code, can be spotted at an early
stage and rectified.
1 0 . 4 . Support Process Category
Develop Documentation Good documentation is essential to maintainability

as software would be extremely hard to understand and modify without it.
Likewise, documentation of the maintenance performed on the software since
release is essential.
Perform Configuration Management As software is modified, version

control and configuration management become vital to track changes to code and
documentation.
Problem Resolution Proper tracking and recording of modifications to the

software is critical to ease future modifications.
Peer Reviews The importance of these has already been asserted above.
11. Conclusions
Much of the work here is subjective but well-known authors in the field of
software quality (eg. [2],[15],[16]) have suggested similar relationships
between the activities of the software process and software product quality in
the past. Empirical evidence to prove the relationships would be hard to collect
and validate and most decisions have been based on intuition rather than
objective method. However, it is held that a direct mapping between ISO 9126
and the Base Practices does exist and that this paper represents one possibility.
By making the relationship between the process and product explicit, we can
begin to better understand the nature of software production.
Page 24
12. References
[1] ISO 9001, 'Quality Systems - Model for Quality Assurance for
Design/Development, Production, Installation and Servicing',
(International Standards Organisation)
[2] Fenton, N. E.: 'Software Metrics - A Rigorous Approach', (Chapman

& Hall, 1991)
[3] McCall, J. A., Richards, P. K. and Walters, G. F.: 'Factors in

Software Quality - RADC-TR-77-363', (Rome Air Development Centre,
1977)
[4] Boehm, B. W. et al : 'Characteristics of Software Quality', (North

Holland, 1978)
[5] Bowen, T. P., Wigle, G. B. and Tsai, J. T.: 'Specification of Software

Quality Attributes - RADC-TR-85-37', (Rome Air Development Centre,
1985)
[6] Grady, R. B. and Caswell, D. L.: 'Software Metrics: Establishing a

Company-Wide Program', (Prentice-Hall, 1987)
[7] Deutsch, M. S. and Willis, R. R.: 'Software Quality Engineering',

(Prentice-Hall, 1988)
[8] Forse, T.: 'Qualimetrie des Systems Complexes', (Les Editions

d'Organisation, 1989)
[9] von Maryhauser, A.: 'Software Engineering Methods and

Management', (Academic Press, 1990)
[10] ISO 9126, 'Software Product Evaluation - Quality Characteristics and

Guidelines for their Use', (International Standards Organisation, 1987)
[11] Gillies, A. C.: 'Software Quality - Theory and Management', (Chapman

and Hall, 1993)
[12] Software Engineering Institute, 'Affiliate Symposium', (Carnegie

Mellon University, 1987)
[13] Dorling, A., 'Software Process Improvement and Capability

dEtermination', Software Quality Journal, 2(4):209-238, December
1993
[14] BPG Product Development Team: 'Baseline Practices Guide - Version

0.06', (SPICE Project, 1994)
[15] Glass, R. L.: 'Building Quality Software', (Prentice Hall, 1992)
[16] Bache, R. and Bazzana, G.: 'Software Metrics for Product

Assessment', (McGraw Hill, 1994)
[17] Humphrey, W. S.: 'Managing the Software Process', (Addison

Wesley, 1989)
Page 25
[18] Vincent, J., Waters, A. and Sinclair, J.: 'Software Quality Assurance -
Volume 1 - Practice and Implementation', (Prentice Hall, 1988)
Page 26

10 1 1 27

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

10 1 1 27

Încărcat de

Drepturi de autor:

Formate disponibile

University Of Strathclyde Department Of Computer Science Research Report/94/171

RELATIONSHIP BETWEEN THE ACTIVITIES OF

Research Report/94/171 December 1994

The term "Software Product Quality" is a rather general concept that is

"The totality of features and characteristics of a product or service that bear on

"Fitness for purpose" "Conformance to specification" "Degree of excellence"

1.2. Software Quality Characteristics

In the ISO definition given above, "totality of features and characteristics"

The first work in providing a software quality framework was conducted by

Boehm et al McCall et al Bowen et al

Figure 1.1 - Software Quality Factors

1.3. ISO 9126

ISO 9126 is an international standard which defines six quality characteristics of

The standard definitions:-

• provide complete coverage of all aspects and views of software

• are minimally overlapping

• are applicable to every kind of software

• apply to software throughout its lifetime

In an appendix of the standard, each quality characteristic is broken down into a

EFFICIENCY Time Behaviour

Figure 1.2 - The ISO 9126 Model

1.4. Process and Product

This is generally accepted within the software engineering community with a

2. The SPICE Project

• The Capability Maturity Model (Paulk et al, 1993) developed by

• TRILLIUM (Bell Canada, 1992): Telecom Software Product

• The Software Technology Diagnostic (Compita, 1993)

• The Bootstrap (CEC, 1990-92) method developed as part of a

The BPG consists of five cohesive process categories:-

Customer-Supplier - Processes that directly impact the customer, supporting

Support - Processes which may be employed by any of the other processes

The degree to which a characteristic is affected by a particular base practice was

Not Supportive The base practice does not to any degree

Partially Supportive The base practice does a little to contribute to

Largely Supportive The base practice largely contributes to providing

Fully Supportive The base practice critically / fully contributes to

Much of the analysis is subjective and the mapping is by no means definitive. It

The Organisation process category was found to be of a different nature to the

4. Universally Applicable Base Practices

4.1 Universal Fully Supportive Practices

4.1.1. Engineering Process Category

Specify System Requirements This will include requirements relating to

Determine Software Requirements Capture and documentation of the

Analyse Software Requirements

Verify the Software Units

Maintain System and Software The maintenance of the system and

4.1.2. Customer-Supplier Process Category

Define the Requirements This covers the requirements practices described

Negotiate Contract This practice includes establishing agreed procedures

Obtain Customer Requirements and Requests

Support Customer Acceptance Review This is where the customer

Install Perfective Upgrades This ties to "Maintain System and

4.1.3. Project Process Category

Manage Requirements Just as the determination, discussion and

Manage Quality "Managing quality involves identifying the required

Conduct Technical Reviews Technical Reviews are a fundamental quality

Manage Commitments Any problems identified during reviewing will be

Establish Statement of Work This is analogous to the determination and

Assess Compliance Compliance to the agreed standards will help ensure

Assess Subcontractor Quality This will encompass assessment of the

4.1.4. Support Process Category

Perform Quality Assurance Quality assurance is the process by which