A Critical Analysis of A Hierarchy Based

Usability Model
Deepak Gupta Dr. Anil Ahlawat Kalpna Sagar
Maharaja Agrasen Institute of Krishna Institute of Engineering Institute of Information
Technology (MAIT) and Technology (KIET) Technology and Management
Delhi, India Ghaziabad, India (IITM)
myself.deepakgupta@gmail.com Delhi, India

Abstract: — For any software, its evaluation is significant for managing, controlling so that we can improve a software development
process. For such evaluation of software, many factors have been recognized in literature surveys. Quality is one of most important factor
which cannot be measured easily, because of its dependency on various other factors. Usability is such important factor on which quality
of software depends. But as far as real world is concerned, we face a major obstacle in implementation of any usability model because
there is a lack in precise definition and concept of globally accepted usability. Literature tends to define usability in overly brief and with
ambiguous terms and describe its application in informal terms. Many software Usability Models which has developed so far, do not
include all usability aspects and it is difficult to integrate these model into current software engineering practices. This paper aims to
define and explain the term ‘usability’ using proposed detailed taxonomy. This paper also presents a critical analysis and evaluation of
hierarchy based usability of software systems. Our proposed hierarchy based usability model presents 7 major attributes and 23 sub-
attributes and these sub-attributes are further decomposed into 42 characteristics which collectively test the usability of a software
product.

Keywords: - software engineering, quality models, usability model, taxonomy, software quality, human computer interaction, hierarchical
model, usability attributes, metrics.

I. Introduction Nielsen, 1993 defines it as “unnecessarily

Software quality is assessed by external and internal criteria.
When the software is in operational mode then experiences of
user are collectively reflected as external quality whereas
code dependency which is not visible to the end user reflects
as internal quality. External quality is very critical to the user,
while internal quality is meaningful to the developer only.
Fig.1. ISO/IEC 9126 software quality model
For evaluation of software, the main challenge is for the
selection of quality characteristics which can be divided
further into different sub-characteristics. ISO/IEC 9126
introduces a base model which classifies software quality
into six categories: portability, usability, maintainability
reliability, efficiency, and functionality which are further
divided into measurable sub-characteristics (Fig. 1.).
The concept of usability derives from the term user
friendly. Many Experts defines the term usability.
Chandor, Graham, & Williamson, 1985 explains usability
in terms of expressions[21], Bevan, Kirakowski, &
Maissel, 1991 explain usability as “acquired a host of
undesirably vague and subjective connotations”[16],
anthropomorphic”[11]. However, all definitions tend to be
brief and informal.
At present, most widely accepted definitions of usability,
are from ISO 9241-11 (1998) and ISO/IEC 9126-1 (2001).
There are various standard or models which characterize
the quality of software and defines the term usability as
follows:
a. The ISO/IEC 9126 defines the usability in terms of
the effort needed for use [9].
b. The ISO/IEC 9126 again redefines the definition of
usability as capability of the software to be
understood by user under certain conditions.
c. The ISO 9241-11 defines usability in terms of
efficiency, satisfaction and effectiveness in a
specied context of use [8].
d. The IEEE Std.610.12-1990 defines usability in terms
of learnability, Input and output efficiency of system
[7].
In simple terms Software Usability is the ease of use and
learnability of a human-made object. The object of use
can be a software application, website, book, tool,
machine, process, or anything a human interacts with. A
usability study may be conducted as a primary job
function by a usability analyst or as a secondary job
function by designers, technical writers, marketing
personnel, and others.
Lot of terms of usability described till now, are informal.
Researchers and standards have not gained consensus in
usability [2], [3]. In research work, Support System for
Novice Researchers is one of supporting tool for usability
evaluation [27].This research theme requires attention so
that we can overcome existing limitation, hence
motivation for this paper aims to describe the concept of
usability which is based on seven attributes, namely,
Efficiency, Productivity, Effectiveness, Memorability,
Safety, Universality and Satisfaction. Here taxonomy is
presented which explains each attributes and their sub-
attributes in non-redundant format.

978-1-4799-6629-5/14/$31.00 2014
c IEEE 255
 II. Literature Review
In the past few decades, many usability models have been
proposed to measure the usability of a software system in
terms of various attributes. These usability attributes are
classified among sub-attributes which can be divided
further into various characteristics. This hierarchy
structure is represented in Fig. 2.
Fig. 2. Hierarchy Structure
This section describes some of existing usability models
and their attributes. Proposed Models by number of
usability experts:
a. The classifications by Nielsen (1993) and by Nielsen
and Loranger (2006) refer to satisfaction, errors,
efficiency, memorability and learnability as usability
attributes [11] [24]. Some researchers, including
Holzinger (2005), consider these to be widely
accepted attributes [23].
b. Preece and colleagues developed an initial
classification that included enjoyableness, efficiency,
effectiveness, and safety (Preece, Davies, Benyon
Keller, & Rogers, 1993) [12]. Subsequently, they
proposed a new classification composed of attitude,
flexibility, throughput and learnability (Preece et al.,
1994) [13].
c. Quesenbery (2001, 2003, and 2004) listed the
attributes of a usable product as error tolerance,
engagement, effectiveness, efficiency, and ease of
learning [25].
d. Abran et al. (2003) proposed extending the ISO 9241-
11 definition by adding two further attributes,
namely, learnability (already adopted by IEEE, 1990;
ISO/IEC 9126-1, 2001; Nielsen, 1993) and
security[2].
e. The classification by Seffah, Donyaee, Kline, and
Padda (2006), defines 10 usability factors
(efficiency, effectiveness, productivity, satisfaction,
learnability, safety, trustfulness, accessibility,
universality, and usefulness) associated with 26
measurable usability criteria [1].
Various attributes of software are identified by different
standards or models. There are also varying denitions
across different sets of standards or authors concerning
more specic attributes (facets, aspects, factors) of
256 2014 International Conference on Contemporary Computing and Informatics (IC3I)
usability. There is a summary of Usability attributes of
various existing models in Table 1[26]. Each row in the
table lists areas of apparent agreement concerning
attributes of usability.
Table 1: Usability Attributes in Various models [26]
Model Usability Attributes
Abran et al. [2] Efficiency, Effectiveness, Satisfaction,
Learnability, Security
Alonso-Rios et al. Knowability, Operability, Efficiency,
[3] Robustness, Safety, Subjective Satisfaction
Bass et al. [14] Modifiability, Scalability, Reusability,
Performance, Security
Bevan et al. [16] Type of Product, Type of User, Ease of Use,
Acceptability
Boehm [4] Portability, Maintainability
Constantine & Efficiency in use, Learnability,
Lockwood [20] Rememberability, Reliability in use, User
Satisfaction
Dix et al. [17] Learnability, Flexibility, Robustness
Donyaee et al. [15] Efficiency, Effectiveness, Productivity,
Satisfaction, Learnability, Safety, Trustfulness,
Accessibility, Universality, Usefulness
Dubey et al. [18] Effectiveness, Efficiency, Satisfaction,
Learnability
FURPS [19] Aesthetics, Human Factors, Online and context
sensitive help, wizards and agents, User
Documentation, Consistency, Training Materials
IEEE Std. 1061 [7] Comprehensibility, Ease of Learning,
Communicativeness
ISO 9126-1 [9] Understandability, Learnability, Operability,
Attractiveness, Usability compliance
ISO 9241-11 [8] Effectiveness, Efficiency, Satisfaction
McCall [10] Operability, Training, Communicativeness
Nielsen [11] Learnability, Efficiency, Memorability, Errors,
Satisfaction
Preece et al. [12] Safety, Effectiveness, Efficiency, Enjoyableness
Preece et al. [13] Learnability, Efficiency, Throughput, Flexibility,
Attitude
Schneiderman et al. Time to learn, Speed of Performance, Rate of
[6] Errors by users, Retention over time, Subjective
Satisfaction.
Shackel [5] Effectiveness, Learnability, Flexibility,
Subjectively Pleasing
III. Proposed Model and Detailed Taxonomy
In the literature review, we can find large number of
international standards and usability models, which
describe usability but with different attributes in non-
homogeneous manner creating confusion among experts
for its usage and application. This inconsistent approach
among usability model is creating major challenge for
evaluation of usability of application. Researchers can’t
attain consensus for usability’s definition and have poor
information for deciding a set of usability factor. This
research theme requires a hierarchical based usability
model which should be consolidated to incorporate
consistency in usability. Hence, usability model would be
generic so that developers can measure usability without
any confusion.
The proposed section gives hierarchical based usability
model which can unify various existing models. After
reviewing of existing literature of usability, the 7 usability
attributes which are mentioned below, are included in our
proposed model:
a. Efficiency, it is that property of software product that need.
enables user to produce desired results with respect to Approachability It is a measure of software system which can
be defined by usage of large numbers of
investment of resources. users regardless of any physical and psychic
b. Effectiveness, it is a measure of software product with features (disabilities, age etc.).
which user can accomplish specified tasks and Utility it is a measure of software system which
desired results with completeness and certainty. helps the users when they don’t remember
about usage of the system
c. Satisfaction, it is a measure of responses, feelings of Faithfulness It is a measure of faithfulness of software
user when users are using the software i.e. freedom product that is delivered to its user.
from discomfort, likeability. Cultural universality It is a measure of software product with
d. Memorability, it is defined as that feature of software which user can use the system but from
that makes the user more understandable about the different viewpoint due to their cultural
backgrounds.
elements mainly because of clarity. Useful User Task output It is a measure of useful output that is
e. Security, it is defined as the property with which risks produced from interaction of user with
w.r.t. hardware and software’s can be neglected. software product.
f. Universality, it reflects the accommodation of Learnability It is a measure with which user can easily
learn the software system in minimum
different cultural backgrounds of diverse users with amount of time.
software product and practical utility of software Memorability of It is that property of software system that
product. Structure/ Elements/ helps the user to remember structure,
g. Productivity: it is defined as the amount of useful Functionality elements and functionality of software
system.
output with the software product. Comprehensibility It is a measure with which software system
The seven factors just described are not assumed to be has clarity to the user i.e. clarity of
independent, which is not the case in some existing elements/structure/functioning.
hierarchical models of usability measurement. Each Consistency in It is that property of software system that
attribute in proposed usability model is broken down into Structure/ Of Elements/ develops uniformity in system.
Of Functionality
measurable criteria (sub-attributes). Table 2 present the safety It is defined as capacity of software system
definitions of 23 sub-attributes. The main attributes so that risks can be avoided.
(factors), sub-attributes (criteria) and the characteristics Error Tolerance It is a measure with which software system
are related to each other in a hierarchical manner is can withstand in error occurring
environment.
presented in detailed taxonomy shown in Figs. 3. To 10.

Table 2: Measurable Usability sub-attributes

Sub-Attributes Definitions
Resource It is a measure of following resource related
attributes for successful completion of tasks
by user.
Time it reflects capability of software product in
term of time investment for activities
includes in performing actions by users,
response time by system, time spent on
errors, and memory Load:
User effort It reflects capability of software product for
producing desired results with respect to
physical and mental efforts that user invests.
Economic costs it involves following expenses required for
software
Likeability/ It is measure of software system to maintain
Attractiveness the attention of all kinds of user.
Convenience It is a measure of software product that
builds strong attitude of user towards its
design.
Aesthetics It is a measure of software system to attract
its user in sensorial terms (visual, olfactory).
Task accomplishment It is a measure of software product in which
user can perform his task with successful
accomplishment of his goals.
Operability It is a measure of software product which
helps user to perform required
functionalities in tasks with accuracy.
Extensibility It is a measure of adaptation of software
product with respect to changing needs of
user.
Reusability It is a measure with which software product
can be reused in another application.
Scalability It is the ability of software product to
continue to function well when it is changed
in size or volume in order to meet a user

2014 International Conference on Contemporary Computing and Informatics (IC3I) 257

 Efficiency
a. User safety
b. Third party
Safety
Effectiveness safety
c. Environment

Security
safety
Satisfaction
USABILITY

Error
Memorability Tolerance

Fig.6: Security Taxonomy

Security
Task a. Quality
accomplishment b. Quantity
Universality

a. Accuracy
Operability
Productivity b. Integrity

Effectiveness
Fig,3: Proposed Usability Attributes
a. Conformability
Extensibility
a. Simplicity b. Controllability
Learnability b. Learning
Time
Reusability
Memorability of
Structure/ Elements/
Functionality
Memorability

Scalability
Fig.7: Effectiveness Taxonomy a. Visual
b. Vocal
c. Motor
Approachability d. Auditory
Comprehensibility Clarity

Consistency in a. User guidance

Structure/ Of b. Interactivity of
Elements/ Of Utility assistance
c. Stability of
Universality

Functionality
documentation

Fig.4: Memorability Taxonomy

a. Utilization
b. Expansion
a. Stability
Resource c. Command Faithfulness b. Reputation
Utilization
c. Intention
d. Throughput
e. Consumption
of resources a. Language
Cultural b. Cultural
universality conventions
a. Response time
Time b. Time spend on errors
c. Memory Load Fig.8: Universality Taxonomy
d. Perform actions by
Efficiency

users Likeability /
Interest
Attractiveness/
Interest
a. System cost
b. Human Resource
Satisfaction

Economic Cost cost

c. Cost of
equipment a. Design
Convenience
d. Cost of b. Use of product
consumables
Documentation
Aesthetics
a. Physical Fig.9: Satisfaction Taxonomy
User Effort b. Mental
Fig.5: Efficiency Taxonomy

258 2014 International Conference on Contemporary Computing and Informatics (IC3I)

 clear, consistent and non-redundant manner. This paper defines

Useful User Task output

the concept of usability in terms of seven attributes, namely,
Efficiency, Effectiveness, Satisfaction, Memorability, Safety,

Productivity
Productivity and Universality and 23 sub-attributes and 42
characteristics as listed in Table 2. A detailed taxonomy of all
these seven attributes is presented in structured and non-
redundant way which will help in evaluation in most effective
and productive manner with ease. This paper also gives the
comparative study of all existing hierarchy based usability model
along with the proposed model. Our proposed model concludes
Fig.10: Productivity Taxonomy consistent definitions which may facilitate the integration of
IV. Comparison of the Proposed Model with usability and its measurement in the software development life
Existing Models cycle. Hence we can analyze the usability in better ways because
A comparison of various attributes described in the we have considered all the potential usability factors and their
existing usability models is given in (Sanjay Kumar sub attributes. If any additional factor for usability will be seen
Dubey et al.)[26]. Here we expanded the table in (Sanjay necessary in future, then a new usability model can be derived
Kumar Dubey et al.)[26] By adding some more existing from our proposed model. Therefore our proposed model can be
models and extra attributes .i.e. Universality and treated as a base model to support the further high level of
usability measurement.
Productivity which are missing in existing models.
Comparison of all these existing models with the proposed
hierarchy based usability model is presented in Table 3. References:
[1] Seffah, A., Donyaee, M., kline, R. B., & Padda, H. K. (2006).
The columns of the Table 3 shows the proposed model Usability measurement and metrics: A consolidated model. Software
and the existing models and rows of the table shows the 7- Quality Journal, 14, 159-178
major usability attributes (a-g) of the proposed model as [2] Abran, A., Khelifi, A., & Suryn, W. (2003). Usability meanings
and interpretations in ISO standards. Software Quality Journal, 11, 325-
discussed in section 3. In Table 4, ¥ indicates that the
338
corresponding attributes is considered by the respected [3] Alonso-Rios, D., Vazquer-Garsia, A., Mosqueria Rey, E. and
model and × indicated that the corresponding attribute is Moret-Bonillo, V., Usability: A Critical Analysis and a Taxonomy,
not considered by the respective model. International Journal of Human-Computer Interaction , 26(1), 53-74,
2010
[4] Boëhm, B., Characteristics of software quality, Vol1 of TRW
Table 4: Comparison of various models with the proposed model series on software technology, North-Holland, Amsterdam,
Netherlands, 1978.
Model Usability Attributes [5] Shackel, B., Usability – Context, framework, definition, design
Proposed Model a b c d e f g and evaluation. In Human Factors for Informatics Usability, ed. Shackel
Alonso-Rios et al. [3] ¥ ¥ ¥ ¥ ¥ × × B. and Richardson S. J., 21–37. NewYork, Cambridge
Abran et al. [2] ¥ ¥ ¥ ¥ ¥ × × University Press, 1991.
Bass et al. [14] ¥ ¥ × × ¥ × × [6] Shneiderman, B. and Plaisant, C., Designing the User Interface:
Boehm [4] × ¥ × × × × × Strategies for Effective Human- Computer Interaction, Addison
Dix et al. [17] × ¥ × ¥ × × × Wesley, Boston, MA, 2005.
Donyaee et al. [15] ¥ ¥ ¥ ¥ ¥ ¥ ¥ [7] Institute of Electrical and Electronics Engineers. IEEE standard
glossary of software engineering terminology, IEEE std. 610.12-1990.
Dubey et al. [18] ¥ ¥ ¥ ¥ × × ×
Los Alamitos, CA: Author, 1990
FURPS [19] ¥ ¥ ¥ ¥ × × ×
[8] International Organization for Standardization. ISO 9241-
IEEE Std. 1061 [7] × × ¥ ¥ × × × 11:1998, Ergonomic requirements for office work with visual display
ISO 9126-1 [9] × ¥ ¥ ¥ × × × terminals (VDTs), Part 11: Guidance on usability. Geneva,
ISO 9241-11 [8] ¥ ¥ ¥ × × × × Switzerland: Author, 1998
McCall [10] × ¥ ¥ ¥ × × × [9] ISO 9126: Information Technology-Software Product Evaluation-
Nielsen [11] ¥ ¥ ¥ ¥ × × × Quality Characteristics and Guidelines for their Use. Geneva, 1991.
Preece et al. [12] ¥ ¥ ¥ ¥ ¥ × × [10] McCall, J. A., Richards, P. K. and Walters, G. F., Factors in
Preece et al. [13] ¥ ¥ ¥ ¥ × × × software quality, Vols II, Rome Aid Defence Centre, Italy, 1977.
QUIM [1] ¥ ¥ ¥ ¥ ¥ ¥ × [11] Nielsen, J., Usability engineering. London: Academic Press,
Schneiderman et al. ¥ ¥ ¥ ¥ × × × 1993.
[6] [12] Preece, J., Benyon, D., Davies, G., Keller, L. and Rogers, Y., A
Shackel [5] × ¥ ¥ ¥ × × × guide to usability: Human factors in computing. Reading, MA:
S. K.Dubey et al [26] ¥ ¥ ¥ ¥ ¥ × × Addison- Wesley, 1993.
[13] Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S. and
Carey, T., Human-computer interaction. Reading, MA: Addison-
V. Conclusion Wesley, 1994.
Many Usability Models which have developed so far for the [14] Bass, L. and John, B. E., Linking usability to software
specification and measurement of usability of software, had architecture patterns through general scenarios. Journal of Systems
some major drawbacks as they lacked support of all usability and Software, 66 (3) 187-197, 2003.
[15] Donyaee, M. and Seffah, A., QUIM: An Integrated Model for
aspects, testing tool support and were not well integrated into Specifying and Measuring Quality in Use, Eighth IFIP
current software engineering practices. As we observe the Conference on Human Computer Interaction, Tokyo, Japan, 2001
existing model’s attributes and their classifications, we find their [16] Bevan, N., Kirakowsk,i J.& Maissel, J., What is usability?
description as divergent. There are ambiguity and informal terms Proceedings of the 4th International Conference on HCI, 651–655,
in the application of usability proposed by various existing 1991
models. From here, we got motivation for such research theme [17] Dix, Finley, J., Abowd, G. and Beale, R., Human-Computer
as it demands the need for a more consolidated usability model Interaction, 2nd ed. Prentice-Hall, 1998.
that will associate usability factors and its sub-attributes in a

2014 International Conference on Contemporary Computing and Informatics (IC3I) 259

[18] Dubey, S.K., Sharma, A. and Rana, A., Usability evaluation in
object oriented software systems using fuzzy logic approach,
International Journal of Computer Science Issues, 2011.
[19] Grady, R. B., Practical Software Metrics for Project
Management and Process Improvement, Prentice Hall, Englewood
Cliffs, NJ, USA 1992
[20] Constantine, L.L. and Lockwood, L.A.D. 1999. Software for
Use: A Practical Guide to the Models and Methods of Usage-
Centered Design, New York: Addison-Wesley
[21] Chandor, A., Graham, J., & Williamson, R. (1985). The
penguin dictionary of computers (3rd ed.). London: Penguin.
[22] Maguire, M., Context of use within usability activities.
International Journal of Human-Computer Studies, 2001.
[23] Holzinger, A. (2005, January). Usability engineering methods for
software developers. Communications of the ACM, 48(1), 71–74.
[24] Nielsen, J., & Loranger, H. (2006). Prioritizing web usability.
Berkeley, CA: New Riders Press.
[25] Quesenbery, W. (2001). What does usability mean: Looking
beyond “ease of use.” Proceedings of the 18th Annual Conference
Society for Technical Communications.
[26] Dubey, S.K., Gulati, A., Rana, A., Integrated Model for
Software Usability, International Journal on computer Science
andEngineering (IJCSE), Vol. 4 No. 03 March 2012.
[27] Maizatul Ismail, Mashkuri Yaacob, Sameem Abdulkareem, Fariza
Nasaruddin & Noorhidawati Abdullah, Support System for Novice
Researchers (SSNR): Usability Evaluation of the First Use , The
International Arab Journal of Information Technology (IAJIT),
Vol. 9, No. 04 July 2012.

260 2014 International Conference on Contemporary Computing and Informatics (IC3I)