IE 361 – HUMAN

FACTOR ENGINEERING

Lecturer Set 2 – Research Methods in HF

University of Engineering and Technology, Peshawar

DESIGN AND
EVALUATION
Human Factors Engineering
SKILLS

AFTER THIS LECTURE YOU WILL BE ABLE TO:

 Brief Review:: Areas of research in field of Human Factor
Engineering
 Understand the relationship between causal independent
variables
LEARNING OBJECTIVES
CASE – THOMAS EDISON

 Great Inventor and a Poor Businessman

 Phonograph vs Cylinder
 A Technology-Centre Device

 Case :: Disc of Un-Known Artist Vs Known ????

BEING FIRST, BEING BEST AND
EVEN BEING RIGHT DOES NOT
MATTERS
What matters is what your customers wants
HUMAN FACTORS DESIGN
ACTIVITIES
 Design product/system interfaces

 Modify designs/correct design flaws

 Safety activities (warnings, analyses)

 Develop training programs/materials

COST/BENEFIT ANALYSIS

COSTS BENEFITS
• HF personnel salaries • More effective product
• Extended time to product release • Safer product
• Analysis tools (software, video) • Cheaper to change design early in development
• Prototyping • Higher sales
• Increased development cost • Fewer liability suits

What other costs and benefits of HF activity in product/system

design can you think of?
 SYSTEM DEVELOPMENT AND HF ACTIVITIES
• Stage 1. Front-End Analysis*
• user analysis • Stage 4. Design of Support Materials
• function analysis • develop manuals
• preliminary task analysis • Stage 5. System Production
• environmental analysis • Stage 6. Implementation and Evaluation
• system specifications • experiments to evaluate system
• Stage 2. Conceptual Design* • Stage 7. System Operation/Maintenance
• function allocation • monitor system performance
•  (also known as task allocation) is a classic • Stage 8. System Disposal
human factors method for deciding whether
a particular function will be accomplished
by a person, technology (hardware or
software) or some mix of person and
technology
• support conceptual design
• Stage 3. Iterative Design and Testing*
• task analysis
• interface design & prototyping
• heuristic evaluation
• cost/benefit analysis of alternatives
• workload analysis (simulation & modeling)
• safety analysis
ROLE OF HUMAN FACTORS
 User-Centered Design: To Centre the design around the user
 Systems designed to fit people (not vice-versa).
 Reduces training time.
 Minimizes human error.
 Improves comfort, safety, and productivity.
USER-CENTERED DESIGN

1. Early focus on the user and tasks

2. Empirical measurements
 questionnaires, usability studies, data collection
3. Iterative design
 prototyping of interface
4. Participatory design
 user input to design, preferences
SOURCES FOR DESIGN DATA
 Data Compendiums
 Engineering Data Compendium: Human Perception and Performance
 Published 4 Vol Publication by Boff and Lincoln (1988)
 Human Factors Design Standards
 MIL-STD-1472D (MILITARY anthropometry, ergonomic specs)
 ANSI/HFES-100 (VDT specs)
 ANSI/HFES-200 (software specs)
 OSHA standards
 General HF Principles/Guidelines
 Textbooks (by topic: interface design, aviation, software)
 Research Journals (Human Factors, Ergonomics, Applied Ergonomics)
 Handbooks (Handbook of Human Factors)
 Conferences (CHI, HFES, IEA)
 Workshops (U of Mich)
HUMAN FACTORS RESEARCH JOURNALS
 Human Factors
 Proceedings of Human Factors and Ergonomics Society
Annual Conference
 Proceedings of Interservice/Industry Training Systems and Education
Conference
 Ergonomics
 Applied Ergonomics
 Human Factors and Aerospace Safety
 The International Journal of Aviation Psychology
 Human Performance
 Transportation Research
 IEEE
 And more
FRONT-END ANALYSIS

ANSWERS THE FOLLOWING QUESTIONS:

1. Who are the users?
 Consumers, trainers, maintainers, etc.
2. What functions are performed by system?
 Allocate functions to machine or person
3. What are the environmental conditions where system will be used?
 Noise, temperature, pressures, etc.
4. What are users’ preferences/requirements?
 Color, functions, shape, etc.
5. What are constraints to design of the system?
 Space, regulations, costs, resources
6. What are the HF criteria for design solutions?
 Shorter training time, less accidents, higher performance
FRONT-END ANALYSIS
USER ANALYSIS

 Who are the intended users?

 Age
 Experience level
 Intellectual level
 Anthropometrics (size & shape)
 Strength
 Culture & Language
 Disabilities (sight, hearing, mobility)

Why does the drive- thru ATM have Braille on its controls?
What are the characteristics of Cell Phone consumers?
FRONT-END ANALYSIS
ENVIRONMENT ANALYSIS
 Where will the tasks be completed and under what conditions?
 Indoors/outdoors
 Temperature/humidity
 Lighting levels
 Noise levels
 Vibration
 Gravity
 Special clothing/gear requirements

• How would the design of a radio transmitter differ if the intended

users were researchers in Antarctica?
• Would a standard vending machine work on the space station? If
not, how would you design a vending machine for that purpose?
FRONT-END ANALYSIS
FUNCTION/PRELIMINARY TASK ANALYSIS
 Function Analysis: What are the basic functions of the system?
 Example: The function of a wheel barrow is to move objects to the
work site.

 Preliminary Task Analysis: What are the jobs, duties, tasks, and
actions that user will be performing?
 Example: Tasks to be performed with a wheel barrow include
 loading it with dirt/building materials,
 transporting,
 mixing,
 dumping materials,
 cleaning,
 maintaining, and
 storing wheel barrow.
FRONT-END ANALYSIS
PRELIMINARY TASK ANALYSIS

Preliminary Task Analysis Methods:

• Interviews
• Focus Groups
• Observation
• Questionnaires

Used prior to design to learn how intended users perform their jobs.

What method(s) would you use if you were designing a new library
cataloguing system?
FRONT-END ANALYSIS
USER PREFERENCES & REQUIREMENT’S
 When different design options are available, must determine the
intended users preferences and needs (perhaps determined
through focus groups).

 User preference issues include:

 What components are automated
 Color features/aesthetics
 Balance between performance and preference
 What makes other products popular/unpopular
 What can be made adjustable and what cannot
 Feasibility of user’s demands
CONCEPTUAL DESIGN ACTIVITIES
FUNCTION ALLOCATION
Function Allocation: the assigning of tasks to humans or machine taking into
account human vs. machine capabilities and limitations

Fitt’s List (1951)

Things Humans Do Well Things Machines Do Well

Ability to: Ability to:

• Detect small amounts of visual or acoustic
energy
• Perceive patterns of light or sound
• Improvise and use flexible procedures
• Store very large amounts of information
for long periods and to recall relevant
facts at the appropriate time
• Reason inductively
• Exercise judgment
• Respond quickly to control signals, and to
apply great force smoothly and precisely
• Perform repetitive, routine tasks
• Store information briefly and then to erase
it completely
• Reason deductively, including
computational ability
• Handle highly complex operations, i.e., to
do many different things at once

Is Fitt’s list still appropriate 50+ years later?

CONCEPTUAL DESIGN ACTIVITIES
DECISION/ACTION & FUNCTIONAL FLOW ANALYSES
 Decision/Action and Functional Flow Analyses: Procedures for
identifying and depicting the sequence of functions, actions, and (See homework
decisions made by the system (user included as part of system). assignment)

 Example: Programming your TV BOX to record a show

Thursday night at 7:30 on NEWS Network

Access “use up & down Determine

Press input arrows to When show
“Program” menu indicate selection” airs

Hit arrow Press Box accepts Hit arrow Press Box accepts
to indicate “Enter” Input, moves to indicate “Enter” Input, moves
day button to next field time button to next field

“Program complete Yes No

Hit arrow Press Select another Press
Box accepts or
to indicate “Enter” program or press “End”
Input No
channel button End to finish”
Yes
CONCEPTUAL DESIGN ACTIVITIES
TYPICAL DESIGN MEETING

Brainstorming
Story-Boarding
ITERATIVE DESIGN AND TESTING
ITERATIVE DESIGN AND TESTING
TASK ANALYSIS (DETAILED)
 Once the Front End Analysis has been performed the designers
have an understanding of user’s needs.
 Purpose of Detailed Task Analysis is to identify:
 User goals and activities
 Tasks required to achieve goals
 Conditions under which task is performed
 Expected outcomes of tasks
 Equipment needed to perform tasks
 Complex decision making, problem solving, diagnoses*
 Complex rule structures dependent on situation*

* Part of Cognitive Task Analysis

TASK ANALYSIS METHODS
INTERVIEWS
 Interviews: Questioning subject matter experts to get them to explain what they
do on the job.

 Tries to answer questions such as:

 How do you perform the task?
 Why do you perform the task?
 Under what conditions do you perform the task?
 What happens before and after performing the task?
 What are the consequences of performing or not performing the task?

 Advantage: simple to use

 Disadvantage: SMEs may not be good at describing tasks
TASK ANALYSIS METHODS
OBSERVATION

 Observation: Watch and take notes of people performing the task

 Suggestions:
 Be unobtrusive
 Use video if possible and with approval
 Review observations with SMEs

 Advantages: See what workers do, not what they say they do.
 Disadvantages: Does not capture cognitive processes.
TASK ANALYSIS METHODS
THINK-ALOUD VERBAL PROTOCOL

 Think-Aloud Verbal Protocol: Worker state what they are thinking as

they perform the task

 Concurrent – verbalize as performing the task

 Retrospective – verbalize what they were doing as they view a
Think,
videotape of themselves think, think!

 Prospective – verbalize as they imagine performing a hypothetical

task
 Advantage: Can get info regarding thought processes and decisions
of experts performing the task.
 Disadvantage: Doing concurrently can interfere with the task, and
doing retrospectively relies too heavily on memory
TASK ANALYSIS METHODS
HIERARCHICAL TASK ANALYSIS

Hierarchical Task Analysis: Graphical (chart or diagram such as QFD) depiction

of task analysis

Example: Table 3.5 from Wickens, Gordon, & Liu

Hierarchical task analysis for using a lawnmower
Step 1. Examine Lawn
a. Make sure grass is dry
b. Look for any objects laying in the grass
Step 2. Inspect Lawnmower
a. Check components for tightness
1. Make sure grass bag handle is securely fastened to grass bag support
2. Make sure grass bag connector is securely fastened to bag adapter
3. Make sure the deck cover is in place
4. Check for any loose parts (such as oil cap) and blade

Advantage: Allows to conceptualize task sequences

Disadvantage: May not accurately represent true parallel nature of task
ITERATIVE DESIGN AND TESTING
LINK ANALYSIS

 Link Analysis: A data analysis technique which Shows the

relationships between components of a system (e.g.,
communication, control, movements)
 For determining:
 layout of related workstations
 placement of controls Link Analysis of CIC of USS Louisville
 communication patterns
 frequency of equipment use
 Advantage: Graphic depiction of relationships among system
features
 Disadvantage: Shows frequency of use, not importance, and
may vary by situation
ITERATIVE DESIGN AND TESTING
FLOW DIAGRAMS

 Operational Sequence Diagram: Graphically depicts

the sequence and timing of activity, and shows the
interactions between individuals and equipment

 Advantage: Connects actions to time and component

 Disadvantage: Does not lend itself easily to interface
design

Diagram for a collision avoidance system

ITERATIVE DESIGN AND TESTING
INTERFACE DESIGN PRINCIPLES

1. Provide a good conceptual model

2. Make things visible

3. Use natural mappings

4. Provide feedback
ITERATIVE DESIGN AND TESTING
INTERFACE DESIGN PRINCIPLES
 Provide a good conceptual model:
 When things are designed to be consistent with our mental models of
how they should work (expectancies) then
 it is easier to predict what will happen when we use them.

 In this metronome, the tempos are arranged around a circle with a knob
and arrow in the middle.
 Most people would expect to twist the knob until the arrow points to the
desired tempo.
 Actually the arrow is a button that you push if you want the tempo to
increase and the knob is not a knob at all.

A metronome is a device that produces an audible beat—a click or other sound—at regular, stable intervals that the user can
set in beats per minute. Musicians use the device to practice playing to a regular pulse.
ITERATIVE DESIGN AND TESTING
INTERFACE DESIGN PRINCIPLES
 Make things visible:
 It is easier to operate a system when the features are clearly
visible

 Starting this Patrol pump is more difficult than it sounds

 because the start button is hidden among the other sea of
information and does not “pop out” at you as a start button either
figuratively or literally.
 Additionally, the use of red usually signals something other than
start.
ITERATIVE DESIGN AND TESTING
INTERFACE DESIGN PRINCIPLES

 Use natural mappings:

 Actions should correspond to the system’s state.
 To make something go up, the action should be up, to make
it go right, the action should be to the right
 This STOVETOP makes good use of natural mappings. The
burners are arranged in the same pattern as the controls.
 Push and Pull DOOR sides
ITERATIVE DESIGN AND TESTING
INTERFACE DESIGN PRINCIPLES

 Provide feedback:
 Feedback is important because it allows one to see whether
the intended actions have taken place. (like hourglass icon on
computer software)

 In this example the feedback is confusing. The light on the

bottom signifies that the smaller quantity of coffee was
requested. The light does not go on when the larger quantity
is requested.

(read “Set Phasers to Stun” in Casey book for another example).

ITERATIVE DESIGN AND TESTING
INTERFACE DESIGN PRINCIPLES

 Other Design Principles (Norman, 1992):

 Simplify the structure of tasks
 Make alternative actions and knowledge of results visible
 Make it easy to determine what actions are possible
 Make it easy to evaluate current system state
 Exploit the power of constraints (remove possibility of error)
 Design for easy error recovery (undo)
 When all else fails, standardize.
ITERATIVE DESIGN AND TESTING
PROTOTYPING

 Prototypes or Mockups:
 Used to test features of the interface prior to
production
 - may be anything from crude cardboard cut-
outs to realistic models

Example of Rapid Prototyping

Software used to model different
phone interfaces
ITERATIVE DESIGN AND TESTING
EVALUATIVE STUDIES
 Types of evaluative studies:
 Cost/Benefit Analysis for Design Alternatives
 QFD identifies the relative importance of potential system features
based on how well they serve user’ goals
 These important features serves as input to COST/Benefit analysis
 Trade-Off Analyses
 To study the alternate design for best performance
 Safety Analysis
 Workload Analysis
 Simulation & Modeling
EVALUATIVE STUDIES
SIMULATION & MODELING

 Modeling programs:
 MicroSAINT
 Mannequin Cognitive Biomechanical
Modeling Modeling
 COMBIMAN
 HUMANCAD
 ANYBODY and more

Mathematical Modeling
EVALUATIVE STUDIES
USABILITY TESTING

 Usability Testing:
 Ease of use (user-friendly)

 May test for:

 Learnability
 Efficiency
 Memorability
 Errors
 Satisfaction
FINAL TEST AND EVALUATION
FIELD STUDIES
 Design: between vs. within subjects
 Test Participants: user population
 Proximal Measures:
 satisfaction
 usability
 performance
 errors
 Distal Measures:
 manufacturing costs
 personnel costs
 number of accidents/disability claims
ALTERNATIVES TO HF DESIGN AND
DEVELOPMENT

 Training
 To use system capabilities
 Methods to make task safer
 Modifying Current Equipment
 Equipment Replacement
 Ergonomic Aids
 Wrist pads
 Foot stools
 Anti-fatigue mats
 Job Rotation (and other admin approaches)
 Vary tasks
 Rest periods
COW FACTORS IN DESIGN
 This is what happens when you fail to consider the user in the design
CONCLUSION

 Some Techniques HF Engineer use

 to Understand user need and
 to Design
 Technology-Centered Design
 User-Centered Design
 Human Factor Criteria for Design
 Many of these depends upon
 Perceptual
 Cognitive and
 Control Characteristics