Usability Engineering Notes

L1

1. Usability is the effectiveness, efficiency and satisfaction with which a set of users can
achieve a set of tasks in a particular environment (ISO)
2. Usability is:
- Effective to use
- Efficient to use
- Safe to use
- Have good utility
- Easy to learn
- Easy to remember how to use (Rogers)
3. U.E refers to concepts and techniques for planning, achieving and verifying objectives for
system usability (Rosson & Carroll)
4. U.E by other names: Interaction Design, UX Design, HCI
5. Usability Engineers are people who focus on evaluating products, using usability methods
and principles.
6. Effective Interaction Design (ID) processes (User need analysis, participatory design,
usability testing) take account of the end user needs and characteristics and have been
shown to reduce cost by 30%
7. Usable products require less training, are less error prone, more satisfying to use, liked
more by users. -> higher sales, lower costs in after-sales service.

L2

1. Crucial Features of U.C. Design: (UPEII)
- User-centred (focused on users and tasks from start of project)
- Participative (involving users as members of the design team)
- Empirical (using formal user tests of usability)
- Iterative (designing, testing, and redesigning as a regular cycle until results satisfy
usability specs)
- Integrated (instruction manuals, online help, phone service)
2. Key interests in user-centred design: (PACT)
- Context (Physical Environment? Social Environment?)
- People (Who are the users? Preferences/Characteristics?)
- Activities (What are users doing? Goals?)
- Technology (What technology is used for tasks?)
3. User-centered Design Process (Refer to LN for details)
1. Identity needs and establish requirements
2. (Re)Design
3. Build interactive version
4. Evaluate
5. Final Product
4. Essential activities in U.C. Design (IDPE)
1. Identity needs and establish requirements
2. Develop alternatives
3. Prototyping
4. Evaluating
5. A requirement specifies what a product should do and how it should perform (What do
users want? What do users need?)
6. Activities for requirements
- Understanding about users, task, context
- Produce consistent requirements
7. Kinds of requirements
a. Users: Characteristics (Ability/Background) and Type (Novice/Expert/Casual)
b. Task Characterization (Users goals and activities? What tasks does the user
perform?)
c. Usability requirements (How long does it take to learn task? Time constraint on
task?)
d. Contextual requirements (Where does the user conduct the tasks/activities?
Physical, Social, Organizational, Technical Environment)
8. Ways to gather data:
a. Interviews
b. Focus groups (Group interviews)
c. Questionnaires
d. Researching similar products
e. Direct observation: in the field and online (ethnography)
f. Indirect observation: diaries, interaction logging (web analytics)
g. Studying documentation
9. Contextual Inquiry, a process of engaging in conversations with users while they carry out
their activities.
- Ethnographic study where user is expect, designer in apprentice
- A form of interview, 2-3 hours long and happens at users workplace
10. Main principles of contextual inquiry. (CPFI)
- Context (see workplace and what happens)
- Partnership (user and developer collaborate)
- Interpretation (observations are interpreted by user and dev. tgt)
- Focus (project focus used to understand what to look for)

L3

1. Design Principles: (VFA CMC)
1. Visibility a system should reveal its intention to user by making relevant parts
visible and make what has to be done obvious.
2. Feedback sending info back to user about what has been done, includes sound,
animation, click.
3. Affordance (give a clue) a relationship between actor and object. Certain attributes
of object allow people to know how to use it. E.g. Button invites pushing, chair invites
sitting.
o Does not make sense to talk about virtual interfaces in terms of real
affordances, but rather perceived affordances or signifiers. (Norman)
4. Constraints limit what can be done with an object/system.
o Interlocks: force ops to be done in correct order (microwave doors)
o Lock-ins: keep ops active, prevent premature stopping (save function)
o Lock-outs: prevent access (password)

5. Mappings relationship between controls, movements, and results
o Natural mappings take account physical analogies and standards which leads
to immediate understanding
6. Consistency design interfaces to have similar ops and use similar elements for
similar tasks, consistent interfaces are easier to learn and use, less error prone.

L4

1. Rich pictures are cartoon like depictions of users in system, relationships, and concerns
2. Rich pictures help to represent and understand problem situations
3. Personas (Who/Context/Motivation/Attitude)
4. Task-artifact problem. (Carroll)
- People engage in tasks and activities and make discoveries and encounter difficulties
-> define requirements for future tech (artifacts)
- New tech open up new possibilities for tasks (artifacts changes to tasks)
- New tasks and new tech becomes baseline and helps to evolve requirements for
future tech dev (Cycle)
- Scenarios carry possibilities and requirements as they co-evolve.
5. Scenarios are simple stories about user carrying out an activity.
6. Scenarios envision design in a flexible way. Ideas can be tried out and can be revised
quickly and easily.
7. Tasks reflect activities and goals, scenarios help to identify tasks

L5

1. We need to understand users cognitively (how and why people interact with products) to
design better interactive products.

2. Cognitive Processes: (APM)
a. Attention Information at the interface should be structured to capture users
attention. E.g. use of windows, colours, sounds, lights, avoiding cluttering
b. Perception and Recognition Design representations that are easy perceivable. E.g.
Text legible, icons easy to distinguish, sounds audible, bordering obvious, adhere to
Gestalt Principles.
c. Memory Simple procedures for tasks for low memory load, design interfaces that
promote recognition rather than recall (visual cues), various ways of encoding info to
help users remember e.g. categories, color, flagging.
3. Gestalt Principles of Visual Perception: (PECCCS)
a. Proximity Objects close that are perceived as a group
b. Similarity Objects share similar attributes (e.g. color, shape) are perceived as a
group
c. Enclosure Objects that appear to have a boundary around them are perceived as a
group
d. Closure Open structures are perceived as closed, complete, and regular when
theres a way to interpret it as such.
e. Continuity - Objects that are aligned tat or appear to be a continuation of one
another are perceived as a group
f. Connection Objects that are connected are perceived as a group

** Closure > Connection > Proximity/Similarity
4. External Cognition (Externalizing to reduce memory load)
- External representation such as reminders, calendars, notes, to-do list to remind us
what to do
- Computational offloading: when a tool is used with an external representation to carry
out a computation. E.g. calculator
- Annotations involve modifying existing representations through making marks. E.g.
crossing off, ticking.
- Cognitive tracing involves externally manipulating items into different orders or
structures. E.g. order of playing cards.
- Provide external representations at the interface that reduce memory load and
facilitate computational offloading. E.g. Info visualizations to allow people to make
sense and rapid decisions about masses of data.

5. Distributed cognition is concerned with the nature of cognitive phenomena across
individuals, artifacts, and internal and external representation. (Hutchins)
6. D.C. analysis involves examining:
- How people work tgt to solve a problem
- Verbal/non-verbal behavior, communication, coordination

L6

1. A prototype is a working model of an information system which emphasizes specific
aspects of that system (Vonk)
2. A prototype can be a series of screen sketches, powerpoint slide show, video simulating
use of the system, cardboard mock-up, etc.
3. Reasons for prototyping:
- Makes conceptual designs visible
- Evaluation and feedback
- Users/Stakeholders can communicate effectively about the design
4. What do we prototype?
- Technical issues
- Work flow, task design
- Screen layout and info display
- Difficult, critical areas
5. Horizontal prototypes display a wide range of features without fully implementing all of
those features. Appropriate for understanding relationships across a broad system and
showing the range of abilities of a system.
6. Vertical prototypes do not attempt to show all that will be in a system but instead focus
on implementing a small set of features in a near-complete fashion. Appropriate when a
certain complex feature of a system is poorly-understood and needs to be explored
7. When to prototype?
- Early prototyping, during requirement analysis
- Middle prototyping, during design stage/evaluating key aspects of design
- Late prototyping, during implementation/maintenance phases to investigate
operational issues.
8. Low-fi prototyping
- Uses a medium unlike final medium, paper, cardboard
- Quick, cheap, easily changed

a. Sketching, important to express, develop and comm design ideas
b. Storyboards, often used with scenarios, show how user progress through a task-
using device.
c. Card-based, each card represents one screen or part often used for web dev.
d. Paper-based, simulates interactivity with system
e. Wizard-of-Oz, users think they are interacting with computer to understand
users expectations
9. To get the design right, we explore alternative ideas by creating lots of ideas and
iterating between them.
10. How to generate alternative ideas:
- Brainstorming, collect a group of people to produce ideas tgt
- Seek inspiration, IDEO Techbox
11. How to choose among alternatives:
- Evaluate with users/peers
- Technical feasibility: some not possible
- Quality thresholds: usability goals lead to usability criteria set early on and check
regularly. Safety, utility, effectiveness, efficiency.

L7

1. Reasons for evaluation:
- Significant costs are wasted
- Users will reject your technology
- Major world events may be altered
- People lives put at risk
2. Evaluate the system as a whole:
- User/task/artefact/context
- Learnability, throughput, efficiency, satisfaction
- Surface features e.g. appearance, deeper features e.g. functionality, support systems
e.g. instructional manuals, contextual variables e.g. product compatibility, marketing
3. Where? In natural settings or lab settings. When? Throughout design, finished products
can be evaluated to collect information to inform new products.
4. Evaluation Objectives (ICEE)
- Establish requirements
- Establish effective use
- Identify interface improvements
- Comparing designs
5. Formative evaluation (usability testing):
- Done during design (prototyping)
E.g. Lab-based evaluation with 4 students
6. Summative evaluation (field study):
- Done after implementation
- E.g. 2-month field study with 28 smokers planning to quit
7. Evaluation Methods
a. User involvement (Data collected for usability evaluation purposes)
o Controlled settings (Usability test, experiments)
o Natural settings (Field studies)
o User reports
b. Non-user involvement
o Expert-based
Heuristics Evaluation
Reviews
Cognitive Walkthrough
o Theory-based
GOMS
KLM
8. Characteristics of Evaluation Methods (Refer to LN)
9. Expert-Based Evaluation (Inspection methods)
a. Reviews
o Present sketches so your critics can understand the key points, gather
feedback, reflect and re-evaluate ideas.
o Types of reviews from informal to formal:
Elevator pitch, for gathering quick reactions to ideas at any moment
Desktop review, for garnering feedback from the people around you
Meeting, event planned whenever you need feedback, you choose
who to attend
Crit (review), periodically scheduled sessions where you present
ideas formally and gather feedback from decision makers, senior
designers, managers, clients and peers
b. Heuristics (accepted rules of thumb concerning usability designs) Evaluation
o Reviewers conduct a usage simulation guided by some high level heuristics
Flow from screen to screen
Compare each screen against heuristics
10. Usability Heuristics (FUCAMER HVH)
1. Visibility of system status
2. Match between system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation

11. Empirical evidence suggests that on average 5 evaluators identify 75-80% of usability
problems.
12. U.H. advantages: fast and inexpensive, can be used early in dev. process, no users and labs
required,
13. U.H. disadvantages: requires expertise, does not involve end-users, tendency to
exaggerate number and severity of problems.
14. Theory-Based Evaluation (Theoretical understanding of HCI)
- Model Human processor (Card, Moran & Newell)
o Models info processes of a user interacting with a computer
o Predicts which cognitive processes are involved
o Enables calculations of time a user takes to do a task
o Leds to GOMS

a. GOMS (Knowledge the user must have to carry out a task)
o Goals, what the user wants to achieve (e.g. find location on map)
o Operators, the elementary acts the user performs (e.g. scan, manipulate)
o Methods, the procedures that combine the elemental acts (operators) (e.g.
scan with manipulate)
o Selection Rules, rules that allow users to choose between alternate methods
(e.g. if method 1, else method 2)


o GOMS is used to predict the time it takes an expert user to do a routine
task
o Only handles input side of interaction and does not address non-procedural
issues e.g. Screen Layout
o Needs specialist skills

b. Keystroke Level Model (KLM), based on GOMS
o Enable designers to calculate task performance times.
o Time to execute a task -> T(execute) = T(K) + T(P) + T(H)
o Used for time critical tasks (predicting time execution)

L8

1. Cognitive Walkthrough, alternative to heuristic evaluation, experts (non-users)
walkthrough a task and focus on ease of learning for new users and note down any
usability problems.
- Conducted by experts
- Based on tasks and scenarios
- Using early prototype of a design
- Evaluators role play a user working with system
- Evaluators ask specific questions at every step of task
- Purpose is to identify gaps
- Can be done in groups
- Conducted before user-based studies
2. Steps involved
1. Identify the characteristics of typical users.
2. Write scenarios and tasks
3. Develop prototype
4. Establish clear sequence of action for tasks
5. Review the walkthrough checklist questions (Will the user know what to do? Will the
user see how to do it? Will the user understand from the feedback whether the
action was correct?)
6. Select evaluators
7. Evaluate
o Experts conduct the analysis by walking through the action sequence in each
task and answering the questions
o Perform user actions required by tasks/scenarios
o Answer each checklist question for each step
o Capture all concerns that you note, including duplicates, observations that
fall outside the scenarios or checklist questions
8. Summary of Results
o Identify problems and its severity
o What changes could be made to address problems

3. Severity of Usability Problems
a. Severity rating helps to classify defects and provide a basis for fixing strategy
b. Established S.R. based on:
i. Scope, how many users experienced the U.P.?
ii. Frequency, how many tasks did the U.P occurred?
iii. Impact, how much of the task time was used to recover & how often the
problem prevent user from completing the task?

L9

1. Knowledge is sometimes described as a mental model:
- How to use the system (what to do next)
- What to do with unfamiliar systems or unexpected situations (how the system works)
- People make inferences using mental models of how to carry out tasks
2. Many people have erroneous mental models (Kempton) & Mental models are often poor,
incomplete, easily confusable, based on inappropriate analogies and superstition (Norman)
E.g. lots of people hit the traffic light button at least twice
3. The system image (conceptual model) materializes the designers mental model.
4. Gulfs of Evaluation and Execution
- Gulfs explicate the gaps that exist between the user and interface
- Gulf of execution is the distance from user to physical system (when user try to
figure out how to use it)
- Gulf of evaluation is the distance from physical system to user (where user try to
figure out what state it is in and whether their actions got them to their goal)
- Bridging gulfs reduce cognitive effort required to perform tasks
5. Normans approximate theory of action (Descriptive model of the stages a user passes
through when interacting with a computer)
- Goal -> Plan/Specify/Perform (Bridge of execution) -> World ->
Perceive/Interpret/Compare (Bridge of evaluation) -> Goal
- 3 levels of processing:
o Reflective conscious action to develop understanding, reason, make
decision, slow, often used in new situations
o Behavioral Learned skilled, aware of the actions but unaware of details, e.g.
driving car, speaking
o Visceral Subconscious, quick, affective (emotional) judgments of good or
bad, safe or dangerous
- How do we bridge the gulfs?
o Execution: Make the input match the user goals, plans and action specs
o Evaluation: Make the output present a good conceptual model that can be
readily perceived, interpreted and evaluated.
- Feedforward (Execution)
o What are the alternatives? (Plan)
o What can I do? (Specify)
o How do I do it? (Perform)
- Feedback (Evaluation)
o What Happened? (Perceive)
o What does it mean? (Interpret)
o Is this okay? (Compare)
6. Conceptual model (additional design principle): The design projects all the info needed to
create a good C.M. of the system leading to understanding and a feeling of control,
enhances both visibility and evaluation of results.
7. Theory of action not realistic:
- Human activity does not proceed in such an orderly and sequential manner
- More usual for stages to be missed repeated or out of order
8. Model useful for U.E.:
- Design systems that help users monitor their actions
- Evaluate systems and explain problems that users encounter

L10

1. Interface metaphors are designed to be similar to a physical entity but also has own
properties. E.g. desktop metaphor, search engine, icon of shopping cart
2. Exploit users familiar knowledge, helping them to understand the unfamiliar
3. Advantages of I.M.
- Makes learning new system easier
- Helps user understand the underlying conceptual model
- Can be innovative and enable applications to be made more accessible to a greater
diversity of users.
4. Disadvantages of I.M. (Nelson)
- Break conventional and cultural rules (recycle bin on desktop)
- Conflict with design principles
- Constraint designers in way they conceptualize a problem space
- Forces users to only understand the system in terms of the metaphor
- Limits designers imagination in coming up with new conceptual models
5. Interaction metaphors
- Conversational (Instructing and Conversing)
o Issuing commands using keyboard and function keys and selecting options via
menus. E.g. print a file, count words, search and replace
o Interacting with the system as if having a conversation E.g. car navigation
system, search engines, help systems
o Advs: allow users to interact in a way that is familiar. Disadvs:
Misunderstanding when system do not know how to parse what the user says.
- Direct Manipulation (Model-world metaphor)
o Exploits users knowledge of how they move and manipulate in the physical
world
o Virtual objects can be manipulated by moving, selecting, opening and closing
them.
o Continuous representation of objects and actions of interest
o Physical actions and button pressing instead of issuing commands with
complex syntax
o Rapid reversible actions with immediate feedback on object of interest
o DM systems have icons representing objects which can be moved and
manipulated with a mouse or cursor
o Advs: novices can learn functionality quickly, users can work fast in carrying
out wide range of tasks, users can retain op concepts over time, error
messages rarely needed, users can immediately see if actions further their
goals, user feels in control
o Disads: Not all tasks can be described by objects and not all actions can be
done directly, some tasks done better via delegating (e.g. spell checking),
moving mouse slower than pressing function keys
6. Collaborative Metaphor is a hybrid of the conversational and direct manipulation
metaphor


L11

1. Hi-fi prototype looks more like the final system, can be useful for selling ideas and
testing technical issues.
2. Problems of hi-fi prototype
- Too long to build
- Reviewers tend to comment on superficial aspects rather than content
- Developers are reluctant to change something they have crafted for hours
- Software prototype can set expectations too high
- Users think they have the full system
3. Lo-fi vs. Hi-fi (Refer to LN)
4. Types of hi-fi prototyping tools: Powerpoint, Balsamiq, Pidoco, Axure
5. Different kinds of prototyping are used for different purpose and at different stages
6. Hi-fi prototypes are good for summative evaluation and testing specific functions

L12/L13

1. Command-based interfaces (e.g. commands in terminal) are efficient, precise and fast but
has large overhead to learning set of commands
2. WIMP leading to GUIs (Xerox Star)
- Windows overcome physical constraints of a computer display, scrollable, more info to
be viewed
- Menus, drop-down, pop-up, expanding ones
- Icons are easier to learn and remember than commands, text labels can be used to
help identity small icon sets (Types: Similar/Analogical/Arbitrary)
3. Mobile and wearable
- Small screen, small number of keys, restricted number of controls. (multi-touch an
innovation)
- Usability and preference varies (depends on dexterity and commitment of user)
- Wearables provide user with means of interacting with digital info while on the move,
needs to be comfortable, easy to wear and usable (e.g. FitBit)
4. VR/MR
- Computer-generated graphical simulations providing the illusion of participation in a
synthetic environment
- Create highly engaging user experiences (Oculus)
- Higher fidelity, induces sense of presence, different viewpoints possible but
uncomfortable to wear
- Some design issues include designing safe/realistic VR to facilitate training, how to
control interactions and movements and level of realism to aim for.
- Augmented reality virtual representations are superimposed on physical objects
- Mixed reality views of real world combined with views of virtual environment
- Applications include medicine, games, flying, exploring
5. Robot & brain-computer interfaces
- Types of robots: remote, domestic, pet, sociable
- Design issues include whether robots should be more human-like or robot-like that
clearly serve a defined purpose
- BCI provides communication pathway between brain waves and external device, such
as a cursor on screen (e.g. Mindflex Game)
6. Tangible and shareable interfaces
- Tangible interfaces: type of sensor-based interaction where physical objects e.g.
bricks are coupled with digital representations. When a person manipulates the
physical object, it causes a digital effect to occur e.g. an animation (URP, digital
shadows, tabletop display and magnifying glass, inFORM renders virtual objects
physically)
- Benefits include can be manipulated in ways not possible using other interfaces (more
than 1 person can explore interface, different ways of exploring problem space) and
people are able to see and understand situations differently (greater
insight/learning, facilitate creativity)
- Design issues include the kind of coupling to use between the physical action and
digital effect and kind of physical artifact to use.

- Shareble interfaces: designed for more than 1 person to use, multiple and
simultaneous inputs such as interactive tabletops where groups interact with info
with their fingertips. (e.g. Smart Table, Surface)
- Design issues include whether size, orientation and shape of display have an effect on
collaboration
7. Speech, touch, gesture and NUI
- Speech: where user talks with system with a spoken language application, used for
inquiring about very specific info and used by people with disabilities (eg. siri, word
processing)
- Touch: Multi-touch devices everywhere.
- Air-based Gestures: Uses camera recognition, sensor and comp. vision techniques,
movements mapped to variety of gaming motions (eg. Kinect)
- Design issues include whether holding a controller is more intuitive than gestures
- NUI: draw on existing human capabilities for comm. and our manipulations of the
physical world, provides richer ways for interacting with digital world in ways which
feels natural.
- Research issues include standards for gesture, voice commands (Standard mappings
for Turning on tv? Changing channel? Turning vol. up?)

L14

1. Five key issues in user-based evaluation: (STRIP)
- Setting goals - decide how to analyze data once collected
- Identifying participants - decide who to gather data from
- Relationship with participants clear, professional, informed consent
- Triangulation look at data from more than one perspective
- Pilot studies small trial of main study
2. Lab Observations/Studies in controlled settings
- Usability testing, observe people using systems in simulated settings
- Record performance/data/observations/errors of typical users doing typical tasks.
- User satisfaction evaluated using questionnaires/interviews
- Lab Observation: Typically formative evaluations to discover usability problems and
make design changes/recommendations.
- Experimental: Summative evaluation, comparison of different system. (Testing
independent variable with relation to dependent variable eg. user performance)
Issues include lacking contextual influences, time, equipment and special skills
needed.
- Comparison between Experimental Evaluation and Usability Lab Evaluation (Refer to
LN)
3. User Reports
- Questionnaires checklists, multipoint rating scale, likert scale, semantic
differential, ranked order scale, fill-in questions, check-box questions
- Interviews comments on look and feel, things that you would change
4. Observational studies in natural settings
- Direct observation in the field
o Structuring frameworks to guide observation
Practitioners framework
The person. Who?
The place. Where?
That thing. What?
Goetz and LeCompte framework
Who is present?
What is their role?
What is happening?
When does the activity occur?
Where is it happening?
Why is it happening?
How is the activity organized?
o Field Studies (Done in natural settings)
Aim to understand what users do naturally and how technology
impacts them
To identify opportunities for new tech
Determine design requirements
Evaluate technology in use
Based on observation, interviews used with indirect observation
methods (eg. diaries, interaction logging)
Qualitative data analysis
Shows how product is integrated into users surroundings
o Ethnography
Set of techniques that include participant observation and
interviews
Ethnographers immerse themselves in the culture they study
Co-operation of people being observed is required
Data analysis is continuous
Questions get refined as understanding grows
Requires time to get immersed and to analyze data

5. Choosing techniques depends on:
- Focus of study
- Participants involved
- Nature of technique
- Resources available

L15

1. Testing and Evaluation Plan (COMMDUCT)
1. Objectives What do you want to find out? What is the data going to be used for?
o Check that the system is efficient to use? (task completion time)
o Can novel users complete their tasks with errors? (count errors)
o Determine whether functionality has good utility (questionnaire)
o Is the system easy to learn? (completion time, errors)
o How does it affect current practice? (interviews, observation data)
2. Criteria What are you going to use as a measure?
o Set usability goals against which the system is to be assessed
o E.g. Users to complete given task in 45 secs, users to complete task with less
than 5% error rate.
o Aim to phase goals in a way that is either pass or fail
3. Data What kind of data are you going to collect and how will it be analysed?
o Physical data eye movements, body language
o Time to complete tasks
o Counts/Rates number of errors, % of users completing tasks
o Comments/Opinions suggestions
4. Users/Participants (How are you going to sample them and get their consent?)
o Characterize the users to be used in the evaluation in terms of experience
with computers, frequency of use, specialized knowledge, special
characteristics such as age, language etc.
o How many participants? Typically 5-10, better to perform multiple usability
tests with fewer users each time.
5. Tasks (What is the user going to do during evaluation? Task/Scenarios constructed?)
o Describe in detail the tasks to be accomplished in users language
o Designed to simulate what the user would do with the software if he use it
everyday
o Select task that are prototypical, critical to completion of goal, common,
problematic or interesting
6. Materials (What equipment is required to conduct the evaluation?)
o System: Prototypes, user manuals
o Test instructions, task scenarios
o Data collection aids: questionnaires, note taking sheets, interview questions
o Recording equipment: stopwatch, video recording, eye-tracking
o Consent forms (Right to know goals of study, privacy of personal info, leave
when they wish)
7. Methods and Approach/Procedure (What techniques are you going to use?)
o Select the methods and techniques to be used in collecting and analyzing
data
o Assess the relative advs and disadvs of various methods, eye-tracking v
mobile set up
o Run pilot test (trials) to try out and improve your procedure before going
into the lab

a. Briefing Evaluator informs user about the test and includes, introduction to
aims of test, prototype, instructions, user consent.
b. Testing Actual conduct of the usability test, involves user completing set
of tasks, noting any difficulties, asking user questions and users encourage
to think aloud. (e.g. What they are trying to do, why they took an action and
how they interpret the system. Gives insight into what user is thinking but
unnatural and hard to talk if concentrating on a task)
c. Debrief Aims to obtain users final opinion, ask user more about his
experiences/difficulties (ie. Interview), and completing a questionnaire.
d. Analysis Task-based analysis attempts to determine how the users tackled
the tasks given, where are the difficulties and what can be done to fix it.
Performance-base analysis seeks to obtain clearly defined performance
measures from the data collected (ie. Freq. of task completion/errors, time
taken)
8. Costs
o Calculate overheads (Time/Funds needed) and iterate because you will not be
able to do all of what you want
o Reduce tasks, fewer users, cheaper methods/analysis, collect less data.

L16

1. Questionnaires Can be conducted before and/or after test, repeatable, cheap
administration. Can be use to create quantitative data, ie. averages, % success.
2. Quantitative data e.g. 3 out of 4 participants did., use means, standard deviations if
they are useful (with larger data sets), can be represented in histograms, pie charts etc.
3. Qualitative data
- Data observations of user interactions, user feedback/opinions from thinking
aloud/interviews
- Transcribe, spot recurring patterns, categorizing data and looking for critical
incidents
4. Video data
- Task based analysis, how users approached the tasks, where major difficulties lie and
what can be done.
- Issues based analysis, concerns and problems that span multiple tasks.
5. Tools to support data analysis
- Spreadsheet simple to use, basic graphs
- Statistical packages
- Qualitative data analysis tools
6. Eye-Tracking Analysis
- Helps us understand where users visual attention goes, to inform how to write
content to attract more attention, or to be more engaging.
- Gaze plots: The journey users eyes make to reach a target
- Time needed to find a target
- Fixation points/heatmap: Regions capturing users attention
- Quantitatively: Aggregate data across large sample for each task through a heatmap
to help us understand how much attention various screen regions attract.
- Qualitatively: Combine eye-tracking with thinking aloud data to identify usability
issues.
7. Presenting the findings
- Only make claims that your data can support
- Graphical representations may be appropriate for presentation
- Analysis from the various sources should be combined to generate a list of usability
problems
8. Steve Howards 8 Golden Rules of User-Based Evaluation
- Get far enough away to see the full picture
- But get close enough to collect useful data
- Dont convince the user the systems good let them tell you
- Make sure the user plays by the rules
- We change what we observe, so be unobstructive
- Listen to the data, it rarely lies
- But interpretations often do
- Deal with the big problems first

L17

1. User experience is about how people feel about using your product and their pleasure and
satisfaction when using it.
2. Cannot design a user experience, only design for a user experience.
3. UX Goals
- Desirable aspects
o Satisfying
o Helpful
o Enjoyable
o Challenging
o Rewarding
o Entertaining
- Undesirable aspects
o Boring
o Frustrating
o Gimmicky
o Childish
4. How do usability goals differ from UX Goals? A lot of overlap on quality of interaction
with a product, focus on satisfaction. Differences in scope and applicability
5. Trade-offs? (e.g. Can a product be both fun and safe?)
6. UX Theories
- Emotional design model (Norman)
o Visceral materials, design, look and feel
o Behavioral Immediate feedback
o Reflective Traditional usability aspects
- Emotional state changes how we think, when happy we are less focused and the body
relaxes so more likely to overlook minor problems and be more creative. (Norman)
- Designers can get away with more for products intended for leisure use.


- MrCarthy and Wrights framework describes the UX in terms of how it is felt by the
user
- Arousing gratification of senses, framework is made up of 4 core threads
o Sensual thread (Subconscious, does the experience make me feel
comfortable, relaxed, engaged?)
o Emotional thread (Does the experience make me feel happy/sad?)
o Compositional thread (Does parts of the experience come together well
enough to improve the experience?)
o Spatio-temporal thread (Does the experience make time seems faster?)

7. Prototyping UX
- Experience prototypes are useful:
o To understand existing user experiences and context
Demonstrate context and identify issues and design opportunities
Use prototypes to provide an experience with a system which cant
be experience directly because it is unsafe, unavailable etc.
E.g. Simulated experience Patients experience kit.
o Explore and evaluate design ideas
Facilitate the exploration of possible solutions
Direct the design team towards more informed development
E.g. Childrens picture communicator
o Communicate ideas to an audience
Let a client or user understand the subject value of a design idea by
directly experiencing it
Persuade the audience that an idea is compelling or that a chosen
design direction is incorrect
E.g. Kiss communicator
8. Evaluating UX
- How would you study the experience of listening to music in shuffle mode?
- Methods used for evaluating UX
o Use of methods you already know that involves users, observations,
interviews, diaries, questionnaires
o Focus on emotional, sensual aspects
o Often qualitatively (Diaries)
o Typically naturalistic to explore spatio-temporal aspects
L18

1. Laboratory experiment seeks to determine the causal relationships between selected
variables in a controlled environment (laboratory)
2. Properties of experiments
a. Research purpose Experiments test hypotheses to discover new knowledge by
investigating the relationship between two or more things, i.e. variables
b. Experimenter controls all environmental factors study relations by manipulating
independent variables and observe effect on one or more dependent variables
c. Quantitative measurement, controlled for bias, validated statistically and
replicable
d. Requires careful planning
3. Experimental designs - Select and assign subjects to groups
a. Different participants single group of participants allocated randomly to
experimental conditions (2 different conditions)
i. Adv: No order effects
ii. Disadv: Individual differences a problem
b. Same participants all participants appear in both conditions
i. Adv: No individual differences
ii. Disadv: Counter-balancing needed because of ordering effects
4. Confidence limits:
- The confidence that your conclusion is correct
- the hypothesis that the menu makes no difference is rejected at the .05 level
means:
o 95% chance your statement is correct
o 5% chance you are wrong

L19

1. Humans sought to extend themselves in many ways
a. Language (terms, concepts)
b. Artefacts (tools)
c. Practices (processes, methods)
d. Others (roles)
2. Social computing is the only technology that draws on all 4 of these characteristics
3. Cricket is an example of non-computer social systems
4. Artefacts to support comm., collaboration, cooperation: Telephone, email, SMS, online
forums, video conferencing
5. Social computing is concerned with the design and use of technologies that affect groups,
organizations, communities, and networks.
6. Many social computing systems dont succeed due to lack of users, benefits of
participation not clear to users and too many lurkers
7. How to encourage participation? Me-to-we design: five stages of social participation
(Simon) (e.g. Nike plus run tracker)
- Start with a focus on individual use and benefit
- Work from individual use towards social uses

Stage 1: Individual consumes content
Stage 2: Individual interacts with content
Stage 3: Individual interactions are networked in aggregate
Stage 4: Individual interactions are networked for social use
Stage 5: Individual engage with each other socially

8. Time and space matrix
- Same time (synchronous) & same place (collocated)
o F2F interactions: decision rooms, wall displays
- Different time & same place
o Continuous task: team rooms, large public display, project management
- Same time & different place (remote)
o Remote interactions: video conferencing, virtual worlds
- Diff. time & diff. place
o Communication + coordination: email, forums, version control, Wikipedia
9. Advantages of social computing
- More efficient at producing results (Wiki, reviews on amazon)
- May be capable of producing more accurate results
- May seem to be more legitimate (Hotel reviews)
- Uniquely Human (Classifying images)

L20

1. Persuasive technology interactive computing systems deliberately designed to change
peoples attitudes and behaviours (e.g. encourage exercise, discourage smoking)

2. Many techniques now used to change what people do or think warning messages,
reminders, personalized messages.
- Commonly referred as nudging where a small change in social context makes behavior
very diff without forcing anyone to do anything

3. Persuasive technology roles (Fogg)
a. Tools (e.g. FitBit)
o A tool increases capability
o A tool can be persuasive by:
Making target behaviours easier to do
Leading people through a process
Performing calculations that motivate
b. Medium (Medium that allows for user actions to interact with computers. E.g. a
virtual projection of yourself) e.g. WiiFit
o Medium provides an experience
o Medium can be persuasive by:
Allowing people to explore cause-and-effect relationships
Providing people with vicarious experiences that motivate
Helping people to rehearse a behavior
c. Social actors (e.g. Pocket Pikachu)
o Creates a relationship
o Can be persuasive by:
Rewarding people with positive feedback
Modeling a target behavior or attitude
Providing social support
4. We are heavily influenced by who communicates information
- Perceived authority, expert
- Peers
- Similarity and feelings we have for the messenger
5. Social and persuasive computing:
- We are strongly influenced by other people
o By their presence (social facilitation)
o What they say and do (competition, cooperation, social learning, norms)
- Social computing connects us with other people who influence our behavior

a. Social facilitation effect: People are more likely to perform a well-learned
behavior if they know they are being observed (by others) via computing
technology.
b. Competition, cooperation and recognition: Computing technology can motivate
users to adopt a target behavior by leveraging humans natural drive to compete
or cooperate. And by offering public recognition, computing tech can increase the
likelihood that a person will adopt a target behavior.
c. Social learning: A person will be more motivated to perform a target behavior id
he can use comp. tech. to observe others performing the behavior and being
rewarded for it.
d. Norms: We tend to do what those around us are already doing, computing tech.
can highlight norms to increase the likelihood that a person will adopt a behavior.