Understanding Your Users: A Practical Guide to User Research Methods

Caine

Part 1

What You Need to Know Before Choosing an Activity

Chapter 1

Introduction to User Experience

WHAT IS USER EXPERIENCE?   4 - 7

Who Does User Experience?

USER-CENTERED DESIGN   7 - 11

Principles of User-Centered Design

Incorporating UCD Principles into the Product Development Life Cycle

Design Thinking

A VARIETY OF REQUIREMENTS   12 - 16

The Product Team’s Perspective

User Requirements

GETTING STAKEHOLDER BUY-IN FOR YOUR ACTIVITY   16 - 19

Arguments and Counterarguments

Preventing Resistance

WHAT IS NEXT?   20

What Is User Experience?

If you are reading this book, you already have some idea about, or at least interest in, User Experience (UX). However, you likely arrived at this book and this field via a different path from the colleague sitting next to you or the classmate across the country who is taking the same online human-computer interaction (HCI) or human factors class as you. User experience practitioners and students come to UX from a diverse range of backgrounds, including computer science, psychology, design marketing, business, anthropology, and industrial engineering (Farrell & Nielsen, 2014). This diversity is an advantage; our community adopts the best practices and benefits from the knowledge of all of these disciplines. However, it also means that there is not one singular activity, style, or approach that defines UX.

There are many definitions of UX (see http://www.allaboutux.org/ux-definitions). The User Experience Professionals Association (UXPA) defines it as Every aspect of the user’s interaction with a product, service, or company that make up the user’s perceptions of the whole. User experience design as a discipline is concerned with all the elements that together make up that interface, including layout, visual design, text, brand, sound, and interaction. Usability Engineering works to coordinate these elements to allow for the best possible interaction by users.

Tip

If you are talking to people who are not familiar with UX and need an easy way to help them understand what you do, tell them you help make technology easy for people to use. It is not a perfect definition by any means, but people get the gist. Kelly discovered this after she got tired of blank stares when telling people her various job titles. She conducted a little user research on her friends, family members, and airplane seatmates to discover a one-line description for what she does. I help make technology easy for people to use always worked and usually made people say, Wow, that’s great! We need more people like you.

Whereas usability is about creating problem-free interactions, user experience is much broader and holistic. Usability is objective and product-based (i.e., a product is usable), whereas user experience is subjective and human centered (i.e., a person and a product co-create the user experience). Very often, user experience research seeks to gather user requirements for the design of technologies (e.g., mobile devices, websites, wearable technologies, software) or evaluate the usability of an existing technology.

User requirements refer to the features/attributes a product should have or how it should perform from the users’ perspective. User-centered design (UCD) is an approach for collecting and analyzing these requirements. This chapter introduces the basic concepts behind UCD, introduces stakeholders and their requirements, and tells you how to get buy-in for your user research activities.

Who Does User Experience?

Lots of people from many different backgrounds do work in user experience (see Figure 1.1). In industry, there are a variety of titles for UX practitioners, including (Farrell & Nielsen, 2014):

Figure 1.1 The disciplines of User Experience ( www.envis-precisely.com ). This work is licensed under the Creative Commons Attribution-Share Alike 30 Unported License. To view a copy of this license, visit http://Creativecommons.org/licenses/ by-sa/30 or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. From http://upload.wikimedia.org/wikipedia/commons/d/d5/Interaction-Design-Disciplines.png.

■ User experience designer

■ User experience researcher

■ Information architect

■ Interaction designer

■ Human factors engineer

■ Business analyst

■ Consultant

■ Creative director

■ Interaction architect

■ Usability specialist

At the executive level, the titles include (Manning & Bodine, 2012):

■ Chief customer officer

■ Chief client officer

■ Chief experience officer

■ VP of user experience

Fundamentally, UX research is about understanding people, the domain, and technology. In that sense, while we have written this book from the perspective of UX research, the methods we describe can be used in any situation where you want to understand more about human behavior, perceptions, ideas, needs, wants, and concerns and how those play out in various contexts and with various technologies.

At a Glance

> User-centered design

> A variety of experiences

> Getting stakeholder buy-in for your activity

Suggested Resources for Further Reading

There are many colleges and universities with master’s and PhD programs in human-centered computing, HCI, engineering psychology, information sciences, etc., that offer coursework that will prepare you for a career in User Experience. If you do not have a degree in these or a related field, the books below can offer an introduction to many of the concepts discussed in this book.

• Norman, D. A. (2013). The design of everyday things: Revised and expanded edition. Basic Books.

• Lidwell, W., Holden, K., & Butler, J. (2010). Universal principles of design, revised and updated: 125 ways to enhance usability, influence perception, increase appeal, make better design decisions, and teach through design. Rockport Pub.

• Rogers, Y. (2012). HCI theory: Classical, modern, and contemporary. Synthesis Lectures on Human-Centered Informatics, 5(2), 1–129.

• Johnson, J. (2014). Designing with the mind in mind: Simple guide to understanding user interface design guidelines (2nd ed.). Morgan Kaufmann.

• Weinschenk, S. (2011). 100 things every designer needs to know about people. Pearson Education.

User-Centered Design

UCD is a product development approach that focuses on end users. The philosophy is that the product should suit the user, rather than making the user suit the product. This is accomplished by employing techniques, processes, and methods throughout the product life cycle that focus on the user.

A Note About Terminology

Some of our colleagues (and even some of us!) do not like the word user. It has negative associations (e.g., drug users), can create subjective distance, and certainly does not convey the complexity and depth of people and their experiences. Don Norman (2006) wrote, Words matter. Talk about people: not customers, not consumers, not users. We agree. However, UX is the term of the trade, so we use it and its components (user and experience) throughout the book.

Principles of User-Centered Design

There are three key principles of UCD (Gould & Lewis, 1985): an early focus on users and tasks, empirical measurement of product usage, and iterative design.

An Early Focus on Users and Tasks

The first principle focuses on the systematic and structured collection of users’ experiences. That is the focus of this book. We will teach you how to effectively collect users’ experiences using a variety of methods.

To maximize the quality of the user experience of a product, the user should be involved from the product’s inception. The earlier the user is involved, the less repair work needs to be done at the final stages of the life cycle (e.g., after a usability test). The UCD process should begin with user experience gathering. By collecting user experiences, you can gain an understanding of what your users really want and need, how they currently work or how they would like to work, and their mental representations of their domain. This information is invaluable when creating a superior product.

Empirical Measurement of Product Usage

The focus here is on classical usability: ease of learning and effective, error-free use. This can be assessed early in the life cycle via usability testing of prototypes. Metrics such as errors, assists, and task completion rates gauge this. In a usability test, users are given a prototype or the final product and asked to complete a series of typical tasks using the product. This activity allows you to identify usability issues with your product. Then, changes are made to improve the product before its release. We describe usability evaluation in Chapter 14.

Image based on cartoon #5 at http://www.usability.uk.com/

Iterative Design

The final principle recommends that experiences are collected and the product is designed, modified, and tested repeatedly. The idea of iterative design is to fail early; it is much easier to change the user interface of an early prototype than a deployed system. This could mean that you and your team start with paper prototypes and iterate at that stage multiple times and only then move on to an interactive prototype. You do not go through the development cycle once; you continue to iterate and fine-tune with each cycle until you get it right. No one gets all the information the first time, no matter how expertly you execute each user research activity.

Incorporating UCD Principles into the Product Development Life Cycle

This book offers research techniques for every stop in the product development life cycle, but it is unlikely you will have the time, resources, or even need to do every one of them. To be successful, it is your job to identify the critical research questions facing your team, company, or academic lab and then to identify the method(s) necessary to answer those questions. Stone (2013) wrote, To me, great UX research is four things—in this order—timely, believable, actionable, and surprising. Timely because we need to learn to work at the same pace as product teams, otherwise our direct impact on the product will suffer. Believable comes from thinking hard about the context of use and structuring your tasks to capture this context. Actionable comes from knowing the decisions the product team is facing, and being on the same page with them…. Surprising comes from understanding how to observe and report on user behavior better than anyone else on your team. This is the only trick I know. Do excellent work and do it fast, and people will notice and thank you for it.

Figure 1.2 illustrates the ideal product life cycle with these UCD processes incorporated. The ‘Concept’ phase (Stage 1) encompasses an early focus on users. The ‘Design’ phase (Stage 2) incorporates an early focus on users and empirical measurement. The ‘Develop’ and ‘Release’ phases (Stages 3 and 4) tend to focus on empirical measurement. Sample activities in each phase are discussed in this section.

Figure 1.2 Product lifecycle with UCD processes incorporated.

Stage 1: Concept

This is the idea phase of your product. You are:

■ Developing user experience goals and objectives

■ Creating user profiles and personas

■ Executing user experience research activities, such as interviews and field studies

Stage 2: Design

At this stage, you begin using the information collected in Stage 1 to create iterative designs. Some user research activities include:

■ User walkthroughs of low-fidelity prototypes (e.g., paper)

■ Heuristic evaluations

■ Execution of user experience research activities, such as focus groups and card sorts

Stage 3: Develop

The developers or engineers now begin to create the product. Some usability activities include:

■ Preparation, planning, and execution of pre-product release heuristic evaluations

■ Preparation, planning, and execution of pre-product release usability testing

Stage 4: Release

The last stage is when your product is released to the public or customer or within your organization. This stage often blends both user experience research activities with other types of empirical measurements. In software environments, formal usability tests are typically executed on the live code. In addition, user experience research collection for the next product release often begins at Stage 4, to gauge users’ feedback on the product that has been released in the real world. Some Stage 4 activities include:

■ Usability testing

■ Surveys or interviews to gain feedback on released code

■ Field studies to see the product being used in its environment

The third principle of UCD—iterative design—is employed throughout the entire cycle, as well as within each stage of the process. For example, you may do a field visit in the concept phase. This activity will begin your user experience research data collection, but may also open up new questions prompting you to run a follow-up activity such as individual interviews. You will then use the results of the interviews to go back and revise and refine or iterate your user experience research document based on your new data.

Design Thinking

If your colleagues have not adopted a UCD process, you have a larger issue on your hands. Conducting a few user research activities will not lead to a cure. One option is to consider design thinking.

If your company, client, or academic advisor does not understand the value of user research, design thinking can be a great way to get them to see the necessity. Design thinking is an approach to innovation that can be applied to all areas of business and practice. It does not refer to a formal step-by-step process, but to a framework and a mind-set. It is focused on a bias toward action, a human-centered viewpoint, and a mode of continual experimentation. The core idea is that by deeply understanding user needs, opportunities for innovation will emerge. These ideas can be further refined through rapid prototypes and iterations with users to result in breakthrough outcomes. The process of collecting user requirements is an integral part of this approach. The design thinking approach provides greater context so people understand why understanding users is so critical to creating great products and services. The Hasso Plattner Institute of Design (Stanford d.school) has done a good job of popularizing this approach in the d.school classes and its executive boot camps. You will now find similar workshops offered by other academic institutions and consultants. With a day or two of training, you can get a team to understand the criticality of user empathy.

Suggested Resources for Additional Reading

If you are interested in building a design thinking culture, check out:

• Hasso Plattner Institute of Design: http://dschool.stanford.edu/.

• Building a Culture of Design Thinking at Citrix: http://www.mixprize.org/story/reweaving-corporate-dna-building-culture-design-thinking-citrix.

You may also decide to employ a change management strategy in order to affect organization structure, processes, and culture. This is no small task. These books provide detailed guidance:

• Bias, R. G., & Mayhew, D. J. (Eds.). (2005). Cost-justifying usability. San Francisco: Morgan Kaufmann.

• Schaffer, E. (2004). Institutionalization of usability: A step-by-step guide. New York: Addison-Wesley.

• Sharon, T. (2012). It’s our research: Getting stakeholder buy-in for user experience research projects. Morgan Kaufmann.

A Variety of Requirements

Thanks to a growing awareness of user experience, many product teams now realize the importance of understanding their users and the consequences that result when users are unable to utilize products with maximum ease and pleasure. As a result of this awareness, many companies and academic labs have incorporated some of the UCD process into their product or scientific life cycles. For many though, user experience still begins and ends with the usability test.

There is a clear difference between usability testing and user experience research. Usability testing determines whether a given solution is usable—easy to use in an error-free manner. User experience research provides insight into the many possible solutions and allows a team to select and investigate the best solution from the users’ perspective. The difference between a good designer and the outstanding designer is the latter’s vision of solutions. Without user research, your vision is seriously limited.

Although usability testing is a critical part of an effective UCD life cycle, it is only one component of the UCD. This book is focused primarily on the user experience research stage, which often receives less attention than usability testing, but is equally important. User experience research can be used to gather user requirements for the design of technologies. By user requirements, we mean the features/attributes the product should have or how it should perform. Requirements can come from a variety of sources—marketing, product development, end users, purchasing decision-makers, calls for proposals, etc. All sources have valid requirements and they must be taken into consideration by the team. For example, if you are building a mobile app for booking travel, some user requirements might include the following:

■ The mobile app must be available on iOS, Android, and Windows phones.

■ Users must register with the site before making purchases.

■ The site must be available in English, Spanish, and French.

■ The site should appeal to all demographics of users.

■ Users should not require training.

We next describe the different types of requirements you may encounter, with a focus on industry settings. The advice here can be applied to other settings such as nonprofits or academia by considering the perspectives represented on your team (e.g., you may not have sales requirements, but you still have stakeholders). In all cases, by understanding a product’s competing requirements, you can better position the user requirements for inclusion.

The Product Team’s Perspective

In industry settings, the product team is composed of everyone who has a stake in building, deploying, and selling the product. The requirements-gathering phase is the period when the product team must do its initial research to determine the direction of the product. They must collect requirements from a variety of sources (e.g., sales, marketing, managers in your company, customers, end users) and use this information to determine what functionality will be included in the product. This stage is critical in creating a basis for the design. Poor requirements collection will impact the remaining stages of the product life cycle depicted in Figure 1.2. You will end up with a misguided product that will not sell or will be unusable and useless to the users and/or the company that purchases it.

There are a variety of different requirements that factor into product development, and there is often confusion between them. Figure 1.3 illustrates some of the many requirements and sources that a product team must deal with.

Figure 1.3 Requirements sources (image based on Weigers, 1999).

We often encounter teams who say, We have already collected our user requirements, but in reality, they have collected functional or marketing requirements, not actual user requirements. Below, we discuss business, marketing, and sales requirements because they are often confused with user requirements. It is important to note that each of these is important, but not a user requirement. There may be overlap, but it is critical for all of the different sources of requirements to be independently collected and then prioritized as a group. You cannot assume that what the salesperson wants to see in the product is the same as what the end user wants to see in the product. To collect the different requirements effectively, you must be able to distinguish among them.

DILBERT © 2003 Scott Adams. Used By permission of UNIVERSAL UCLICK. All rights reserved.

Business Requirements

The people who are considering purchasing your product have requirements for that product. These people are typically corporate professionals or executives. They are often referred to as the decision-makers. Their requirements often reflect the current business practices of their company or new practices they want to adopt to employ cost savings. They want to make sure the product matches their requirements. If you want to keep these customers, being aware of their business requirements is very important. Sometimes these requirements overlap with the users’ requirements, but often, business requirements are higher-level and/or more technical. In academia, the decision-maker(s) may be your advisor or thesis committee.

Marketing and Sales Requirements

The marketing and sales departments want to ensure the product sells and their requirements reflect this goal. They may have requests for features or functions that they think customers want, that competitors have or do not have, etc. Marketing requirements tend to be at a high level that lacks detail. Marketers are not interested in sending a message about the minute details of the product; they want to send high-level messages to potential customers that will lure them to the product. For example, for a travel app, they may have a requirement that the app should have more airline choices than all other competitors or that it will find the lowest guaranteed airfare.

The sales representatives are in the field with customers day-in and day-out, so the requirements they have are frequently based on what they are hearing from their customers during product demos. Keep in mind that they are usually demonstrating the product to purchasing decision-makers and not end users. They may have requirements such as It needs to be fast or It needs to look like the number one travel app in the marketplace. It is important to remember that these requirements may be very customer-specific and not applicable or scalable to other current (or future) customers.

Sales and marketing departments do not typically collect detailed information regarding what users must be able to do with the product, how they must be able to use it, and under what circumstances they must be able to use it; however, some marketing and sales requirements do represent a form of end user requirement. Consequently, you will likely see some overlap between the user requirements and what sales and marketing have uncovered. The reality is that if the product does not do what users want, it does not matter how usable it is. Marketing and sales folks often talk to end users, and sometimes, they even talk to them about usability, even if only at a high level. Mostly, they talk to users about features and capabilities. This is valuable information. Its weakness is that the information they collect is often incomplete, and they almost always collect it in demo mode (i.e., selling rather than listening). They may try to understand users, but because it is not their primary goal, they do not have time or motivation to gather true user requirements. In addition, they may encourage new features to be included in the product because the latest technology is easier to sell, not because users actually want it or will end up using it.

User Requirements

Whether you are working in academia, in private industry, or at a nonprofit, you will have stakeholders with their own goals to meet. Those stakeholders have to keep in mind the needs of a government agency that is funding the grant for your research (e.g., NIH), or private donors, or shareholders in your company. It is everyone’s job to balance user needs against business, reporting, or sales needs. But before a product or service is complete, you want to make sure that end users can actually use it and that it contains the features they need. If you ignore this important goal, increased training and support or decreased user productivity will lead to unsatisfied users. That can harm future development efforts, your scientific success, and future sales or decrease renewed licenses or product upgrades. It can also lead to a poor reputation, unhappy funders, or no new customers.

As was discussed above, you may think you understand what the end users want and need because other sources have told you on their behalf. This is the number one mistake that product teams make. In reality, purchasing decision-makers, sales, and marketing may think that users interact with the product in a certain way, but because they (decision-makers, sales, and marketing) are not the true end users, they are frequently mistaken. In other cases, they receive information from one single source that has spoken to the end user, but much gets lost in the translation and interpretation by the time the information gets to you. Figure 1.4 illustrates many of these problematic communication paths from which the product team receives user information.

Figure 1.4 Communication paths from the user to the product team (image based on Weigers, 1999).

As a result, you must talk to the actual users—the people who will use the product at the end of the day—to gain an understanding of their perspective and needs. This understanding includes their tasks, goals, context of use, and skills.

By addressing your users’ requirements, you can create a product that fulfills their needs. This fulfillment will in turn:

■ Increase sales and market share due to increased customer satisfaction and increased ease of use

■ Decrease training and support calls that result from user interfaces that do not match the users’ needs

■ Decrease development time and cost by creating products that contain only the necessary functionality

Getting Stakeholder Buy-In for Your Activity

If you are lucky, the stakeholders you are working with already know the value of user research. However, we have found that many times, the evidence about how user experience research drives innovation, acceptance, productivity, and profit has not made its way to all stakeholders. Therefore, one key skill you need as a user experience professional is to be able to effectively convince people of the importance of user research.

UX brings a huge amount of value to a project beyond just financial benefits. However, when you need to convince stakeholders to buy in to your activity, money often talks. Good UX leads to increased productivity, increased user satisfaction, decreased errors, decreased training costs, decreased support costs, increased sales, savings on redesign costs, increased audience/traffic, and better online reviews (Bias & Mayhew, 2005). Here are some specific pieces of evidence for you to use to help demonstrate the value of UX:

■ Understanding UX is critical to innovation. Innovation is not invention. Innovation may involve invention, but it requires many other things as well—including a deep understanding of whether customers need or desire that invention (Keeley, Walters, Pikkel, & Quinn, 2013).

■ The quality of a firm’s customer experience strongly relates to loyalty measures such as willingness to consider the company for another purchase, likelihood to switch business, and likelihood to recommend to friends and family. These factors are strongly related to yearly revenue (Burns, Manning, & Petersen, 2012).

■ Improving customer experience, even in small ways, can translate into billions of dollars of incremental revenue per year…. No industry is totally immune from the revenue impact of customer experience (Manning & Bodine, 2012).

■ Integrating UX early can save a huge amount of redesign effort in the long run. Sun demonstrated that $20,000 of upfront UX research could have saved $152 million (Rhodes, 2000).

Arguments and Counterarguments

Along the way, you may encounter arguments for avoiding user experience research. In Table 1.1, we present the most common arguments we have encountered and suggestions for how to handle them. On the left side of the table, there are statements you can quote directly, and on the right side are explanations of the rationale behind each quote.

Table 1.1

Arguments Against Doing UX Research and How to Counter Them