June 23, 2010

Best Practices For Launching Site

Redesigns
by Harley Manning
for Customer Experience Professionals

Making Leaders Successful Every Day

For Customer Experience Professionals

June 23, 2010

Best Practices For Launching Site Redesigns
by Harley Manning
with Richard Gans and Shelby Catino

Exec ut i v e S u mma ry
When companies redesign sites, users’ aversion to change results in problems like spikes in call
center traffic and social media backlash. That’s because changes in site design force users to learn new
interfaces and new process flows in order to accomplish their goals. Customer experience professionals
can minimize the pain of launching a redesign by following best practices like engaging users during the
design process and educating executives on what to expect.

table o f Co nte nts NOT E S & RE S OU RCE S

2 Even The Perfect Redesign Can Turn Into The Forrester interviewed customer experience
Perfect Nightmare professionals at companies that had recently
The Science Behind Why People Dislike launched redesigns of all or part of their sites.
Change
2 Best Practices For Minimizing Problems
Related Research Documents
Caused By Redesigns “Web Site Reviews: What, Why, And When”
April 9, 2009
recommendations
7 Start By Rolling Out A Truly High-Quality Site “Low-Cost User Research And Usability Testing
Techniques”
February 3, 2009
“Scenario Design: A Disciplined Approach To
Customer Experience”
July 19, 2004

© 2010, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available
resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar,
and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To
purchase reprints of this document, please email clientsupport@forrester.com. For additional information, go to www.forrester.com.
2 Best Practices For Launching Site Redesigns
For Customer Experience Professionals

even the perfect redesign can turn into the perfect nightmare
There’s no way around it: Changing the design of a site is risky. Site owners we’ve interviewed
report a list of problems that include huge spikes in call center volume as people call to get help or
complain (or both), deep declines in customer satisfaction ratings, and social media backlash. For
example, Yahoo!’s customer satisfaction rating dropped 5% in 2006 from the previous year because
its redesigned site didn’t resonate with users.1 And when Facebook launched a redesign in March
2009, 1.7 million of its users joined a user group called “Petition Against the New Facebook” within
the next two weeks.2

The Science Behind Why People Dislike Change

Firms may have the best intentions when making changes to their sites, but users don’t always see it
that way. Why is change so hard for users to deal with? Changes that result from site redesigns can
degrade the online customer experience because they:

· Place a cognitive burden on users. Redesigns require users to learn low-level, page-specific
details like what a particular button or link will do. Users must also learn higher-level task
concepts — such as what they have to do to open an account — in order to complete their
goals.3

· Make two key usability metrics decline. Usability experts judge sites based on a handful of
measurable human factors. These include the time it takes to learn how to use the site and
the time it takes to complete a task. Both of these metrics move in the wrong direction for
experienced users encountering a redesign for the first time. With the old design, the time
required to learn a task like buying a product or finding favorite content was zero — they
already knew how to do it! With the new design, expert users become novices and have to start
learning from scratch.

· Fill users with frustration and anxiety. Urgent tasks aggravate these problems. Even at the best
of times, customers are easily frustrated by high-involvement tasks such as transferring funds
between accounts. That’s because the tasks can be inherently complex and the downside risk of
getting them wrong is high (“Where did my money disappear to?”). Now imagine rushing to
meet a bill payment deadline or trying to buy that last-minute anniversary gift only to find that
the familiar interface changed. Is it any wonder that switchboards light up in the call center?

best practices for minimizing problems caused by redesigns

Fortunately, there are best practices that reduce problems related to redesigns. We spoke with
customer experience professionals who successfully manage the risk of their site changes, both large
and small, through a disciplined approach to rollouts. After analyzing their advice, we conclude that
firms should:

June 23, 2010 © 2010, Forrester Research, Inc. Reproduction Prohibited

 Best Practices For Launching Site Redesigns 3
For Customer Experience Professionals

· Engage users during the design process. Involving your users throughout the design process
not only improves the quality of the finished product but also creates advocates. For example,
online card sorting exercises improve menu redesigns by revealing how real users think about
navigating a site. And because online card sorting is so efficient, compared with the original
model of having users literally sort pieces of paper, some companies involve thousands of
customers in the process — at an affordable price.

· Preview the design with the relevant user base. Customer experience professionals should let
users take control of when they learn a new design by sharing their beta version well in advance
of a launch. An email campaign and a prominent link on the home page (or the landing page
that’s closest to the change) does the trick (see Figure 1 and see Figure 2). Preview versions
should include a feedback mechanism so users can report a final round of flaws and, just as
importantly, feel heard (see Figure 3). Companies can estimate how long to leave up their betas
(or “previews” or “demos”) by examining how frequently their most important customers visit
the part of the site that’s about to change.

· Educate executives on what to expect. Customer experience professionals should take the time
to brief executives on what they’re changing, why they’re changing it, and the steps they’ve taken
to ensure quality (e.g., usability testing). The briefing should include highlights of the testing
results and verbatim user feedback that supports the quality of the change. Most importantly,
the customer experience team should remind executives that users dislike change and that,
despite all of these efforts, they should expect emails, phone calls, letters, social media backlash,
and a spike in call center traffic — all of which will eventually go away.4

· Prepare call centers for a spike in traffic. Real-world data points range from no detectable
spike — which is rare — to a 300% spike in the first month, tapering off to zero over the next
five months. The size of the spike will depend on the number of people affected by the change,
the extent of the change, the quality of the design, and the quality of the communication
program leading up to the change (see Figure 4). The length of the spike will depend on the
length of time it takes for all affected users to learn how to accomplish their goals (see Figure 5).
Firms can help prepare their call center teams by including them in the redesign project team
from the beginning as well as during the quality assurance process. Call center teams should
also be provided with the results of usability testing, a list of changes and fixes that were made
as a result of the usability testing, and a list of potential problems that you expect users may
complain about.

© 2010, Forrester Research, Inc. Reproduction Prohibited June 23, 2010

4 Best Practices For Launching Site Redesigns
For Customer Experience Professionals

Figure 1 Hoover’s Directed Users To Preview Its New Site Via An Email Campaign

Hoover’s included a
prominent link to
the new design.

Source: Hoover’s email campaign

57159 Source: Forrester Research, Inc.

June 23, 2010 © 2010, Forrester Research, Inc. Reproduction Prohibited

 Best Practices For Launching Site Redesigns 5
For Customer Experience Professionals

Figure 2 Bloomberg Directed Users To Preview Its New Site Via A Link On The Home Page

Source: Bloomberg Web site

57159 Source: Forrester Research, Inc.

© 2010, Forrester Research, Inc. Reproduction Prohibited June 23, 2010

6 Best Practices For Launching Site Redesigns
For Customer Experience Professionals

Figure 3 Hoover’s Allowed Users To Provide Feedback Regarding Its New Site

Source: Hoover’s beta Web site

57159 Source: Forrester Research, Inc.

June 23, 2010 © 2010, Forrester Research, Inc. Reproduction Prohibited

 Best Practices For Launching Site Redesigns 7
For Customer Experience Professionals

Figure 4 A Formula For Estimating The Size Of A Call Center Spike As A Result Of Change

To estimate the size of the spike, calculate A x B x C x D, where:

The percent of user population affected by change = A

The degree of change = B (≤ 1.00 where 1.00 means total overhaul)

The quality of improvement = C (≤ 1.00 where lower means better)

The quality of communication program = D (≤ 1.00 where lower means better)

57159 Source: Forrester Research, Inc.

Figure 5 Factors For Estimating The Length Of A Call Center Spike As A Result Of Change

To estimate the timing of the spike, look at the size of your user population
that will encounter the change in a given time period. For example:
Population A visits site daily = bigger spike, shorter duration

Population B visits once per month = smaller spike, longer duration

57159 Source: Forrester Research, Inc.

R ec o mme n d a t i o n s

start by rolling out a truly high-quality site

If customer experience professionals assume that their new site is really good in an absolute
sense, that’s probably a bad assumption. Virtually every site Forrester tests has basic design flaws
that could have been avoided.5 To create a positive experience the first time out of the door, firms
should:

· Conduct upfront user research and testing. Many problems that result from a redesign can
be avoided by conducting upfront user research, such as ethnographic research and persona
creation. These robust research methods can be combined with various testing methods —
including prototyping, user experience reviews, and low-cost research techniques like
surrogate user interviews. Together, these techniques help firms understand user goals that
relate to the business purposes of their various channels.6
· Pick a Scenario Design process and enforce it consistently. In order to design a superior
experience, a firm needs to understand who its users are, what their goals encompass, and
what they need to do in order to achieve their goals. Forrester calls this framework Scenario
Design.7 Scenario Design helps companies avoid the pitfalls that come from designing based
on a product-centric view of the world and instead design based on a customer-centric view

© 2010, Forrester Research, Inc. Reproduction Prohibited June 23, 2010

8 Best Practices For Launching Site Redesigns
For Customer Experience Professionals

of the world. Firms should adopt a Scenario Design process and enforce it across the entire
organization and throughout the entire project life cycle — helping keep customers top of
mind.
· Be willing to put rollouts on hold, based on user feedback. All too often, firms get caught
up in the fact that they’ve invested a great deal of time and money in an initiative and want
to get it live even if it doesn’t meet their users’ needs. Firms must listen to the feedback they
receive — especially when it comes from actual users during beta testing — and be willing
to make changes based on this feedback. For example, a giant financial services company’s
personalized customer portal never made it out of beta because it received a lukewarm
reception.

Endnotes
1
This 5% drop was seen according to the American Customer Satisfaction Index annual eBusiness report,
which is released by the University of Michigan’s Stephen M. Ross School of Business (http://www.bus.
umich.edu/), Ann Arbor, Mich. This national survey is part of a study of 70,000 customers who rank their
experiences with more than 200 companies in 45 industries. It is updated quarterly. Source: Chantal Todé,
“Yahoo’s Customer Satisfaction Takes a Hit,” DMNews, August 14, 2006 (http://www.dmnews.com/yahoos-
customer-satisfaction-takes-a-hit/article/92328/).
2
Even though 1.7 million users only amounted to around 1% of Facebook’s total 175 million user base, it still
represents a large number of dissatisfied users, which can further increase due to the nature of social media.
In addition, Facebook’s own new layout poll — called “Vote on the new Facebook layout” — showed how
unpopular the redesign was with users. Out of more than 1.2 million votes, only just more than 75,000 gave
a thumbs up to the new layout. Source: Daniel Ionescu, “Facebook Redesign Revolt Grows to 1.7M,” PC
World, March 23, 2009 (http://abcnews.go.com/Technology/PCWorld/story?id=7149776).
3
These two types of knowledge that users need to require are “syntactic knowledge” and “semantic
knowledge.” In general, it’s better to minimize the amount of syntactic (low-level) knowledge that users
must learn in order to use a site because it doesn’t benefit the users. For example, the users can’t carry
that knowledge with them to a new site and get value from it there. Requiring users to learn semantic
knowledge (e.g., task flow) can be even more problematic. To ease the burden, designers need to relate
the new concepts to what the users already know, describe the model for use (e.g., showing a diagram of a
process like applying for a loan online), and provide contextual help. For more information on this topic,
refer to one of our favorite books on design, Designing the User Interface. Source: Ben Shneiderman and
Catherine Plaisant, Designing the User Interface, Addison Wesley, 2004.
4
Customer experience professionals should provide realistic estimates of the size of problems. For example,
when expecting a spike in call center traffic, executives should be offered an estimate of the size and length
of the spike. One way to arrive at these estimates is to look at historical norms for past redesigns and then
consider the size of the audience affected by the current redesign.

June 23, 2010 © 2010, Forrester Research, Inc. Reproduction Prohibited

 Best Practices For Launching Site Redesigns 9
For Customer Experience Professionals

5
Because Forrester’s Web Site User Experience Review methodology is intended to diagnose potential
problems rather than simply rating sites as “acceptable,” only a small proportion of the sites we review
achieve a score of 25 or greater. Across more than 1,200 evaluations completed to date, the average score is
0.3.
6
During these uncertain economic times, it’s even more important to satisfy customers by offering them
a high-quality Web experience. Web site reviews (also known as expert reviews, heuristic evaluations, or
scenario reviews) offer an efficient way to identify — and then remove — usability problems that lead to lost
sales and customer dissatisfaction. Companies can gauge their online user experience with an expert review
methodology like Forrester’s Web Site User Experience Review, which is available online (http://www.
forrester.com/cxpbenchmark). For more information about what constitutes a Web Site User Experience
Review, why these reviews are valuable tools for identifying site design flaws, and when they should be used
as part of a mix of usability inspection techniques, see the April 9, 2009, “Web Site Reviews: What, Why,
And When” report.

It’s important to realize that even during a slumping economy when firms can expect shrinking budgets,
user research and testing don’t have to become cost-cutting casualties. To learn more about several lower-
cost user research and usability testing methods that can be employed to gain insight into users, see the
February 3, 2009, “Low-Cost User Research And Usability Testing Techniques” report.
7
Firms know that customer experience is important — but they deal with it haphazardly. As a result,
customers suffer through needlessly painful interactions. That’s why firms need a more disciplined approach
to customer experience. Forrester recommends that companies adopt Scenario Design, a concept built on
a simple assumption: No experience is inherently good or bad, and it can only be judged by looking at how
well it helps customers achieve their goals. This approach requires companies to continually ask — and
answer — three questions: Who are your users?; what are their goals?; and how can you help them achieve
those goals? See the July 19, 2004, “Scenario Design: A Disciplined Approach To Customer Experience”
report.

© 2010, Forrester Research, Inc. Reproduction Prohibited June 23, 2010

 Making Leaders Successful Every Day

Headquarters Research and Sales Offices

Forrester Research, Inc. Forrester has research centers and sales offices in more than 27 cities
400 Technology Square internationally, including Amsterdam; Cambridge, Mass.; Dallas; Dubai;
Cambridge, MA 02139 USA Foster City, Calif.; Frankfurt; London; Madrid; Sydney; Tel Aviv; and Toronto.
Tel: +1 617.613.6000
Fax: +1 617.613.5000 For a complete list of worldwide locations
visit www.forrester.com/about.
Email: forrester@forrester.com
Nasdaq symbol: FORR
www.forrester.com

For information on hard-copy or electronic reprints, please contact Client Support

at +1 866.367.7378, +1 617.613.5730, or clientsupport@forrester.com.
We offer quantity discounts and special pricing for academic and nonprofit institutions.

Forrester Research, Inc. (Nasdaq: FORR)

is an independent research company
that provides pragmatic and forward-
thinking advice to global leaders in
business and technology. Forrester
works with professionals in 20 key roles
at major companies providing
proprietary research, customer insight,
consulting, events, and peer-to-peer
executive programs. For more than 26
years, Forrester has been making IT,
marketing, and technology industry
leaders successful every day. For more
information, visit www.forrester.com.

57159