USABILITY TEST: HUMANE SOCIETY

AND SHELTER SOUTHCOAST

ENL 620

Melanie Martin
12/15/2015
.

TABLE OF CONTENTS
I.

EXECUTIVE SUMMARY

II.

INTRODUCTION

III.

TEST PLAN AND PROCEDURES

IV.

USER DESCRIPTIONS

V.

SCENARIOS

VI.

FINDINGS

VII.

SEVERITY CODES

VIII.

ANALYSIS

IX.

RECOMMENDATIONS

APPENDICES

I.

APPENDIX A HEURISTIC EVALUATION

II.

APPENDIX B INTERVIEW QUESTIONS

III.

APPENDIX C PERSONAS

IV.

APPENDIX D MODERATOR SCRIPT

V.

APPENDIX E PRE-TEST QUESTIONNAIRE

VI.

APPENDIX F POST-TEST QUESTIONNAIRE

VII.

APPENDIX G QUESTIONNAIRE RESULTS

VIII.

APPENDIX H VIDEO CONSENT FORMS (VIDEOS AVAILABLE UPOND REQUEST)

EXECUTIVE SUMMARY
This report identifies and synthesizes the findings of a usability test performed on the Humane
Society and Shelter Southcoasts website. This test was run by myself, as a component for a graduate
course in usability studies at the University of Massachusetts Dartmouth. I selected this organization
because of my interest in rehoming, and rehabilitating dogs. I am also looking to adopt a second dog,
so I became a part of my own audience. This usability test was created to study whether users needs
and expectations are being met by the websites communication. In other words, I examined the
combination of the sites ethos and functionality using four scenarios. When analyzing the results, I
considered basic rhetorical principles alongside three out of Queensburys five Es of usability: effective,
efficient, and engaging.
Before testing actual users, I began this endeavor by creating a series of documents leading up
to the user test, which began with a set of systematic interview questions asked to dog park patrons.
From these results, I created a persona, heuristic evaluation, and test plan. These documents are
included in the appendix for your reference.
Top Findings
Problems: There are problems in three major
areas that contributed toward the diminishing
ethos of the organization: visual, textual,
information architecture. The website was not
created with the users primary goal in mind,
which is finding and adopting a pet. At first, the
organization has a situated ethos as being
associated with the national humane society. But
as the participants went deeper into the study,
the invented ethos of the organization
communicated inconsistency, outdatedness, and
unprofessionalism.
Information architecture

Recommendations: A website overhaul could

rebuild their invented ethos. Organization needs
to understand that most users will come to their
site with a schemata in mind. They will have
looked at other pet finding websites, and have
certain criteria they are looking for in a pet. The
goal of the organization should be to establish
itself as a trustworthy source, provide a visually
engaging platform to most effectively gauge the
personality of an animal before meeting it and
also to have critical information delivered clearly
and efficiently on their website.

Visual

Add information on adoption process,

including fees and vetting. Include a
Choose me button next to the pet
profile that links to the adoption process
Mission Statement should be on the
home page to immediately introduce the
purpose and goals of the organization.
Update with professional photographs
Update with video for more engagement
with the dog or cat profile
Photographs should be taken with
someone knowledgeable about dog

behavior and body language (No tucked

tails or ears back).
Textual

Standardize informational descriptors

(like neutered, cat-friendly, age etc.),
listed in the pet profiles. If a pet is not
cat-friendly, other dogs should
specifically list cat-friendly or not cat
tested. The user should not assumed that
the absence of information makes the
opposite true.
Ensure accurate information
Mission is not written clearly. Should it
match or mirror the mission of the
National Humane Society?

INTRODUCTION

Based on the results of my heuristic evaluation, I created the following plan of how I will
test real users to represent the organizations target audience. This document describes a test plan
for conducting a usability test of the website for the Humane Society and Shelter SouthCoast,
located in Dartmouth, MA. The purpose of the usability test is to establish a baseline of typical
user performance of the site. More specifically, I am looking to determine if users needs are
being met by the way the organization presents information online. Are the users able to
successfully find the information they need about available pets and how to adopt them?

My goal for this test was to identify what was working well for the site, as well as highlight
any credibility and usability issues focusing around three main aspects of the website.

The Search: Identify any user difficulties searching for animals based on preferred
characteristics. Determine how the filtering option works in conjunction with the pet
profile descriptors. Is there a professional level of accuracy and consistency?

The Adoption Process: Users will navigate the site to learn about the adoption process,
but will not find any information. I wish to include this in the test in order to highlight
this omission because nearly every other shelter website I search gives some information

about the adoption process including: the application, steps toward adoption, length of the
process and the fee for each type of animal. The lack of information about this topic
needs to be addressed. Does the lack of information here frustrate the user? Does it
compromise, and call into question, the organizations ethos?

The Ethos: Based on the information communicated on this website written, visual,
auditory, implied etc. what does the user think about the character and credibility about
HSSSC?

TEST PLAN AND PROCEDURES

Four users participated in this user test, but I only analyzed data from three. The first user
was a practice test and caused some changes in the content scenarios. Only some of my users
would have been considered a natural audience for my user test, but I had to use accessible given
the constraints of the project.
User

Age

Occupation

Level of

Currently own

Looking to

Education

a pet/used pet

Adopt in

finding websites next 3

Leah

Jonathan

23

26

Student/TA/Equestrian Masters

before

months

Yes- Dog

No

trainer

student

Customer Service

Bachelors

Yes- Dog

Yes

Applied behavior

Masters

Yes- family

Maybe

analyst

student

owned dog &

Rep/Freelance
Photographer/Former
Shelter Volunteer
Kim

25

cats
Aubrey

23

Student/TA/Cat-owner Masters
student

Yes family
cats

No

The methodology employed was a simple think-aloud protocol, recorded by my laptops

webcam. To begin, I used my moderators script to explain the test in a systematic way. I briefly
introduced the test, showed a video that explained think-aloud protocol and invited each user to
complete a pre-test questionnaire, which is included in the appendix. The users also signed a
video consent form, which allowed me to record the sessions video and audio, freeing my time
to probe the user for feedback. Each testing session took approximately one hour, including the
questionnaires and consent forms.
Users engaged with four scenarios related to the website, which they accessed via the
Google Chrome browser on my laptop. They used the think-aloud protocol to give feedback and
articulate their experience. After each scenario, I asked some follow up questions. I collected
both qualitative and quantitative data throughout the testing, but leaned more on qualitative data.
I used affinity matching to synthesize and organize my findings.

USER DESCRIPTIONS

The personas I created for my heuristic evaluation were based on answers to systematic
interviews that I conducted with dog park patrons. While I recognize that the humane society
potentially has four main audiences (people looking to adopt, people looking for pet-related
resources, people looking help by donating/volunteering, people looking to surrender an animal),
I chose to focus on the subgroup that was looking to adopt a dog, since that is what most of my
interviewees associated with the Humane Societys main mission. For this user test plan, I
intended for the users to be a natural audience for this website, in that they are looking to adopt
a dog. Because this was not always possible, I did at least find users that had gone through the
process of adopting a dog or cat before, either independently or with their family. I do not
believe this significantly affected my findings.
SCENARIOS

I created the scenarios to emulate a narrative of adoption. One task naturally flows into
the next as they would if a real user were visiting the site for the purpose of adopting a dog. The
purpose of the first task is to extract a users first impression of the website and determine their
initial impression of the organizations character, purpose and credibility. The second task was
created to test how efficient and effective the website was at helping their audience find an

adoptable dog with specific qualities. The third task is a qualitative assessment of how engaging
the visuals are, and what they communicate to the user about the organization, and about the pet
specifically. Finally, the last task relates again to the idea of effectiveness. IF the main goal of
this organization is to rehome abandoned pets, then we should know how easily and efficiently a
user can access information about the adoption process.
SCENARIO 1

Youve stumbled upon the Humane Society and Shelter Southcoast after doing a Google search
for animal shelters in the Southeastern Massachusetts area. Explore the home page to find out
whether this site is credible and what kind of organization this is. Can you find the mission
statement?
Post Test Questions. Answers recorded on video

What do you think about the character and credibility about HSSSC?
Would you choose this site on your own to look for a dog, if this was not part of a
usability test? Why/why not?

SCENARIO 2

Select a dog that matches the characteristics you are looking for.

1-4 years old

Housetrained

Cat-friendly

Any size or breed, except the landlord wont allow pit-bulls.

Post Test Questions. Answers recorded on video

Do the descriptions provide accuracy and consistency in their details?

Is this important? Why or why not?

SCENARIO 3

Take a close look at the photographs of the available animals on the pet profiles and/or
featured on the home page. Based on these photographs, what information about the animals
personality can you infer? Give two specific examples of how the photograph of the animal
makes you feel or what it makes you think about him or her?

Post Test Questions. Answers recorded on video

When you look at a photo for a potential dog, what are exactly you looking for?
Are you finding what you are looking for in the visual language surrounding the
adoptable pets?

SCENARIO 4

You have decided on a dog that really seems like a good fit, and are ready to move forward in
the process. Before you and your family come in to meet him/her, you want to know what the
adoption process entails. Navigate the website to find the information you need about the
adoption process.
Post Test Questions. Answers recorded on video

Were you able to find what you were looking for?

What would make it easier? Are there any suggestions or comparisons you could draw
to a model that similar sites use?

FINDINGS
Those who participated in the user test revealed more negative findings than positive, but both
sections are included in this section. I aimed to triangulate the findings by including a quote from the
user, screenshots, and linking these issues to one of Queensburys five Es.
POSITIVE FINDINGS:

Whats Working

Accessible Contact
Information

Humane Society
Situated Ethos

User Quote

Heuristic
Affected

Severity
Rating

The phone number,

address and hours of
operation are listed in
the same spot on
every page. 3/3 Users
approve

I like to see the

contact
information in
the home page
because that
makes me feel
like I can just
reach out to
them. Aubrie

Efficiently allows
user to engage
with the shelter

Positive

The situated ethos, or

inherent association
with the National
Humane Society makes
users feel its an
organized, reliable
organization. 3/3 Users
alluded to this situated
ethos upon first
impression

It seems like its

a credible shelter.
I see the logo
here - Kim

User more likely

to engage with
the site because
they have heard
of the humane
society

Positive

NEGATIVE FINDINGS:
Overall, the negative findings contributed toward diminishing the organizations trustworthiness and
reliability. In the tables below, I have grouped the problems under three subcategories related to visual
problems, textual problems, and information architecture problems.
VISUAL

Photograph
Quality
(Technique,
Composition,
Development)

Content
Quality
(Dog body
language)

Whats the
problem?

User Quote

Examples

Some
photographs
are not of
professional
quality.
Tried to
import
photographs
into cool
zooming
templates,
but they do
not correctly
fit the image
and result in
poor
usability.

Thats
F***ing
creepyIs that
dogs smile that
Photoshopped?
Whose legs are
those? It looks
like hes
choking that
dog.- Aubrie

See figures
below. (1.A
and 1.B)

Engagement- Moderate
If
photographs
are poor
quality, users
might
disengage
with the site
because they
only want to
deal with
professionals

See figures
below (1.C)

Engagement
users may
get the
wrong
impression
about a dogs
personality
and therefore
not pursue
him/her.

Really grainy,
unprofessional.
It looks like
someone just
did a drive by
shooting with
their phone. Jonathan
Photographs The ears are
of dogs are
really
not
important
composed
because they
with body
show you how
language in
the dog is
mind.
feeling... If the
ears or the
head are
cropped off,
thats bad
The
form. It. This
composition close cropping

Heuristic
Affected

Severity
Rating

Moderate

of a
photograph
projects a
certain
persona
about the
dog.

thing makes
me think there
is something
gross in the
background
they are trying
to hide. Jonathan

Visual Recommendations:
Hiring a professional photographer is a luxury that most non-profit shelters cannot afford. A more
practical and creative idea is to offer a professional opportunity to UMass Dartmouth photography
students. The school can partner with the University Service Learning Experience or internship credit.
The student photographers need to be paired with an animal handlers that could play with the dogs to
bring out their best side for the camera. If a dog likes to run, these teams would get a picture of him
catching a ball. If a dog is a love bug, they would get a picture of her enjoying grooming session. These
kinds of photographs not only appeal to pathos, but also to ethos. If a user sees a photograph of a dog
in their glory, that person is not only going to experience an aww moment, but also feel confident
that the shelter values the quality of a dogs life. Additionally, this communicates that both the shelter
and the university care about serving local community needs. To further user engagement by brining
proximity to the user, I would include short one minute videos to be uploaded on each dogs profile.

Figure 1.A

Figure 1.B

Figure 1.C

TEXTUAL

Inconsistency in
descriptions

Whats the
problem?

User Quote

Makes it
difficult to
search animals
with specific
qualities such as
neutered or
cat-friendly.
The omission of
a search term,
like prefers no
cats could be
interpreted that
cats are okay,
but it is unclear

It doesnt say
that she has a
problem with
cats, but it
doesnt say
that she is catfriendly
eitherso Im
probably not
going to look
at Petunia
because I dont
want to risk
it.-Kim

Unclear Mission
Statement

What are the

goals of this
organization? If
a user wants to
find out exactly
what HSSC
does, the
mission is vague
and poorly
worded, and

Example

Heuristic
Affected

Severity Rating

See Figure 2.A

below

Efficiency
Users have to
spend a long
time
comparing
descriptors
and searching
through each
profile to
figure out
whether an
animal has a
desired
quality.

Catastrophic

See figure 2B
below.

Effectiveness
and
engagement

Moderate

I like the blub

about each
animal, but it
still doesnt
give me the
essential
information
that Im
looking for...

Its super dry

and I dont
know exactly
what they are
going to do for
meI think
they are taking
in any kind of
stray animal,
but I dont

includes a lot of
unnecessarily
historical
background
information.

Potentially
inaccurate
information

There are two

dogs posted
that appear to
be the same
dog, but have
slightly
different
descriptors.
One is prefers
no dogs.

know if its all

kinds or if
there is a
capacity
number. I
dont know
what theyre
doing.Aubrie
I have to take
the time to
compare-is
this the same
dog, or is it
not?Clearly
they need to
organize that
better. Now I
dont know if I
can trust it.-

See figure 2C
below.

Effectiveness, Catastrophic
efficiency and
engagement

Recommendations:
First, I would ensure that the organizations mission statement clearly expresses the purpose and the
goals of the organization, not simply its historical roots. The directly impacts the engagement of the
audience, and how effective the shelter is at getting their audience to work with HSSC. Next, I would
make all pet profiles follow a standard schema, or layout. All pet profile should include age, gender,
spay/neuter status, vaccination status, housetraining status, cat friendly or prefers no cats, dog friendly
or prefers no dogs, family friendly or only older children please, as well as any other special need. These
are the basic things that users will ask about a dog. Additionally, HSSC staff can write a little bio about
the individual pets personality, but these main points must be addressed and made searchable for the
sake of efficiency. Finally, the site needs to be checked for errors and duplicate information. When the
reader is confused and doubtful of the organization, the ethos is diminished. Also, from the perspective
of the student conducting this study, I would also like to mention that not once have the pet profiles
been updated since I have started this project. I find it difficult to believe that no animals have been
surrendered or adopted since September. Each pet profile should listed the date the information was
lasted updated.

Figure 2.B

Figure 2.B

Are these the same dog?

Figure 2.C

INFORMATION ARCHITECTURE/SITE ORGANIZATION

Adoption
Process

Whats the
problem?

User Quote

It is not
included! 3/3
users want to
know the fee,
the application
itself and any
rules of
adoption. This is
an omission of
users primary
goal.

I clicked on
the services
tabbut I was
disappointed
to find that all
they talk
about is
boarding and
grooming.
This is
annoying.
-Aubrie

After checking
every tab,
could not find
a tab or
section that
provided
details about
the adoption
process This
information
does not exist
on the site, but
does exist on
nearly every
other shelter
website.
Mission
Statement

The Mission
Statement
should be
placed logical on
the home page
or in its own tab,

Example

Heuristic
Affected

Severity
Rating

N/A

Effective - This Catastrophic

is not an
effective way
to engage the
audience. Even
if the adopter
found an
animal they
like, they is no
practical
information
about the
process.
Calling or
emailing is not
always the
most efficient
way.

N/A

Efficient- it
should not be
difficult to
determine the
main goal of a
website.

I need to
know if I can
bring my dog
to meet the
dog before we
adopt.Jonathan

Id expect to
find some kind
of button here
that would
take me to the
adoption
process. Aubrie
Id like this to
be on its own
so you can get
the first
impression

Major

Filtering
Option

not under the

services tab. 3/3
Participants
searched home
page first when
they were asked
to find the
mission
statement.

right away.
Jonathan

The animals are

not sorted
correctly. There
are rabbits and
small animals in
with the cats

Ohthats
not a catDo I
have to scroll
through all of
these Aurbie

See figure below.

3.A

Effective
Does not sort
animals
accurately

Major

There are only

two options to
filter through
pet profiles:
dogs and cats. It
would be
efficient if users
could sort
through dogs by
certain qualities.

Recommendations:
The most important thing to remedy here is the lack of information on the adoption process. Create a
tab specifically for adoption that includes the process, associated fees and what kind send-off your pet
will receive (check-up, free collar etc.). The adoption requirements, policies or application should also
be available. This is a key piece of information that users are looking for. The filtering option should also
be more complex, in order to increase search efficacy and accuracy. The mission statement should
either be included on the home page, or in its own tab, in order to be up front with users about the
goals of the organization. The mission does not fit under Services.

Figure 3.A

SEVERITY CODES
Below is the rating system used to identify priorities in the user test results.
Positive Helps the user efficiently and effectively achieve their goal. Engaging content.
Minor Problem- Not the most efficient or effective way to help user achieve goal. Neither engaging nor
disengaging.
Major Problem-Causes significant frustration in the user. Inhibits them from accomplishing their goal,
or user is able to only partially achieve goal. Causes user to disengage.
Catastrophic Problem- Resulted in task failure. User is not able to accomplish their goal. Site does not
meet their needs.

ANALYSIS
My analysis of Pre-test questionnaire and post questionnaires, as well as the post-task verbal
questions, leads me to conclude that this website was not truly created with the users needs in mind. It
goes back to the mission statement. If the organizations main purpose is to find homes for adoptable
dogs and cats, then everything on that website should be effective, efficient, engaging for the
audience. People looking to adopt have a schema in mind of what kinds of traits they want in a
potential dog (or a shelter/rescue). If the user cannot sift through the information on the site naturally
and connect with a dogs profile, they will disengage and move on to another website. If they are
engaged with the visual communication, they will become annoyed at the lack of logic when they
cannot find any information about the adoption process on the website.
RECOMMENDATIONS

At first glance, users had a neutral or positive impression of the website, because of the
nature of the organization. They enjoyed seeing pictures of the animals and were excited to be
helping a worthwhile cause. However, their level of patience with the site, as well as their trust
in the organization diminished. Below are my short-term and long-term recommendations for
better meeting the needs of the average user.
I would start by creating a page dedicated to the adoption process, because this is vital
information that is missing from the site, deterring people from adoption with HSSC. This page
should include fees, process, and application. Later on, they could make the application
available electronically via PDF or an online portal. Then, I would tackle the consistency and
accuracy in the pet descriptors, and making sure people could efficiently filter through
potentials pets, and be assured of when the information was last updated.
I would encourage making the recommended modifications to the HSSC website, and then
retesting afterwards. It could be useful to test a larger pool of people who are confirmed members of
their authentic audience. Additionally, HSSC could conduct a user test focused on whether the site
meets the needs of a different subcategory of audience, like current pet owners in need of resources.
The test could be conducted on a different demographic based on age and experience as well. All in all,
this user test helped to identify and analyze what the HSSC website needs to do in order to improve
their ethos, effectiveness, efficiency and engage with the users needs.

APPENDICES

I.

APPENDIX A HEURISTIC EVALUATION

II.

APPENDIX B INTERVIEW QUESTIONS

III.

APPENDIX C PERSONAS

IV.

APPENDIX D MODERATOR SCRIPT

V.

APPENDIX E PRE-TEST QUESTIONNAIRE

VI.

APPENDIX F POST-TEST QUESTIONNAIRE

VII.

APPENDIX G QUESTIONNAIRE RESULTS

VIII.

APPENDIX H VIDEO CONSENT FORMS (VIDEOS AVAILABLE UPOND REQUEST)

HEURISTIC EVALUATION

SouthCoast Humane Society & Shelter

Heuristic Evaluation

Melanie Martin

ENL 620 Usability Studies

TABLE OF CONTENTS

INTRODUCTION.......................................................................................3

METHODOLOGY .............................................................................................................4-9

EVALUATION
PROCESS......................................................................................................................4

TASKS...........................................................................................................................4

PERSONA & SCENARIO OF USE........................................................................................4-5

HEURISTICS.................................................................................................................69

RANKING SYSTEM.........................................................................................................9

FINDINGS....................................................................................................10-21

TASKS: INITIAL REACTIONS....................................................................................10-18

RANKINGS & RECOMMENDATIONS.............................................................................18-21

CONCLUSION.............................................................................................21

Introduction
I conducted a heuristic evaluation of the Humane Society and Shelter SouthCoast to
evaluate the sites usability. Heuristics are a set of relevant qualities or criteria that are used by
experts to analyze and measure the usability of a particular website. I like to think of it as a
rubric used to grade a website.
I conducted this heuristic evaluation through the eyes of Alana Mattos, a persona I
created from the results of systematic interviews. She represents one type of audience that the
HSSC site may have, but does not represent all user groups.
I selected this local organization because animal shelter websites are the first
impression a person has about a potential pet adoption. The rhetoric of the image and the
surrounding text can determine whether a person will be persuaded to make the next step
toward changing an animals life. The level of the sites usability and usefulness directly effects
whether a user would continue with HSSC or go to a different site that could better meet their
needs, like Petfinder or Forever Paws. I have prior experience in terms of content knowledge as
a dog-owner, shelter volunteer, animal handler and photographer/videographer for the
Waterford, CT Humane Society.
My goals in evaluating this website are to identify areas of the site that hinder the user
from searching through pet profiles, finding contact information, and learning about the
adoption process some of the main reasons people go on hsssc.org. Eventually, I may present
my findings to the organization for their own use.
This report will: define the heuristics used to evaluate the tasks performed by my
persona, walk through the user experience, identify the findings and rank them by severity,
suggest recommendations, and detail my conclusions.

Methodology

Evaluation Process
In this heuristic evaluation, I set out to uncover evidence of poor usability and potential
frustration triggers for the user of hsssc.org. A dissatisfied user or a user who cannot find what
they are looking for might not take the next step toward visiting the shelter and adopting.
Alana completed three tasks on the website and noted her comments about the website. This
think-aloud process organically captures how the user thinks and feels as they use the site. I
then took my personas initial feedback and synthesized it in the context of my heuristics,
which you will see in the tables under Rankings.
Tasks
Using the website for Humane Society & Shelter SouthCoast (http://hsssc.org), please
complete the following tasks:
1. Find and select a dog that is good with cats as well as children under five.
2. Find information about the adoption process, such as an application/screening,
adoption fee, or waiting period.
3. Locate contact information and hours of operation.

Persona & Scenario of Use

My personas name is Alana Mattos, and she was created from the results of systematic
interviews. Alana is a 32 year old receptionist with a husband, a five year old, and cat. They live
in a house with a fenced-in yard in Westport, Massachusetts. Her daughter has been asking for
a dog for several months now, and Alana finally feels like it is the right time to expand the
family. She is using hssc.org, as well as other shelter websites, to search for a family dog that
will be happy and fit in well in this type of environment. She conducts these searches from her
home computer, usually on the evenings or weekends so the rest of her family can participate
too.

ALANA MATTOS
We want to add another fur baby to family!
Demographics:

33 years old
BA Degree/Receptionist
Married
Mother of 1 cat, 1 human child (age 5)
Local homeowner with fenced yard

.Goals & Concerns:

Wants to add a canine friend to the

family
Concerned with making sure its a good
match between the dog and her cat,
Humphrey. Doesnt want to put her
kitty in danger

Adoption Information Expectations:

Photo
Breed
Age
Temperament (good with cats and
kids)
Spay/Neuter status
Vaccinated
Application Process
Adoption Fees

Alana is an animal welfare advocate because she

has donated her time to help ease the suffering
of abandoned animals at shelters. She has
signed petitions on behalf of animal rights issues

and policies. Her past experience with the

Humane Society has been a positive one, since

she last adopted her cat, Humphrey, from the

Humane Society in Waterford, CT.
Services Expectations:

List of local recommended contact points in

the community for the following:

Boarding, cost
Grooming, cost
Training, cost
How to volunteer

Heuristics Set

I selected three heuristics from the set used in the research by Ginny Redish et al, New
Heuristics for Understanding Older Adults as Web Users. I found that this study revealed
heuristics that could address the needs of diverse user groups, not just older adults. I especially
wanted a heuristic that specifically mentioned writing for audience and purpose, since the
whole point of usability studies revolves around the audiences needs. I selected the three
heuristics I felt were most relevant to the website I am evaluating.
1. Visually group related topics

Is the amount of information sparse, dense, or in between appropriate for the

audience and type of site?
Are the most important and frequently used topics, features, and functions, close to the
center of the page rather than in the far left or right margins?
Are the task-related topics grouped together?
Are the frequently used topics, actions, and likes above the fold?

2. Make it easy to find things on the page quickly.

Is the amount of text minimized; is only; necessary information present?

Is there liberal use of headings, bulleted lists, and links to assist skimming?

3. Focus the writing on audience and purpose

Is the content written in active voice, directed to you?

Are sentences short, simple, and straightforward?
Are headings, labels, and captions of descriptive of associated content?

I also selected four heuristics from the set used in the research by Grice et al, in Heuristics
for Broader Assessment of Effectiveness and Usability in Technology- Mediated Technical
Communication. I found that this study revealed heuristics that were more qualitative than
Nielsons ten heuristics, which is something that is useful to a user who is looking for an animal
companion. Nielsons ten did not fit the needs of this evaluation because they seemed too
focused on error prevention, which is not the focus here.

1. Design for usability

a. Follow standard usability guidelines
Confusing non-traditional design < > User recognized standard elements
Product Metric: Design follows usability guidelines.
Behavioral Metric: User understands the design based on other experiences.
Survey Metric: User describes experience as a familiar one
b. Enforce readability (font large enough to read; break up blocks of text)
User disoriented or led astray < > User easily perceives site content
Product Metric: Design is well organized and easy to navigate.
Behavioral Metric: User finds what he or she is looking for in a timely manner.
Survey Metric: User describes experience as efficient.
c. Use professional quality design components
Design perceived to be standard < > Design perceived to be enhanced
Product Metric: Appearance and content suggest professionalism to user.
Behavioral Metric: User prefers design vs. other designs.
Survey Metric: User describes experience as professional.
d. Follow general conventions where available
Highly unfamiliar < > User experiences familiarity where expected
Product Metric: The design is organized and consistently familiar.
Behavioral Metric: User is more comfortable with the design vs. others.
Survey Metric: User describes the experience and familiar and enhanced
e. Offer simple ways to do what users want to do
Many navigation complications < > Quick, free user motion throughout
Product Metric: Components are in correct locations. Links work.
Behavioral Metric: User efficiently navigates through site/design.
Survey Metric: User describes experience as uncomplicated.

2. Make users feel welcome

a. Make users feel welcome

User feels put off or unwelcome < > Users feel welcome
Product Metric: Design and experience feels welcoming and friendly.
Behavioral Metric: User lingers/spends more time in initial, welcoming screens or areas.
Survey Metric: User describes experience as welcoming or inviting.
b. Use visuals to draw users in
Users is intrigued by visuals < > User is annoyed by visuals
Product Metric: User is engaged by visuals, not distracted by them.
Survey Metric: User describes visuals as enhancing the experience or as highly useful and
helpful.
c. Engage the affective domain with visual language (color, icons, and symbols)
User unresponsive to design < > Appropriate user emotions are triggered
Product Metric: Visual elements stimulate user emotional engagement.
Behavioral Metric: User responds to visual language, is drawn in.
Survey Metric: User describes visual language used as engaging, enhancing the experience, or
in terms of appropriate emotional response.

3. Make a connection
a. Engage people in what is going on; create connectedness
User feels detached < > Users feel drawn in
Product Metric: Users can relate to elements of the experience.
Behavioral Metric: User is focused on the product. User takes less time to learn. User is
immersed in the experience.
Survey Metric: User rates the connectedness of the experience highly, or describes it as
immersive.
b. Understand potential barriers and offer users identifiable ways to overcome them
Users get stuck < > Users overcome barriers quickly and easily

Product Metric: Barriers are minimal; universally identifiable and easily grasped hooks offer
routes through any necessary barriers.
Behavioral Metric: User does not encounter design barriers, or easily overcomes obstacles.
Survey Metric: User perceives experience to be barrier-free. Users describe hooks they
encounter as easily understood.
c. Use well-crafted storytelling to immerse users in the encounter
User uninvolved, rejects premise < > User is drawn into story/encounter
Product Metric: Story is worked into experience seamlessly.
Behavioral Metric: User is invested in story and encounter, does not want to leave experience.
Survey Metric: User rates storytelling highly, describes encounter as immersive

4. Plan to continue the engagement

a. Design for the next engagement
User is stuck in the past < > User is ready to continue
Product Metric: Relationship with product is ongoing, can persist beyond a single experience or
task.
Behavioral Metric: User is drawn into/stays with experience, is willing to return to it. Survey
Metric: User rates continuity of engagement highly, spends more time with the experience.
b. Make calls to action clear
User is stuck deciding < > User understands what to do next
Product Metric: The experience includes beneficial guidance for the user and clear action
options.
Behavioral Metric: User makes easy progress from action to action.
Survey Metric: User rates calls to action as clear and easily understood.
c. Invite users to continue connections past the current encounter
User has no interest in continuing < > User pursues deeper connection(s)
Product Metric: Progress beyond any given point is available to the user.
Behavioral Metric: User moves deeper into the experience.
Product Metric: User rates access to further experiences highly.

Severity Rankings
Low Minor issues with little to no impact on user experience. Task can be completed.
Medium Recurring or more noticeable issues with significant impact on user experience. User
is able to complete task, but it is delayed or the quality of experience is affected.
High Major issues that may result in major hindrance, task failure, or lead to high user
frustration.

Findings
Tasks: Initial Reactions
This section of the evaluation will describe Alanas interaction with the website as she performs
the task. Included here is the personas narrative walkthrough and accompanying screenshots.
This section only highlights her comments as directly related to task completion. Other
comments and findings about that relate to the heuristics will be noted in the rankings section.

TASK 1: Find a dog profile compatible with cats as well as children under five.

Okay, Im starting at the home page here and

looking toward the green horizontal bar with the
logo and title. This text is a bit small for a menu, but
as I expected, I see All Our Pets so Ill click there.

I cannot filter the pets, like I can on Petfinder. There are only two options:
Dog or Cat- which is funny because they have a rabbit here under the cat
section. What about small animals?
Also, I think the photographs on this site where done by an amateur
because they are constantly blurry, have red eye, poor composition, or
poorly cropped cut off at the bottom of the page...

Okay so after I click on Dog in the filter

section, I can now see the profiles. I have to
do a lot of vertical scrolling to see all of
them.

The descriptors are not consistent in how

they present information. For example, the
profile on the left says prefers no cats and
prefers no dogs. The profile on the right
does not list those descriptors, so does that
mean I can assume Frosty prefers cats or at
least tolerates them?

When I click on the picture, the site gives me a little more

information. This dogs name is Petunia. HSSC suggests homes
with a fenced yard, which we have, and children over 5. Im sure
age 5 would be ok. It depends on each dogs personality
HmmmShe looks like she could be a nervous or shy dog because
her tail is tucked and her ears are back in this photograph.
Ive also heard that many Greyhounds dont mix well with cats
because of their instinct to chase. I wish it said specifically
whether she was ok with cats or not

What a sweet girl! She has a friendly face here and it looks like
shes wagging her tail. Well have plenty of room for her. It
doesnt say whether she is specifically cat-friendly, but her
description does say that she is calm and docile. It also does not
say that she is up-to-date with shots or neutered, but I know that
is standard shelter practice. Anyway, I think we should go and
meet her!

TASK 2: Find information about the adoption process.

Im starting at the home page again, with the
headless dog at the bottom. Hah! Anyway, it is unclear
to me where adoption information might be. The
menu at the top just lists: All our pets, our services,
help us and contact. I was just under all our pets, so I
know its no there. Let me try the services page.

A headless dog. Bad

choice in general, but
especially for the home
page.

This page does not have what Im looking for either. In fact, its a poorly organized page in
my opinion. The mission should be under an about tab or on the home page, not under
services.

So
the last
logical
to look
information
about
adoption
As
a side
note,
thereplace
are only
twofor
services
listed on
this the
page
when I
process
is
under
the
Contact
page,
but
again,
it
is
not
here.
Oops
I
scroll down -boarding and grooming. What about training,
think I just completed
the many
final task!
The
phonethat
number,
address and
spay/neuter
programs and
other
services
the National
hours ofsociety
operation
are clearly available on this page with a map, and
Humane
provides?
on the right hand side of every page.

And while Im
talking about the
contact
information, let me
just mention that
its really annoying
that they do not
have an email
address! The hours
of operation are
very limited (MonSat, Noon-5). Im at
work during most
of these times, so
an email address
would be most
efficient.

Well they do have this contact form here if I scroll to the

bottom, so I could send a message that way. I just find it
strange they dont have a direct email.

To be absolutely through, I
clicked on the last possible
option, Help Us in effort to find
information about adoption
process, application fees.
Nothing. Just some information
about bricks. I guess if I want to
meet Precious, I will either have
to call during my break at work
to make arrangements to see
her on a Saturday- if shes still
available by then.

Rankings & Recommendations

Task 1: Find and select a dog that is good with cats as well as children under
five.

Obstacle & Recommendation

Heuristic Affected

Severity Ranking

There were pictures of rabbits and

guinea pigs grouped in with the
cats. A more specific filter is needed
to search for an animal that meets
the needs and preferences of the
user.

Visually group related topics

Medium The user

was able to find a
dog, but if she were
looking for a guinea
pig she would have
a difficult time or
even miss it.

The descriptions and captions

under pet profiles need to be
consistent and more through. Ex.
Can we assume that the omission
of

Focus the writing on audience and

purpose

Medium-not
enough or not
thorough enough to
meet the needs of
the audience

They did not use professional

quality design components.
Photographs are grainy, poorly
cropped, low quality images. Dog
body language in the photographs
is sometimes not welcoming (tail
tucked, ears back etc.). Use animal
handlers to help photographers
best capture the natures and
personality of the animal.

Design for usability

High- This affects a

users perception of
the organizations
ethos, but also their
perception of an
animal.

Some of the text is a bit small,

especially for navigation. It is
readable

Design for usability

Low

User understands the design based

on other experiences since she is
using other website to search for a
compatible dog. This schema she
has allows her to expect how to
search the site.

Design for usability

Low

User does not linger on the home

page because of poor design and
organization. The screen seems too
large for a typical computer screen.
Too much vertical scrolling

Make users feel welcome

Medium- The home

page should be like
a welcome mat for
the user

The user should be highly engaged

with the photographs, and
descriptions of the animals. The use
of high quality photos and perhaps
even video would make the user
feel welcome and well informed
about different animals and the
shelter itself.

Make users feel welcome

Medium- This
affects a users
perception of the
organizations
ethos, but also their
perception of an
animal.

Site needs to better engage the

affective domain with visual
language. Though the icon and
green color scheme is pleasant,
there needs to be more emotional
engagement. If each profile had an
accompanying video, the user
would feel closer to the animal.

Make users feel welcome

Low

The descriptions could benefit from

well-crafted storytelling to
immerse users in the encounter and
get them invested in story of each
animal. Could potentially write
them from the first person point of
view.

Make a connection

Low

TASK 2: Find information about the adoption process.

Obstacle & Recommendation

Heuristic Affected

After checking every tab, could not

find a tab or section that provided
details about the adoption process.

-Plan to continue the engagement

Visually group related topics
-Make it easy to find things on the
page quickly.

Severity Ranking
High task failure,
high user
frustration.

-Focus the writing on audience and

purpose
This information does not exist on
the site, but does exist on nearly
every other shelter website.

Design for usability

Make users feel welcome

High-Task failure,
high user frustration

Make a connection
Plan to continue the engagement

TASK 3: Locate contact information and hours of operation.

Obstacle & Recommendation

The phone number, address and

hours of operation are listed in the
same spot on every page. I would
recommend slightly cleaning up the
visual appearance (larger block
text).
User could not find a direct email,
only a contact form. A direct
contact persons email may help
the user feel more connected and
help them continue the
engagement at their convenience.

Heuristic Affected

Severity Ranking

Make it easy to find things on the

page quickly.

Not an issue/low

-Focus the writing on audience and

purpose

Low

-Make users feel welcome

-Plan to continue the engagement

Conclusion:
My heuristic evaluation of the South Coast Humane Society & Shelter exposed many usability
issues that were of low to medium severity, and one that was ended in task failure. Because of
the lack of content about next steps in the adoption process, I think it would be more helpful to
present this heuristic evaluation to the organization than to user test something that is not
there. Going forward with the usability test plan, I will focus on how the user interacts with the
filtering option while trying to find a compatible companion.

INTERVIEW QUESTIONS

Humane Society & Shelter-SouthCoast

Interview Questions
1. How many and what kind of pet(s) do you currently have?
2. Are you looking to adopt another pet within the next 3 months? Dog/cat?
3. Do you think of yourself as an animal welfare advocate? Why or why not?
4. What is your perception of the Humane Society, as in what the Humane Society is and
what are some of the organizations goals?
5. What is the purpose of the Humane Societys website? Why would you look at it?
6. What key information would you expect to find on the home page?
7. If you are looking to adopt a pet from a shelter, what kinds of information do you expect
to find on their website?
8. If you are looking for services (boarding, spay/neuter, grooming, training, animal care
tips etc.) for your animals, what kinds of information do you expect to find on their
website?
9. If you are looking for a place to surrender stray or unwanted animals, what kind of
information do you expect to find on the Humane Societys website?

ALANA MATTOS
We want to add another fur baby to family!
Demographics:

33 years old
BA Degree/Receptionist
Married
Mother of 1 cat, 1 human child (age 5)
Homeowner with fenced yard

.
Goals & Concerns:
Wants to add a canine friend to the
family
Concerned with making sure its a good
match between the dog and her cat,
Humphrey. Doesnt want to put her
kitty in danger

Alana is an animal welfare advocate because she

has donated her time to help ease the suffering of
abandoned animals at shelters. She has signed
petitions on behalf of animal rights issues and
policies. Her past experience with the Humane
Society has been a positive one, since she last
adopted her cat, Humphrey

Adoption Information Expectations:

Photo
Breed
Age
Temperament (good with cats and kids)
Spay/Neuter status
Vaccinated
Application Process
Adoption Fees
Services Expectations:
List of local recommended contact points in the
community for the following:
Boarding, cost
Grooming, cost
Training, cost
How to volunteer

PERSONAS

TODD CABRAL
Roscoe ate the couch again.
Demographics:

26 years old
Associates Degree/Physical Therapist
Engaged
Has 1 dog, No children
Rents apartment with fenced dog park

Goals & Concerns:

Todd believes he is an animal welfare advocate

because he is an avid dog lover and adopted
Roscoe. He isnt really sure what it means to be
an animal advocate. He perceives the Humane
Society as a good organization that helps with
pet adoption and maybe that recuses animals
from abusive situations.

Wants to get another dog, but not

compound Roscoes behavior issues
(counter surfing, chewing)
Would like to adopt a dog who is very
playful and would fit in with their
athletic lifestyle

Adoption Information Expectations:

Photo and Video
Breed
Age
Temperament
Spay/Neuter status
Vaccinated
Website regularly updated
Services Expectations: (Didnt really know this
was available, but excited when he found out)
Contact Information for local trainers,
including personal recommendations or
ratings
Contact info for doggie day
care/boarding,
Melanie Martin

MODERATORS SCRIPT

Welcome! Thank you for participating in this usability testing session. For the purpose
of keeping these tests systematic, I will be reading from a script.
The objective of this study is to test the usability and usefulness of the Humane Society
and Shelter Southcoasts website. I am interested in studying how the average user interacts
the website and what kind of information the user gathers from its content. The point of this
user test is to identify any struggles a user might have when visiting the website for its most
common purpose finding and adopting a pet. I want to emphasize that your honest
feedback, both positive and negative, will help improve the website.
You will be using the think-aloud protocol in conjunction with pre-tests, post-task, and
post-test questionnaires to provide feedback. The think-aloud protocol is a method of thinking
out loud when solving a problem, which allows me to better understand if and how the website
is helping the users reach their goal. Lets take a minute to watch the instructional video that
explains the think-aloud protocol.* http://www.nngroup.com/articles/thinking-aloud-demovideo/
I am also recording the session on video - primarily, to remember your thought process
throughout the tasks, but also to capture any emotional reactions the user may have. If you are
comfortable with this, please signed the Video Consent Form. *
Before we get started, I am going to ask you to take a few moments and complete the
Pre-test Questionnaire* to discover what your expectations are for this website.
For the testing process, I will read aloud three scenarios and ask you, the user, to
perform some related tasks using the website. I will also provide you with the written version of
the scenario for your own reference as you approach the task at hand. I will remain seated near
you, and may take notes while you interact with the website. I might prompt you for more
feedback during the test, otherwise, Ill remain silent. Unless you have any questions, lets
begin!

Scenario 1
Youve stumbled upon the Humane Society and Shelter Southcoast after doing a Google
search for animal shelters in the Southeastern Massachusetts area. Explore the home page to
find out whether this site is credible and what kind of organization this is. Can you find the
mission statement?
*Post Task Questions
Scenario 2
You are looking to adopt a dog and want to search through local shelters to find just the right
one that will fit your family and lifestyle. You are searching for the following traits:

1-4 years old

Housetrained

Cat-friendly

Any size or breed, except the landlord wont allow pit-bulls.

Select a dog that matches the characteristics you are looking for.
*Post Task Questions

Scenario 3
Take a close look at the photographs of the available animals on the pet profiles and/or
featured on the home page. Based on these photographs, what information about the animals
personality can you infer? Give two specific examples of how the photograph of the animal
makes you feel or what it makes you think about him or her?
*Post Task Questions

Scenario 4
You have decided on a dog that really seems like a good fit, and are ready to move forward in
the process. Before you and your family come in to meet him/her, you want to know what the
adoption process entails. Navigate the website to find the information you need about the
adoption process.
*Post Task Questions

Thank you for taking the time to participate in this user test. Your feedback is valuable to us.
Please help yourself to some refreshments on the way out.

QUESTIONNAIRES

Name: _______________________

Pre-Test Questionnaire

1. What is your perception of the Humane Society? What are some of the organizations
goals?
_________________________________________________________________________
_______________________________________________________________________
2. What is the purpose of the Humane Societys website? How should the website support
the mission of the organization?
_________________________________________________________________________
______________________________________________________________________

3. What key information would you expect to find on the home page?
_________________________________________________________________________
_______________________________________________________________________
4. If you are looking to adopt a pet from a shelter, what kinds of related information do
you expect to find on the shelters website about adopting a pet?
_________________________________________________________________________
_______________________________________________________________________

5. Do you have any pets? If so, how many and what kind?
_________________________________________________________________________
_______________________________________________________________________

Name: _______________________

Post-Test Questionnaire
1. After your interaction with this website, what is your opinion about the character and
credibility of this organization? Give examples to support your opinion.
_________________________________________________________________________
_______________________________________________________________________
2. On a scale of 1-10, how well did the website support the mission of the organization?
Explain.
_________________________________________________________________________
_______________________________________________________________________
3. On a scale of 1-10, how satisfied are you with the visual and verbal information
presented in the pet profiles?
Explain.___________________________________________________________________
_____________________________________________________________________
4. On a scale of 1-10, how easy was it to clearly find the information the information you
needed on this site? Explain.
_________________________________________________________________________
_______________________________________________________________________
5. On a scale of 1-10, how important is it for the adoption process information to be
available online? Explain.
_________________________________________________________________________
_______________________________________________________________________

6. On a scale of 1-10, how likely is it that you would adopt from HSSC if you were looking
to adopt? Explain.
_________________________________________________________________________
_______________________________________________________________________

7. Do you have any other suggestions or comments? Can you draw any comparisons to
similar, successful websites?
___________________________________________________

Name: _______________________

Post-Task Questions (Verbal)

Scenario 1: The Home Page/Ethos

What do you think about the character and credibility about HSSSC?

Would you choose this site on your own to look for a dog, if this was not part of a
usability test? Why/why not?

Scenario 2: Find a Dog

Do the descriptions provide accuracy and consistency in their details?

Is this important? Why?

Scenario 3: Pet Photos/Visual Language

When you look at a photo for a potential dog, what are exactly you looking for?

Are you finding what you are looking for in the visual language surrounding the
adoptable pets?

Scenario 4: The Adoption Process

Were you able to find what you were looking for?

What would make it easier? Are there any suggestions or comparisons you could draw
to a model that similar sites use?

APPENDIX: QUESTIONNAIRE RESPONSES

VIDEO CONSENT FORMS