Usability Test Exercise

Team Members:
TEAM_MEMBER_1
TEAM_MEMBER_2
TEAM_MEMBER_3
TEAM_MEMBER_4
TEAM_MEMBER_5
TEAM_MEMBER_6
TEAM_MEMBER_7
 TEST PLAN

Purpose

 To identify the features and usability deficiencies of the site.

 To verify how efficient, effective and user-friendly the website is to the prospective users.
 To know the ease of remembering the way a system must be operated.
 To determine how quickly and easily users can gain information they need from the
website which is new for them.

Problem Statement/ Test Objectives

Below presents the most common general issues while browsing through websites:
 How fast user can complete the task?
 Is the user interface design helpful in accelerating user’s activity?
 Do users easily identify clickable buttons or links?
 Do search bar helpful in completing the task?
 Is the consistency of the design achieved?
 Is navigation links matches to what user expect it to be?

The following shows the most common issues or activity done by users, specifically
students in accessing websites. This leads to some usability deficiencies of the site.
 Students usually apply try-and-error method in exploring websites, trying to recognize
pages-by-pages until they reach to their destination rather than recalling previous
processes.
 Most of the students often neglect other pages because they directly navigate to the page
which title known to them or they think is intended for them.
 Today’s generation, students rely themselves on using search bars instead of using menus
in accomplishing tasks.
 Mostly of the students when get lost, immediately start from the very beginning that is
navigating back to home page and start all over again.

User/ Participant Profile

 Candidate students are those who are looking for general information about universities
through their websites. These include students who are going to be freshmen in college,
undergraduate college students, and graduate students who are planning to take their
masters and doctorate degree.
 At most 15 participants
 Ages 15-above
 Could be female or male
 Either has their experience of viewing university websites or not.

Test Monitor Role

  She will be the one to give and instruct participants in answering the questionnaires.
 The participant could only interact with the test monitor not to the rest of the team or
observers.
 She could also interact with the rest of the team while the participant is currently doing
the task for some problem matters or anything
 She should be consistent in every participant to avoid bias, that’s why a script is given to
her before the actual testing to familiarize
 She will not help the participant in solving the task or rescuing the participant
 In case of errors she will only motivate or encourage the participant to proceed

Methods

 Test Roles
o Usability testing members are assigned to different test roles that are the
following:
 Test monitor – TEAM_MEMBER_5 act as the test monitor. She will be
the one who has the direct contact/ interaction with the participant, while
the rest of the team was not allowed to interact with the participant. She
assists, orient and give follow-up questionnaires to the participant. In short
she is responsible for administering the test.
 Timer and Video Recording operator – TEAM_MEMBER_1 ,
TEAM_MEMBER_3 and TEAM_MEMBER_4 are responsible for this.
They record the amount of time taken by the participant in performing the
given task. They also operate video cameras to take the behavior of the
participant while performing the task.
 Data Loggers – in persons of TEAM_MEMBER_7 , TEAM_MEMBER_2
and TEAM_MEMBER_6 . They take down notes of the performance of
the participants while doing the test. They also take down notes of the
responses of the participants during debriefing.

 Pilot Testing
Practitioners should always plan for a pilot test before running a usability test. A
usability test is a designed artifact, and like any other designed artifact needs at least
some usability testing to find problems in the test procedures and materials. These tasks
were conducted in the following manner:
1. Have an initial walkthrough with a member of the usability test team or some
other convenient participant.
2. After making the appropriate adjustments, the next pilot participant should be
a more representative participant.
3. If there are no changes made to the design of the usability test after running
this participant, then the second pilot participant can become the first real
participant (but this is rare).
4. Pilot testing should continue until the test procedures and materials have
become stable.
 Performance Testing/ Measurement
The Performance test consisted of a series of tasks that participants carried out
while being observed. These tasks were conducted in the following manner:
1. Prior to the performance test, it is rest assured that the usability testing team
(test monitor, data logger, timer, video recording operators, test observers) are
prepared for their respective assigned duties and so as the test environment.
2. Participants are brought in the room one by one at a time with the test
monitor. Each participant was made to feel comfortable and relaxed.
3. The participants are asked to fill out a short pre-study questionnaire, which
gathers background information about the website.
4. Each participant receives a scripted, verbal introduction and orientation,
which explained the purpose and goals of the test by the test monitor itself.
5. Participant will be given the script, which details the task to be accomplished.
Then they start the task.
6. During the test, whenever the participant struggles to accomplish a certain
task, he/she may approach the test monitor for clarification, and so does the
test monitor must be sensitive by the verbal and nonverbal language of a
participant.
7. Also, a participant is not comply to do all task, and is free to stop even at the
midst of performance test or complete the series of task.
8. A participant is ought to mark check the corresponding task if he/she
successfully completed it within the given time, otherwise, mark X and
proceed to the succeeding task.
9. After the series of task, the participant completed a short survey to gauge their
responses to the tasks. During the performance test the monitor made notes on
elapsed time and participant errors.
 TASK LIST

 Below is the generic task given to the participant during the actual testing. Each task is
numbered accordingly to help the proponents keep track of the activity and to ensure
proper execution and organized things as it is decided to be. An allocated time is given to
the participant in completing task; this will be used by the proponents to conclude some
deficiency found on the website. In addition, the time allocated is just a rough estimate
that the website is expected to respond, regardless of the internetwork connection factors.

Task Task Description Timing

Number (mins:secs)
1 You are given 3 minutes of free viewing and exploration of the 03:00
website.
2 Make sure you are now on the home page; if not kindly 00:05
navigate to the home page of this site.
3 View the entrance exam result for the current school year or 03:00
next school year whichever is available. On the paper provided
list the first three (3) qualifiers.
4 Look for the offered courses. See if your preferred course is 05:00
available.
5 Look for available scholarships. 03:00
TOTAL 14:05

Data Collection Instrument

 Below is a simple instrument used by the proponents in recording or tracking the

participant’s performance during testing. Each observation is matched to the
corresponding task number performed and accomplished.

Date: Participant No.: ____

Task No. Time Elapsed: Observation/s
1 05:00  He views student page
 He views faculty and staff page…
 He feels uneasy…
Test Environment

 Materials
 2 laptop with the university site already opened
 1 TV or monitor or another laptop connected to the camera
 Task cards that contain task given to the participant
 Camera to document the event
 Questionnaires, that consist of the pre and post questionnaires, screening
and background information questionnaires on separated sheets
 Watch or any timer gadgets
 Script for the test monitor
 1 vacant room
 Chairs

Figure 1 Modified Single Room

The test is to be conducted in a modified single room setting where everything is setup as
illustrated above. The proponents of this test use this type of environment to lessen the isolation
and uneasy feeling of the participant while performing the test. Behavior of the participant will
recorded and viewed from the TV connected to the camera, where data loggers may use to take
down notes of them aside from direct seeing.
Evaluation Measures

Evaluation measures will differ slightly depending on the task in question. In general
however, we will use the following criteria to measure the usability of the [school site] web
environment:

Quantitative Measures
o Number of participants completing each task within the allotted time
o Time taken to complete each task
o Number of general errors (cannot print, link, bookmark, and so on)

Qualitative Measures
o Pre-test, post-scenario, and post-test interviews and/or questionnaires, and
observation
o Verbalizations and “thinking out loud,” including spontaneous comments
o Facial expressions and body language
 SCREENING QUESTIONNAIRE

Name:____________________
Contact No./ E-mail Add.(in case you qualify for the
test):___________________________________________________________
Direction: Please check the boxes for appropriate choices.

Gender:
[ ] Female
[ ] Male
Age category:
[ ] Under 13
[ ] 14-20
[ ] 20 and above
Educational status:
[ ] Junior High School Student and below
[ ] Senior High School Student
[ ] Undergraduate in College
[ ] Going Master or Ph. D.
[ ] Out of School Youth
How much experience do you have with a computer?
[ ] None
[ ] One to four years
[ ] More than four years
How often do you use computer?
[ ] Daily
[ ] Frequently
[ ] Occasionally
[ ] Never or rarely
In a week, how many times do you spend using the web?
[ ] less than once a week
[ ] 2-9 times a week
[ ] 10-20 times a week
[ ] more than 20 times a week
Do you have experience exploring with the website(s)?
[ ] Yes
[ ] No
What computer platform do you usually use?
[ ] Firefox
[ ] Google Chrome
[ ] Internet Explorer
[ ] others, pls. specify_________
Rate yourself when it comes to analyzing and answering evaluative questions?
[ ] Excellent [ ] Highly satisfactory
[ ] Satisfactory [ ] Poor
 ORIENTATION SCRIPT

A pleasant morning to everyone. My name is ________________________________ and

these are my group mates _________________________. We would like you to participate in
this test that will help us evaluate the four different websites we will show you. These websites
are the websites of four known Colleges/Universities in the Philippines, namely: University of
the Philippines Diliman, Ateneo de Manila University Loyola Schools, De La Salle University
Manila and Mindanao State University Iligan Institute of Technology.

We would like you to simply explore the website, click on random buttons and see what
you want to see. Feel free to browse and look around. But while you do so, we want you to make
an assessment. You can freely express of your first impressions about the website and how it was
presented to you as a user. We want to see how the easy the website can be navigated by students.
There are many things that make a website appealing and useful to the user, and we want to see
how these websites makes use of those. Does the website make it easy for you to find the list of
courses you need? Is the website appealing to you at first glance? These are questions that we
want to answer through this evaluation.

We will provide with a laptop where you can browse the websites at the testing room.

Should you have any problems, clarifications and other questions, please don’t hesitate to
ask me. So, do you have questions?

Before we begin with the actual evaluation, I would first like you to answer some
questions related to our test. Please answer honestly.
 Background Information Questionnaire

Participant No.:_________

Question Possible Answers

How much experience do you have with ___None
computer applications? ___One to Four Years
___More than Four Years
What do you prefer in obtaining information? ___Books
___Online(Internet)
___News Papers
___Magazines
Have you ever use the Internet as a source of ___Yes
information? ___No
Have you ever used a website before to get ___Yes
hold of information to particular software or a ___No
website itself?
If yes, how often do you use a website? ___Once a day
___Once a week
___Once a month
___Never
What do you usually do on the internet? ___email
___find reference materials
___games
___entertainment
___reading news or articles
 Pre-Test Questionnaire

Participant No.:_________

Direction: Please check the boxes for appropriate choices.

Have you visited university websites before in planning for your study?
[ ]Yes [ ]No

If yes, what was your last experience with the site?

[ ]Very Good [ ]Good [ ]Terrible

Have you visited these websites? Check the box of the websites you have visited.

Mindanao State University – Iligan Institute of Technology ( msuiit.edu.ph )

Ateneo de Manila University – Loyola Schools ( ls.ateneo.edu )
De La Salle University – Taft ( dlsu.edu.ph )
University of the Philippines – Diliman ( upd.edu.ph )
In what part of the university website did you usually visit?
[ ]Students [ ]Faculty & Staff [ ]Academic

[ ]If others, please specify: ______________________________

Have you ever used a website before to get hold of information to particular software or a
website itself?

[ ] Yes [ ]No

Please rate yourself on your knowledge of the university website.

[ ]Never Used Before

[ ]New User (Used between 0 to 3 mons.)

[ ]Proficient User(Used between 4 to 6 mons.)

[ ]Expert User (Used between 7 to 12 mons.)

 Debriefing Questions

Direction: Please fill in the blanks appropriately.

 Were all the information in the page related to the University itself?

________________________________________________________________________

 Was it easy to get back to the home page after you went to other pages?

_______________________________________________________________________

 What are the difficulties you had on the website?

________________________________________________________________________

 Upon committing errors, did the website give informative feedback?

________________________________________________________________________

 Was there something missing about the university website that you were expecting
to see, but didn’t? Explain if yes.

________________________________________________________________________

 How easy was it to find the information you were looking for? Ex. course list.

_______________________________________________________________________

 Is the website easy to read (both font style and size)?

 ________________________________________________________________________

 Did the website appeal to you? How so?

________________________________________________________________________

 How intuitive and helpful is the navigation system?

________________________________________________________________________

 Did the website serve its purpose for potential enrollees and other visitors of the site?

________________________________________________________________________

 What did you like best on the website? What about the least?

________________________________________________________________________

 On a scale of 1-10, how will you rate the overall effectiveness of the website?