Sunteți pe pagina 1din 21

Milestone 4

Tempus Fugit

Milestone Four
Tempus Fugit
1 May 2011

Casey Gordon - Austin New - Paul Prae - Hal Tift

Table of Contents
INTRODUCTION ........................................................................................................................................................................ 2 USABILITY TESTING PART 1: A GUIDED JOURNEY ............................................................................................................ 3 Testing Demographics and Process .................................................................................................................................3 Reasoning .....................................................................................................................................................................................3 User Tasks (As Described to Test Participants): ........................................................................................................4 Error Counts and Timed Tasks ...........................................................................................................................................6 Usability Testing Analysis .....................................................................................................................................................7 USABILITY TESTING PART 2: EXPLORATORY SURVEYS .................................................................................................... 8 Execution ......................................................................................................................................................................................8 Errors in Execution ..................................................................................................................................................................8 Exploration Survey Analysis ................................................................................................................................................9 EXPERT EVALUATIONS ........................................................................................................................................................... 9 Expert review #1 Cognitive Walkthrough and Heuristic Analysis.............................................................. 11 Tempus Fugit analysis of expert review #1 ............................................................................................................... 14 Expert review #2 Usability Review ............................................................................................................................ 17 Tempus Fugit analysis of expert review #2 ............................................................................................................... 18 General Summary and Reflections of Expert Reviews .......................................................................................... 19 GENERAL PROJECT SUMMARY AND REFLECTIONS WHERE DO WE GO FROM HERE?........................................... 20 Platform confusion................................................................................................................................................................ 20 Dialog/functionality consistency ................................................................................................................................... 20 Overall ......................................................................................................................................................................................... 21

Milestone 4

Tempus Fugit

Introduction
We have now completed the first evaluation of our first prototype. This completes the first iteration of our interaction-design lifecycle. The results, findings, and observations that will be presented in this report will guide the next iteration. These results will help clarify our current design choices and lead us to new ones. We will first use the results from our evaluation to reestablish and fine-tune our user requirements and needs. The cycle will then continue with a redesign phase followed by the next interactive prototype. This evaluation was designed to be as complete and thorough as was possible given our resources and time. We had one and a half weeks to enact the evaluation plan that was presented in our last report. During this time our research team was able to perform usability testing, field studies, and expert evaluations. We were not able to do any of the above in great depth, but these various approaches provided our team a lot of information Our goals for this evaluation were as follows: To discover how our design could change the way people manage their time, goals, and tasks. To help identify the opportunities where our technology can improve current practices. To discover how where our system is inefficient in performing a task. To find out where users of our system make errors. To make sure that we are fully prepared for a redesign of the system. To plan for improvements to the current design based on our findings. To be sure that we are considering the user needs and requirements that have been established earlier in our design life cycle. To be sure that we continue to refine our understanding of our potential users needs and requirements by defining more as we go through our evaluation process. It is important to note that the prototype that we evaluated was brought into the evaluation phase before the development team was ready for any user testing. We had the time to implement a core structure that will act as the foundation for more advanced features in upcoming designs. This core functionality was evaluated so we could decide what features to add, what features to keep, and what features to discard. This core set of features and functionality still had its problems. Production of the last few months occurred at a fast pace. The current prototypes production was cut short due to deadlines. This resulted in a prototype that had many functionality issues. We understand that no software system is ever complete, especially during an early interactive-design iteration, but this prototype was given to users prematurely. The results of our time constraints are obvious after our evaluation results are reviewed. Participants from our many usability testers to our few industry experts reported many obvious flaws and problems with our software. Though our development team was very aware of many of the issues reported, this evaluation also provided our team with a wealth of new knowledge to apply to our future efforts. We begin our exploration of our current interactive prototype through the eyes of several users. This report will begin its presentation of our evaluation via the approach of usability testing.

Milestone 4

Tempus Fugit

Usability Testing Part 1: A Guided Journey


Testing Demographics and Process
Due in part to limited time, the demographic spectrum for our usability testing is not as broad as we would ideally have. Most noteworthy is the fact that eight of the nine participants were female. The ages range from 20 to 60+, however. In future iterations we will make an effort to broaden the demographic pool of usability test participants. We administered a Likert scale survey to nine participants after having them use our application to perform specific tasks. The survey was administered online, and the the majority of testing participants responded electronically from remote locations. For this reason, we did not have direct observation over a substantial portion of the usability testing. We therefore made an effort to be especially specific in laying out the tasks that each participant should perform before taking the survey. Our intent was to have these tasks, in total, be exhaustive of the applications functionality. In cases when we could not physically be present for testing, we spoke on the phone with participants before, after or sometimes even during the process. Participants would inform us of errors they encountered while performing tasks. Ideally we would be there and keep thorough records of errors and durations for every participant. Given the relatively early stage of our design, however, we can learn a lot from the data we received in this survey from the Likert responses and from the error and time records that we did collect.

Reasoning
We considered usability testing to be a vital stage of our testing phase because we knew that at its current stage, some functionality of the prototype may not be evident to all users. We therefore made it a priority to guide users through each function of the system, ensuring that we would gain important user feedback for every aspect of the system. The tasks we chose for usability testing were meant to guide each user through the entire functionality of the application. We went to our prototype and explored it thoroughly ourselves, taking note of all potential tasks. We then formed a succinct list of tasks that was meant to be as short and quick as possible for users while also providing a complete experience of the application. We had participants perform tasks that illustrated most of the standard calendar application, including such tasks as adding a normal, timed event or switching between the daily and monthly views. More important for our application was the testing of features related to untimed events. We had users add a specific untimed event (they were instructed to add the same pre-decided event). They were then asked to add specific descriptions, to edit the events and to drag the events into

Milestone 4

Tempus Fugit

the calendar view. Our goal was to have each user interact with untimed events in every way that we allow in our prototype. We are mostly satisfied that we achieved this goal in hindsight, although the task of deleting an event was overlooked. The survey was intended to provide insight into the user experience of each participant for each task. We looked at each task performed and posed questions to help indicate users experience. The Likert scale was used throughout the survey because of its effectiveness in illustrating participants experience on a measurable spectrum. Given the current state of our prototype, which is still in a somewhat primitive stage of development, there were also several questions posed to get a better understanding of whether or not our application was effective in theory, assuming that functionality was more complete (i.e. the general appeal of having untimed events integrated with a calendar). While we asked many such questions of potential users during our initial research phase, we believe the added advantage of seeing the applications layout would serve to give these participants a greater understanding of our vision, enabling them to give more precise responses than research participants. Below is the full list of tasks performed by each participant (as discussed in this section).

User Tasks (As Described to Test Participants):


Switch from the daily view to the monthly view of the calendar and vice versa. Add a new goal into the calendar view at a specific time by clicking on the timeline: Enter a title of Yoga Enter a remark of Dont forget workout clothes Enter a location of Yoga World Enter a time and date one hour from right now Save it by clicking Save Add a new goal into the calendar view by clicking on the add new goal button: Enter a title of Watch Dexter Enter a remark of Season finale Enter a location of Home Enter a time and date two hours from right now Save it by clicking Save Add a new goal onto the Some day, Maybe list: Give it a title of Play guitar Give it remark of Practice scales Give it location of Home Save it by clicking Save Edit the Yoga event from the calendar view: Change title from Yoga to Meeting with John

Milestone 4

Tempus Fugit

Change remark to Reschedule Yoga Change location from Yoga World to Work

Edit the Play Guitar goal from the Some day, Maybe list. Change remark to Practice bar chords Transfer the Play Guitar goal from the Some day, Maybe list to the calendar view: Set start time for 3 hours for now and end time for 4 hours from now The next section is the complete survey we gave to the participants and the averaged response to each question.

Likert Scale Survey


We stated in the questionnaire that for the purpose of our application and this study, untimed events are events that have no predetermined schedule (i.e. playing guitar), and timed events are events that do have a predetermined schedule (i.e. a meeting or television show). Likert Scale Questions, from 1 (Agree) to 6 (Disagree). Average Response (1-6) It is clear that the left column of the application contains untimed events, and the right view is a standard calendar view. Navigation buttons along the top of the application (the "New Goal" buttons, the Day/Month View buttons, etc.) are organized clearly. Adding the timed event "Yoga" was simple and straightforward. Editing the timed event "Yoga" was simple and straightforward. Adding the untimed event "Playing Guitar" was simple and straightforward. Editing the untimed event "Playing Guitar" was simple and straightforward. Adding the "Playing Guitar" event to the calendar view by dragging and dropping was simple and straightforward. The drag and drop style of moving an event into the calendar is natural for me. Exploring features by trial and error is easy for me. The functionality of this application is apparent without explicit guidance. This application would benefit from supplemental help/documentation. The application is aesthetically pleasing. The ability to integrate untimed events into my calendar is appealing. 3.25 3.44 4.89 5.00 3.22 3.25 2.13 5.00 3.44 3.67 4.67 3.33 4.11

Milestone 4

Tempus Fugit

I would like it if the list of untimed events (left side of the application) were automatically prioritized. Using this system to add untimed events to my schedule would enable me to accomplish tasks more efficiently. Overall, I am satisfied with the ease of completing tasks in this application.

4.00 3.44 2.89

Error Counts and Timed Tasks


We were only able to track times and error counts for three of our nine participants. Despite the small sample size, we believe that the results were meaningful. For our records we averaged the times and error counts of each participant. Tasks were timed with a stop watch; It was started after a participant acknowledged that s/he understood the step and stopped on the steps completion. The errors were calculated by counting the number of incorrect actions taken before the correct task was accomplished. Correct actions taken that failed because of a previous incorrect action were disregarded (e.g. if a participant clicked the wrong Add New button and continued with the next steps, these next steps were not counted). Results:
Task Add a new goal into the calendar view at a specific time by clicking on the timeline: - Enter a remark of Dont forget workout clothes - Enter a location of Yoga World - Enter a time and date one hour from right now - Save it by clicking Save Add a new goal into the calendar view by clicking on the add new goal button: - Enter a title of Watch Dexter - Enter a remark of Season finale - Enter a location of Home - Enter a time and date two hours from right now - Save it by clicking Save Add a new goal onto the Some day, Maybe list: - Give it a title of Play guitar Actions Time Errors (sec) 2 37 11 10 2 7 13 14 5 8 1 2 12 1 8 2 26 1 31 3 3 1 12 1 1 5 2 1 6 1 3

Milestone 4

Tempus Fugit

- Give it remark of Practice scales - Give it location of Home - Save it by clicking Save Edit the Yoga event from the calendar view: - Change title from Yoga to Meeting with John - Change remark to Reschedule Yoga - Change location from Yoga World to Work Edit the Play Guitar goal from the Some day, Maybe list: - Change remark to Practice bar chords Transfer the Play Guitar goal from the Some day, Maybe list to the calendar view: - Set start time for three hours for now and end time for four hours from now. Switch from the daily view to the monthly view of the calendar:

16 5 1 3 20 16 6 4 20 16 12 1

4 3 1 5 7 6 2 7 7 112 20 1 15 3 2

Usability Testing Analysis


In all, the survey results indicate that there is still much work to be done in future design iterations. The lowest score went to perhaps the most important functionality that we set out to implement with this design: the ability to drag and drop untimed events into the calendar view, which also caused by far the most errors with 15 and took polled users an enormous average of almost two minutes to complete. Other questions concerning untimed events were also among the lowest scores in simplicity and straightforwardness, including adding a new untimed event (3.22 out of 6) or editing an existing one (3.25 out of 6). Even more, the general layout of the application was apparently unclear to some participants (i.e. it was unclear that the Someday, Maybe list contained untimed events). One priority moving forward will be to clarify the layout, perhaps with different labels or brighter colors, which would go hand in hand with the additional need to improve the applications aesthetics (as indicated by another survey question). However, despite the fact that some participants rated certain aspects of the application poorly, we are encouraged by other survey responses and are convinced that the application is headed in the right direction for future design iterations. For instance, the survey results indicate that much of the standard calendar functionality is quite usable, allowing us to focus more on perfecting the

Milestone 4

Tempus Fugit

interaction with untimed events from this point forward. Additionally, participants gave positive responses to questions concerning their interest in an application that allowed management of untimed events, including an average score of 4.11 out of 6 in the overall appeal of such functionality and a 4.0 out of 6 in the appeal of an autoprioritizing list of untimed events (a feature that will be implemented in coming iterations). Participants also answered consistently that the drag-and-drop functionality, while not hugely effective in our current prototype, is a natural style of interaction. While there is no denying that we are several iterations away from a polished product, the usability testing has highlighted our most glaring errors while at the same time giving us reason to believe that this is still a product to invest in. Because of these results, we are confident that our system, with tweaks, will be satisfactory and appealing to users.

Usability Testing Part 2: Exploratory Surveys


Execution
Four test users were interviewed and surveyed to provide a qualitative angle to our testing phase. While the sample size was small, the nature of the research in this field study was enough to provide valuable insight, with participants having the chance to experiment with the prototype without explicit guidance and relate to us what functionality was evident or not. The demographics of this portion of the study were very limited. All four participants were women, ages 20, 27, 43 and 47. All were white and from the southeastern US. Participants were told to explore this application by themselves without any outside help for at least 10 minutes, at which point they could continue exploring or say they had finished. Only one participant took more time after the 10 minute period. They explored the interface with no mouse unless they asked for one, all participants but one asked for a mouse. This was unintentional but became pronounced in the testing process. Participants were all given the same yellow pad of paper and pen to respond with. The first command, Draw a map of the applications functionality as you see it, was meant to force the participant to think about our application in visual terms beyond the application itself. This question was posed first so as to avoid the influence of other questions. Other questions were general in nature and addressed their perceived uses of the application and their perceived flaws of the application.

Errors in Execution

Milestone 4

Tempus Fugit

There originally was no mouse to test the participants with (on a laptop). They were given a mouse later. Additionally participants where tested in familiar environments, but they were not in consistent environments or in environments of their choosing. Some participants became stuck on phrasings like draw a map, resorting list of activities, and vital functions. Some participants were friends and family, while they were not explicitly told that this was our academic project, there is still bias in their demographic similarity.

Exploration Survey Analysis


Some users figured out the drag functionality while others did not, which is a high priority for the next design phase. Several noted problems included a dislike of military times and the fact that the interface was all but impossible to navigate without a mouse. The category labels in the Someday, Maybe list were confusing, and the inability to use them was a marked reason for confusion. A related issue, the color-coding functionality, was confusing but liked. The colors were deemed as the most aesthetically pleasing aspect of the application, but they had no evident meaning (our intention is for the colors to act as visual tags, but that functionality is incomplete). Participants analysis of the Someday Maybe events varied from thinking they were useful and possibly motivating to thinking they distracted from the point of a calendar.

Expert Evaluations
Tempus Fugit invited four usability experts to review our prototype. We received two responses. Here are the instructions we sent them:
Subject: Invitation to review calendar application Tempus Fugit Dear expert reviewer, Thanks for helping us with this review of our prototype calendar application, Tempus Fugit. Tempus Fugit is an early stage prototype. It was envisioned as adding functionality to standard calendar applications, such as Google calendar. Calendars handle timed events and deadlines well. Tempus Fugit would add the ability to handle untimed goals and events and would extend "events" and goals to include additional location and context functionality. Think Google Tasks but more flexible and coordinated with your calendar. Tempus Fugit can be found at: http://www.haltift.com/tempusfugit/ This is a heuristic analysis and we have been advised by our professor to simply use Jakob Nielsen's 10 web usability heuristics,

Milestone 4

Tempus Fugit

http://www.useit.com/papers/heuristic/heuristic_list.html. For your convenience, I have listed the ten guidelines below: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Provide suitable help and documentation

Because time is short and we don't want to waste your generosity, we are advising you in advance about two usability problems. 1. Activities done in the calendar area, creating events & goals, editing events etc. are done in a dialog box which closes automatically when the events are saved. The dialog box for editing goals does not close automatically; you must click "Save" and then click "Close". 2. The shortcut/expert actions in the goals area such as cloning, sorting, and deleting are prototyped for look and feel only. Their changes are not permanent and will revert with a page refresh. Editing or creating a goal does actually work as expected. Feel free to cite these shortcomings, we just didn't want to waste your time puzzling these problems out. Some typical tasks we have asked another testing group to perform are: 1. Create a new event by clicking on the calendar timeline. Add the title "Walk the dog" to the description. 2. Create a new goal by clicking on the add new goal button. Add the title "Read War & Peace" to the description. 3. Edit the goal. Change the title to "Read War & Peace". 4. Sort the goal to a new category. 5. Schedule the goal by dragging to the calendar. 6. Delete the goal from the calendar.

10

Milestone 4

Tempus Fugit

Finally, for our documentation, please include a short, one or two sentence biography, listing your experience with information technology, usability, and design. You may send us your review by simply replying to this email. All team members will receive it. Again, thank you for helping us with this project! The Tempus Fugit team

Expert review #1 Cognitive Walkthrough and Heuristic Analysis


Conducted 4/28/11, 1:30 PM -3:00 PM by Dr. Linda S. Gilbert Our first reviewer was Dr. Linda S. Gilbert. Dr. Gilbert received her doctorate in Instructional Technology from the University of Georgia in 1999, where she focused on performance support systems for intellectual work. She is currently the Director of Learning Technology Research at Georgia Gwinnett College. Dr. Gilbert comments, In her personal time, she tries to create and manage her own list of someday goals, and would love to see a later version of this application. Here is Dr. Gilberts review, quoted below:

Review Conditions - Dell PC, running Windows 7 - Browser: Firefox 3.6.12 o Javascript enabled o Pop-up blocker enabled o Default privacy/history settings (accepts cookies, remembers history for 90 days) - NOTE: Initially attempted review using Internet Explorer 8, and had functionality issues. Contacted Casey Gordon, and learned that this prototype should be reviewed using Firefox only. Heuristics Used Jakob Nielsens Ten Usability Heuristics http://www.useit.com/papers/heuristic/heuristic_list.html 1. Visibility of system status 2. Match between system and the real world 3. User control and freedom 4. Consistency and standards 5. Error prevention
11

Milestone 4

Tempus Fugit

6. 7. 8. 9. 10.

Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Provide suitable help and documentation

Task Descriptions In the task descriptions, yellow highlighted text represents issues, blue indicates strengths. Numbers in brackets refer to the usability heuristics listed above. Additional comments also included in brackets. 1. Create a new event by clicking on the calendar timeline. Add the title "Walk the dog" to the description. a. ACTION: For all instances, I clicked on the timeline to create an event. i. RESULT: In each case, the system presented an unnamed [4] dialog box in which I could create the event. 1. ACTION: Typed name in event. Selected Create Event a. RESULT: Event created as expected 2. ACTION (New event): Typed name in event, then selected Edit Event Details a. Edit Event dialog open. b. Subject line blanked.[5 though more of a functionality issue] Had to re-enter Walk the dog c. All date/time information also blanked, despite being created on timeline. [5 again, more functionality] d. ACTION: I did not re-enter all fields. Pressed Save i. Field required messages were presented in red for date/time fields. [2, 9 not clear what would happen if I continued.] ii. ACTION: Pressed Close 1. Event was not created. No error message. [5, 9] 2. Create a new goal by clicking on the add new goal button. Add the title "Read War & Peace" to the description. a. ACTION: Scanned for new goal button. Decided New Someday Goal must be intended. Selected New Someday Goal. [Issue with instructions, not program.] i. RESULT: System presented Create new Someday Goal dialog box [1] 1. ACTION: Entered subject War and Peace and entered text in remarks. No location entered. Pressed Save.

12

Milestone 4

Tempus Fugit

a. Dialog box informed me event added successfully [4 goal, not event] b. ACTION: Pressed Okay i. Goal (event?) dialog disappeared. ii. Goal appeared, listed under HOME (??) [1,2 where does a goal go by default? Do the categories on the left correspond to locations? If not, how DO they relate?] 3. Edit the goal. Change the title to "Read War & Peace". a. ACTION: Attempted to open goal by double-clicking. i. RESULT. None. [7, 4 however, this was NOT on your task list ] b. ACTION: After examining screen and instructions, clicked on E to open goal i. Unnamed [4 confusing that some are named, some not] dialog box appeared, with fields identical to Create new Someday Goal dialog box. Text as previously entered.[1] 1. ACTION: Added location Work (since title was already Read War and Peace) . Pressed Close. a. Closed. No dialog/error message. [1, 5] b. Changes were not saved, on inspection. 2. ACTION (re-opened): Added location Work (since title was already Read War and Peace) . Pressed Save. a. Dialog box informed me Calendar updated successfully [2 this is a goal] i. ACTION: Pressed okay ii. Dialog box closed b. Changes WERE saved, on inspection c. Goal still located under Home on goal list, despite location being Work. [1,2. 9, 10 DO location and categories relate?] 4. Sort the goal to a new category. a. ACTION: Sorted this goal (and others) by dragging i. RESULT: Goals sometimes dragged seamlessly, but sometimes (1) opened dialog box for editing goal [1,2,9] or (2) duplicated instead of moving. [1,2,9] All goals did re-position as desired. 5. Schedule the goal by dragging to the calendar a. ACTION: Dragged goal to calendar i. RESULTS: First few attempts had no effect. About the 3rd try, a dialog box opened with the prior Subject, Location, and Remark of the goal, plus the time and date of the calendar location entered. [1, 2, 6]

13

Milestone 4

Tempus Fugit

1. NOTE: Unable to make War and Peace into a calendar event [3?] (perhaps because I had played with duplicating/deleting beforehand), so used See Eiffel Tower. ii. ACTION: Pressed save. 1. Dialog box informed me Calendar updated successfully [1,2 helpful, but wondering if you should also alert that there is now no goal, just the event from it.] 2. ACTION: Pressed okay a. Dialog box closed b. New calendar entry created i. OBSERVATION: Entry in green, a different color from Walk the Dog, which was purple. It was also wider than Walk the Dog. [4,8 are the color/size changes meaningful?] c. Goal no longer in list [2 this is what I actually expected would happen, but wonder if everyone would.] 6. Delete the goal from the calendar. a. ACTION: Clicked on calendar entry created from goal, and pressed Delete in dialog box. i. Goal deleted. [1,2,3] Additional Issues Noticed - Time on calendar in military time. On first entry, I selected 3:00, intending 3:00PM. I had created the event from calendar, but edited it on initial creation; the default time on the pick list started at 2:30 (AM), which was close to the current time of 1:30 (PM). I thought it was selecting a time close to now. [2,4] - Also duplicated a goal, by clicking on D and dragging, as screen suggested o Attempted to edit duplicated goal. It opened the edit goal dialog box as a new tab in Firefox instead of in a dialog box. o Occasionally had difficulty in making a goal duplicate. o Observed that duplicated goals did not seem to function as original goals - Wanted to move calendar entry back to goal list, but unable to do so. [3] This is a function I would like to have as a user.

Tempus Fugit analysis of expert review #1


General Summary
Heres a count of the number & types of usability issues: Heuristic #01: 11 issues Heuristic #02: 12 issues Heuristic #03: 3 issues

14

Milestone 4 Heuristic #04: 6 issues Heuristic #05: 4 issues Heuristic #06: 1 issue Heuristic #07: 1 issue Heuristic #08: 1 issue Heuristic #09: 5 issues Heuristic #10: 1 issue

Tempus Fugit

Clearly, the majority of issues in this evaluation involved visibility of system status, match between system and the real world, consistency and standards, error prevention, and helping users recognize, diagnose, and recover from errors. Some usability issues arose from server functionality problems. Our app shares services on an underpowered server with very slow response times, occasionally failing completely. Users perceived an error or lack of response but a more powerful server would eliminate these apparent problems. In a significant fraction of the issues calendar and goal objects didnt function in the same way. Certain calendar functionality is missing from the goals list and certain goal functions dont interoperate correctly with the calendar. Further plans for integration will be summarized below. Also, time limitations prevented a more thorough review of interface language basic functionality was a higher priority. In the next iteration, dialog box titles and instructions will be corrected and synchronized.

Analysis of Specific issues and strengths 1.a.i, 3.b.i Issue: the system presented an unnamed [4] dialog box, i. Unnamed [4 confusing that some are named, some not] Causes/Comments: Dialog boxes should be clearly labeled. This is a logic error in the program. Possible solution: Correct programming to properly title dialog boxes. 1.a.i.2.b, 1.a.i.2.c Issue: Subject line blanked, All date/time information also blanked Causes/Comments: First dialog box should pass this information to reduce user load. Programming error. Possible solution: Correct programming to properly pass information between dialog boxes. 1.a.i.2.d.i, 1.a.i.2.d.ii.1 Issue: Field required messages were presented in red for date/time fields, Event was not created. No error message

15

Milestone 4

Tempus Fugit

Causes/Comments: This is a standard response for required date fields. Real problem is user having to re-enter time and date data. Possible solution: Correct issue 1.a.i.2.b, 1.a.i.2.c. 2.a.i.1.a Issue: Dialog box informed me event added successfully [4 goal, not event] Causes/Comments: Dialog box language error. Possible solution: Correct dialog box title. 2.a.i.1.b.2 - see 3.b.i.2.c Issue: Goal appeared, listed under HOME (??) [1,2 where does a goal go by default? Do the categories on the left correspond to locations] Causes/Comments: This functionality not yet implemented. Should respond as tester expected. Possible solution: Implement full goal functionality for goal location, context/categories and sorting next development cycle. Test new functionality for usability. 3.a.i Issue: Attempted to open goal by double-clicking. RESULT. None Causes/Comments: Not currently implemented functionality. Possible solution: Investigate desirability of adding function. 3.b.i.2.c - see 2.a.i.1.b.2 Issue: Goal still located under Home on goal list, despite location being Work. [1,2. 9, 10 DO location and categories relate?] Causes/Comments: This was a recurring and significant problem with testing. Functionality not yet implemented. In spite of email warning, testers strongly expected this behavior. Possible solution: Implement full goal functionality for goal location, context/categories & sorting next development cycle. Test new functionality for usability. This is important. 5.a.i Issue: Dragged goal to calendar. RESULTS: First few attempts had no effect Causes/Comments: Limitations of current web server.. Possible solution: Move prototype to more powerful server. 5.a.i.1 Issue: Unable to make War and Peace into a calendar event [3?] (had played with duplicating/deleting beforehand)

16

Milestone 4

Tempus Fugit

Causes / Comments: This goal functionality not yet implemented. Should respond as tester expected. Possible solution: Implement full goal functionality for goal location, context, sorting, and editing. 5.a.ii, 2.b.i Issue: Dragged Entry in green, a different color from Walk the Dog, which was purple. It was also wider than Walk the Dog. [4,8 are the color/size changes meaningful?] Causes/Comments: Part of the goal category functionality. Not yet implemented. Possible solution: Implement full goal functionality for goal location, context and categories next development. Additional Issues Noticed Issue: Time on calendar in military time. Causes/Comments: Allowing 12hr time format is a usability improvement. Possible solution: Add configuration choice in program options.

Issue: Observed that duplicated goals did not seem to function as original goals Causes/Comments: Again, full goal functionality is not yet implemented. Possible solution: Implement full goal functionality for next development cycle. Test new functionality for usability. This is important.

Issue : Wanted to move calendar entry back to goal list, but unable to do so. [3] This is a function I would like to have as a user. Causes / Comments: This is a desirable and useful functionality that is not yet implemented. Supports user control, exploration, and reversibility heuristics. Possible solution: Implement drag and drop from calendar to goal list as well as goals to calendar.

Expert review #2 Usability Review


Conducted 4/28/11, 2:56 PM by Jimmy Hansen Our second reviewer was Jimmy Hansen. Jimmy has no formal or academic training in HCI or usability issues. However, he is the Principal Web Developer for the College of Family and Consumer Sciences and has been a professional web developer for more than 10 years. His

17

Milestone 4

Tempus Fugit

response was considerably less rigorous than Dr. Gilberts. We sent Jimmy the request for a heuristic analysis and received this response. Here is Jimmy Hansens review, quoted below: Hey Casey, I spent a few minutes with this and found a few things that could make it easier to use: 1. Initial reaction was to maximize window. Would be nice if contents scaled. 2. Aesthetic improvement would be to add margin to left and right to "Create event" button 3. In Month view, the ability to click and drag an event would be nice, e.g., near the top would indicate an early time and the bottom would indicate a later time. Or able to move events before/after other events. 4. In Month view, when adding an event, it could default to one hour in the location where the user clicks, e.g., at the top of the day it could indicate early, near the bottom, late. An area of the page could indicate the hour the mouse is hovering over. 5. When entering an event, carriage return would be convenient instead of clicking "Create event" or tab + space (after typing tab to focus on the "Create event" button, I was unable to get focus back to the What: text box) 6. Clicking on event title produced output: [0]... [10] with 1, 0, -1, and day/time stamps. Regular users will not understand this. 7. After adding an event, it was added before a multiple-day spanned event. Clicking and dragging it to other days placed it below the existing event, however dragging it back to the original day placed it before, making its placement logic inconsistent. 8. Editing an event has "Save (S)" and "Delete (D)". I'm not sure how to enter that keyboard shortcut that it seems to be implying.

Tempus Fugit analysis of expert review #2


1. Issue: Would be nice if contents scaled. Causes / Comments: Our initial designs were for a mobile interface, since location & context are intended as major features. When development in a mobile emulator ran into problems, a web interface was used for development and was designed to resemble a mobile interface as much as was feasible. The calendar web page was intentionally reduced in size, with scaling disabled, to make the app look more like mobile. The small page size helped our team experiment with design solutions, but it confused to testers who were viewing as a web page and expected it to act as such. Possible solution: In the next development cycle, visual design of the two interfaces should be clearly separated and maximized for each intended platform.

18

Milestone 4 2. Issue : add margin to left and right Create event button Causes / Comments: Minor aesthetic improvement. Possible solution: Add in next development cycle.

Tempus Fugit

3,4. Issue: ability to drag an event near the top to set an early time or near the bottom to set a later time. Causes / Comments: These two suggestions arent part of our design specs. They could add expert/shortcut usability. However more fundamental design problems exist (see summary below). Possible solution: These suggestions will have to be reviewed in the perspective of a future design. 5. Issue: When entering an event, carriage return would be convenient instead of clicking Create event" Causes / Comments: This is standard form functionality which users will expect. Possible solution: Will be implemented in next cycle. 6. Issue: Clicking on event title produced output: Causes / Comments: This is a debugging tool that was inadvertently left enabled during testing. Possible solution: Will be disabled in next testing cycle. 7. Issue: After adding an event making its placement logic inconsistent. Causes / Comments: This is a display logic error. Possible solution: This will be corrected in the next prototype. 8. Issue: Editing an event has "Save (S)" and "Delete (D)". I'm not sure how to enter that keyboard Causes / Comments: Dialog language error. Keyboard shortcuts are not intended as a feature for this or the next development cycle. Possible solution: Dialog language will be edited to remove implications of shortcuts.

General Summary and Reflections of Expert Reviews


The most frequent issues mentioned by the experts involved functionality that was not yet implemented for the goals. Despite our warning that The shortcut/expert actions in the goals area such as cloning, sorting, and deleting are prototyped for look & feel only. Their changes aren't permanent and will revert with a page refresh., testers expected everything to work correctly. The functionality that is most important to implement according to the experts are as follows: 1). Dialog boxes should be clearly and consistently labeled. 2). Drag goals back from the calendar into the goal list.

19

Milestone 4

Tempus Fugit

3). Newly created goals should display within the Location or category entered from the creation dialog box. 4) Color should have meaning related to category. 5). To be able to edit cloned goals. These are all reasonable expectations of useful functions and should be developed for the next prototype

General Project Summary and Reflections Where Do We Go From Here?


Two major, and many minor, issues will be addressed in the next development cycle. The two major issues are 1). Platform confusion and 2). Dialog/functionality consistency.

Platform confusion
Initial designs for this app were for the mobile platform, since location & context are intended as major features. When mobile development ran into problems, our fallback solution for the prototype became a web interface. Because we were required to develop at breakneck pace, we have been guilty of starting from one (relatively unexamined) solution and going forward without reexamining our work. Mobile ideas and solutions have become intermingled with web designs and solutions. In the next development cycle, design for the two platforms should be clearly separated and the mobile interface should become the primary design. We speculate that some design solutions will involve simplifying the interface. Complex shortcuts that work on a web page like tiny buttons, hover menus, and some of the more complex drag & drop operations will be unsuitable for noisy, casual, one handed mobile use. These functions can be moved into the Edit dialog box, for example. Other solutions will be explored.

Dialog/functionality consistency
A fundamental problem with this design has not been specifically mentioned by reviewers, though many of their issues stem from this underlying problem. A cursory walkthrough of some common tasks will show that creating or editing goals and events can follow 2 different pathways that are sometimes conflicting or confusing. If a user clicks on an empty spot on the calendar, a dialog bubble will pop up that displays summary information and a link to an Edit details dialog. Then they are taken to the full Edit dialog. This is modeled directly on Google calendar.

20

Milestone 4

Tempus Fugit

If a user clicks the Create new Event, Create New Goal, or clicks the Edit Goal button, they are taken directly to the full Edit dialog box. This has caused confusion in some participants and caused logic errors and other problems in the prototypes programming. The next development cycle will involve testing user interaction with both edit function paths, with possibly a third, new, design. This testing will establish the appropriate dialog path which will be implemented for all creation/editing functions.

Overall
This evaluation has produced a tremendous amount of valuable information. With these results in mind, we will continue onto the next iteration of the interaction-design lifecycle.

21