Documente Academic
Documente Profesional
Documente Cultură
This manuscript has been reproduced from the microfilm master. UMI films
the text directly from the original or copy submitted. Thus, some thesis and
dissertation copies are in typewriter face, while others may be from any type of
computer printer.
In the unlikely event that the author did not send UMI a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
INSTRUCTION IN MICROBURSTS: THE STUDY
TO ONLINE HELP
by
Jean A. Pratt
of
DOCTOR OF PHILOSOPHY
in
Instructional Technology
M ajor Professor
A.J.,
)r J. Sleven4oulier D r.'tw via HaMey
Committee Member Committee Member
2000
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 9999108
___ __©
UMI
UMI Microform 9999108
Copyright 2001 by Bell & Howell Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
Copyright © Jean A. Pratt 2000
AH Rights Reserved
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
ABSTRACT
to Online Help
by
The purpose o f this research was to define ways in which minimalism, an instructional
methodology designed for print-based training materials, could be effectively modified and applied to
online help systems, which are computer-based and typically informational and referential. Five different
online help systems were designed, each implementing a slightly different implementation o f minimalism.
1. Users require (and prefer) less instructional support to complete and learn software
2. Users who are prompted to interact with the application spend longer amounts o f time learning
tasks, but. once learned, complete tasks faster without instructional support.
3. Users will be more effective and efficient at identifying and correcting errors if they have
4. Users using an online help system that is matched to their learning style will complete
software application procedural skills more quickly and with fewer errors than users using a non-matched
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
First, there was a lack o f overall statistically significant differences among the treatment groups.
However, statistically significant differences were obtained for all groups between the pretest and posttests.
Gains in performance were retained after a one-week delay in assessment. All the implementations o f
minimalism were equivalently effective in teaching subjects how to complete software application
procedural tasks.
Second, time to complete tasks was not an indication o f successful performance. Instead, today’s
target audience possesses enough computer expertise to learn new software application skills at
Third, placement o f error information (either next to the error-prone step(s) or in a distant location)
was not beneficial in reducing the number o f errors subjects committed, increasing the percentage o f errors
they corrected, or reducing the amount o f time they spent recovering from errors.
Fourth, a subject’s preference for a verbal- or visual-learning style is not a predictor of student
performance given a specific implementation o f minimalist online instruction. There was no apparent
pattern o f correlations among time, completeness, and accuracy measures among the different
(208 pages)
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
V
DEDICATION
who oftentimes felt as though the dissertation was more important than they were,
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
vi
ACKNOWLEDGMENTS
Funding for this dissertation research was provided in large part by the Society for Technical
Communication. Near-matching funds were provided by Brendan J. Pratt. Additional funding was provided
by the Women and Gender Research Institute at Utah State University. Major in-kind support was provided
by the Department o f Business Information Systems and Education and the Department o f Instructional
Technology. This research would not have been possible without the above support.
I am indebted to my mentor and dissertation chairperson. Dr. M. David Merrill, who hired me at
the beginning o f my master’s program and then mentored me through the doctoral program. Dr. Merrill,
Special thanks are owed to Dr. Hans van der Meij. the principal co-theorist behind the principles
o f minimalism (which were implemented in this research). Hans, thanks so much for your patient
Special thanks are also owed to Dr. Lloyd Bartholome. who made magic happen when it needed
to. Lloyd, you’re a great friend. I d o n ’t know how you did what you did— thanks from the heart.
Special thanks are not enough to express my gratitude to Kasey Child, who watched hundreds o f
hours o f digitized video and coded nearly one half the data for me. Kasey, you're the greatest!!! I literally
could not have done this without your tireless help. How can I ever repay you?
Special thanks are definitely due to my friend. Dr. Thomas Hardy, whose generous creation o f the
Thanks are definitely due to Tim Slocum and Dan Robertson who were always available to work
through the statistical challenges with me. You guys are the best!! 1 had a blast in SPSS!
1 extend thanks also to my buddies, Rick Cline and Bob Mills, who traversed the path before me,
warned me o f the barriers to avoid, supported me when I encountered barriers o f my own. and were
extremely good sounding boards for new ideas. Thanks, guys, for being such great buds.
Jean A. Pratt
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
vii
CONTENTS
Page
DEDICATION ............................................................................................................................................................... v
CHAPTER
I. INTRODUCTION ......................................................................................................................................... 1
Statement o f Purpose...................................................................................................................................I
R atio n ale.................................................................................................................................................... 1
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
viii
Presession M aterials................................................................................................................................. 43
Experimental M aterials............................................................................................................................44
Selection ............................................................................................................................................ 55
Stratified-Random Assignment to G ro u p s ....................................................................................55
Subject Attrition ............................................................................................................................... 56
AVI P la y e r......................................................................................................................................... 62
Dissertation Data Sheet ...................................................................................................................63
Activity Tasks C h eck list..................................................................................................................63
Help Tasks C h eck list....................................................................................................................... 63
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
ix
V. RESU LTS........................................................................................................................................................67
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
X
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
LIST OF TABLES
Fable Page
10. Descriptive Summary o f Error Commission, Correction, and Recovery: Exp 1 ...............................74
12. Descriptive Summary o f Error Commission. Correction, and Recovery: Exp 2 ...............................76
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without perm ission.
xii
R e p ro d u c e d with perm ission of the copyright owner. Fu rther reproduction prohibited without permission.
xiii
LIST OF FIGURES
Figure Page
18. Inferential procedural topic with "On your own" prom pt......................................................................38
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
xiv
51. Interaction o f online help, verbal learning, and time during p re te s t.................................................... 99
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
XV
52. Interaction o f online help, verbal learning,and accuracy during treatm ent........................................ 99
R e p ro d u c e d with perm ission of the copyright owner. Fu rther reproduction prohibited without permission.
CHAPTER I
INTRODUCTION
Statement o f Purpose
The purpose o f this research was to define ways in which minimalism, an instructional
methodology designed for print-based training materials, could be effectively modified and applied to
online help systems, which are computer-based and typically informational and referential. Two separate
experiments (using the same sample population) were conducted with the following general hypotheses:
1. Users require (and prefer) less instructional support to complete and learn software
2. Users who are prompted to interact with the application spend longer amounts o f time learning
tasks, but. once learned, complete tasks faster without instructional support.
3. Users will be more effective and efficient at identifying and correcting errors if they have
4. Users using an online help system that is matched to their learning style will complete
software application procedural skills more quickly and with fewer errors than subjects using a nonmatched
Rationale
Most computer software is accompanied by documentation, much o f which is placed online. The
trend in software application development to place as much o f the documentation as possible online
(Boggan. Farkas, & Welinske, 1996) is causing a shift in the way we view the role o f technical
communicators (Hailey & Hailey. 1997) and online help (Pratt, 1998). Technical communicators are now
asked to develop training materials (Techwriters, 1997), and the quality o f software applications is judged
by the effectiveness o f the online help system to teach the user how to operate the software (Greer. 1984;
Rosenbaum. 1998). Effective, well-written, helpful documentation strongly influences the perceived
effectiveness o f the software by users (Greer, 1984; Rosenbaum, 1998) to the point that documentation can
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
"make or break a software application” (Greer, 1984, p. 7). Inadequate or poorly written online help results
in user frustration with and possible rejection o f the software (Kearsley, 1985b; Lieberman, 1987) and
possible loss o f business (Rosenbaum, 1998). Specific guidelines are needed to help technical
design approach used to develop training manuals, to the development o f online help systems. Minimalism
was developed by John Carroll and his colleagues at the IBM W atson Research Center as a “ less is more”
approach to addressing problems users encountered when learning a new software application via self-
instruction. Carroll suggested that this “new” approach was in sharp contrast to the traditional instructional
systems design (ISD) approach proposed by Gagne’ (1985) and used by most instructional designers in the
development o f training materials. The main focus o f minimalism is to conduct iterative testing and analysis
(similar to the iterative nature o f formative evaluation in ISD) in order to provide users with “ju st enough"
information to guide their exploration and learning o f the software. The end product is similar to a well-
The implementation o f minimalist principles to online help instruction could result in increased
learning o f the subject matter content and acceptance o f the software application; however, the application
of minimalism to training materials has met with mixed reviews, and its application to online help has not
been evaluated thoroughly. There are a few problems associated with adopting the minimalism principles
First. Carroll intended these principles to be used only as general guidelines, not a prescriptive
"recipe" for “[cranking] out a training manual” (Carroll. 1990). Technical communicators differ in their
The second problem is that minimalism is a model o f instruction applied to training manuals. It
was not designed for use in online help systems, but its “less is more” approach is directly applicable to the
limited availability o f screen space in online help. Microsoft and Lotus are shifting their help systems to
minimalism (Boggan et al., 1996). Some o f the leading authors in online help development (Boggan et al.,
1996; Brockman, 1990b; Farkas, 1993; Hackos & Stevens, 1997) include minimalism in their online help
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
guidelines. Other technical communicators propose a variety o f ways to apply the minimalism principles to
online help (Techwriters. 1997). Logical heuristics seem to be the main guidelines for the transition o f
This brings us to the third, related problem. The limited empirical research conducted on
minimalism used training manuals, not online help. These studies resulted in contradictory findings. Carroll
(1984. 1990) reported case studies indicating support for minimalism. Carroll (1990) and Van der Meij and
Carroll (1995) cited a number o f studies they and others have conducted which have "repeatedly proven the
effectiveness o f minimalist instruction” (p. 243). However, Jansen (1994) discovered that minimalism is not
an etl'ective approach for all types o f learners. The findings from the Nowaczyk and James (1993) study
supported one minimalist principle but refuted another. No empirical support for the application of
Research on the application o f the principles o f minimalism to online help was needed to assist
technical communicators in the design and development o f effective online help instruction. The results o f
this research can be used by technical communicators to develop online help systems: it can also be used by
instructional designers to develop online instruction. This research supports the minimalist approach to
developing online help for subjects who have an intermediate to advanced skill level and who are learning
and performing intermediate- to advanced-level skills. This research does not support the implementation o f
one variation o f minimalist instruction over another. That is, given a solid base o f minimalist instruction,
subjects can leam equivalently despite slight variations in the instruction. These findings can be extended to
the development o f any type o f online instruction: provide users with less information and require them to
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
4
CHAPTER II
The purpose o f this literature review was to provide a foundation for applying the principles o f
minimalism to online help and design a study to test that application. In a deductive manner, the review of
literature begins with the procedures used to locate, include/exclude, and categorize the research for ease o f
use in this study. It then examines a trend in technical communication that is leading practitioners toward a
researched use o f minimalism, then follows with an overview o f and problems with minimalism. It
concludes with an analysis o f the related research on minimalism and suggestions for how this research may
answer questions and/or fill in the gaps identified through previous research.
Since this is a new area o f research, several related fields o f research had to be investigated for
The two primary sources of data for this literature review were the listserves and computer-assisted
searches o f relevant research databases. Utah State University’s online public access catalog (OPAC) was
The archives for two listserves, Techwriters (1997) and WinHelp (1997), were searched for
discussions o f online help and minimalism. Once a discussion thread was located, it was followed until
dropped. Some o f the same people participated in the discussions on both listserves. Although not empirical
data, discussions on the listserves provide timely input on what practitioners (the target audience) are doing
The empirical data for determining what to research and how to conduct the research study were
obtained by searching relevant databases back to 1980. (Carroll’s seminal paper on minimalism was
published in 1 9 8 4 :1 searched back to 1980 to guarantee that no other studies had been conducted prior to
that.) Databases searched included ERIC, USU Libraries— Electronic Journals, Dissertation Abstracts,
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
5
Education Abstract, Social Science, and Applied Science and Technology Abstracts. The following
descriptors svere used alone and in combination to expand and narrow the searches: minimalist, minimalism,
online help, online help, ICAl, expert system, intelligent tutoring, embedded training, coaching, feedback,
tutoring, multimedia, computer-based instruction, computer-based training, adaptive help, adult learning,
Inclusion/Exclusion Criteria
A wide search was used in order to identify studies in related areas that could be used as a guide to
develop the current study since no studies with direct application were located. Studies were included if
they described the use o f instructional features (e.g., examples and feedback via graphics or text) that could
be used with minimalist instruction. Studies were excluded if they (a) referenced library or corporate
information databases without applications to help or training systems, (b) provided general descriptive
information without any specific reference to instructional design features, or (c) required advanced
programming for the implementation o f the instructional design features (e.g.. "intelligent" agents).
Exclusion o f articles based on the above criteria did not bias the study: rather, it focused the study for more
direct application to the target audience: online help developers and instructional designers.
measurement tool used in this research). All articles were included (although only three were even in related
fields o f study) because it was the test— not the subject matter content— that was o f importance. (A brief
Carroll’s (1990), Van der Meij and Carroll’s (1995), and Black, Carroll, and McGuigan’s (1987)
works were the principal sources for setting up the study and creating the five different online help systems
used for the treatments. However, since their works are based on manuals and simulated software, other
included articles were used to aid in the application o f the principles o f minimalism to online help. Such
The remaining theoretical and empirical articles were used to identify the current trend in software
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
6
documentation, previous research conducted on the principles o f minimalism and the related principles, and
possible online (Boggan et al.. 1996). Documentation could include tutorials, reference and training
manuals, links to information databases, web pages, and online help systems. Brockman (1986) suggested
that "manual-less" software would be in common use by the year 2000 either because documentation would
be put "online" or software would become increasingly intuitive. However. Kearsley (1985a. 1985b) points
out that what is intuitive to one may not be intuitive to another and that there are very few software
applications that are so simple and intuitive to use that they do not require some type o f training or
embedded documentation.
Two ramifications o f this trend seem to be emerging. First, technical communicators are taking on
new roles such as trainers and webmasters (Techwriters, 1997). Hailey and Hailey (1997) suggest that
industry demands force technical communicators to “ lead instructional technology into the workplace" (p.
3 1). a role typically assumed by instructional designers. Technical communicators are also becoming
information suppliers by providing access to information and the ability to cooperatively create more
information. Second, there is a blurring in the purposes and uses o f the different forms o f online
documentation. Pratt (1998) suggested that online help could be used to teach new users how to use the
software, an instructional role typically reserved for tutorials and training manuals.
Several leaders in the field o f technical communication have assisted technical communicators in
their transition from producers o f print documents to producers of online documents. Their publications
provide general and specific guidelines for producing online documents as well as specific emphases on the
technical aspects (Boggan et al., 1996), standards (Hackos & Stevens, 1997). evaluation (Duffy. Palmer. &
Mehlenbacher, 1992). and conversion from paper media to digital (Brockman, 1986. 1990b; Horton, 1990,
1994). An implicit assumption in these publications (with the exception o f Duffy et al., 1992) is that the
same guidelines can be applied across the various forms o f online documentation (both informational and
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
7
instructional). This assumption is indicated by the types o f online documentation the authors list as the
target o f their guidelines (Brockman, 1990b; Hackos & Stevens, 1997; Horton, 1994) and by the use o f a
training methodology (minimalism) to guide the development o f referential online documents (Boggan et al.
1996: Brockman, 1990b; Hackos & Stevens, 1997). But what is minimalism?
Minimalism
Minimalism was first introduced in the early 1980s (Carroll, 1984), but there has been a resurgence
in the discussion o f minimalism and its application to documentation. Past discussion on the listserves
(Techwriters. 1997; WinHelp, 1997) leads one to believe that minimalism is new to many, very familiar to a
few. and not well defined, understood or applied by the majority. This is despite the fact that Microsoft has
adopted it as a standard, Mary Deaton used it in a jo b description. JoAnn Hackos presented a one-day
Society o f Technical Communication seminar on it and Stephanie Rosenbaum conducts semiannual, one-
day seminars on it at University' o f California, Santa Cruz. What is minimalism? Why is it important to
technical communication research? How can we apply its principles to online help systems? According to
Carroll (1990):
The key idea in the minimalist approach is to present the smallest possible obstacle to
learners" efforts, to accommodate, even to exploit, the learning strategies that cause
problems for learners using systematic instructional materials. The goal is to let the
learner get more out o f the training experience by providing less overt training structure.
(pp. 77-78)
Minimalism was developed by Carroll and his colleagues at the IBM Watson Research Center as a
"less is more" approach to addressing problems users encountered when learning a new software
application via self-instruction. This "new"’ approach was in contrast to the traditional instructional systems
design (ISD) approach proposed by Gagne’ (1985) and used by m ost instructional designers in the
development o f training materials. Carroll’s impression o f instruction designed using the ISD process is that
it "is designed with little consideration o f the learners and no consideration for the context within which
learning will occur” (Carroll, 1990, p. 74). However, a comparison o f minimalism and the ISD process
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
8
rev eals they both utilize an iterative design based on analysis o f the user needs, learning environment, and
content/tasks to be learned (Horn, 1992). Carroll’s main complaint about systematic design is the tendency
for instructional designers using this approach to break down tasks into minute, detailed components
preceded and followed by lengthy descriptions and conceptual explanations (Carlson, 1992; Hallgren.
1992). Instead, Carroll advocates providing users with just enough information to learn the application as
The number and focus o f the specific principles o f minimalism have changed over the years. The
most recent listing by Van der Meij and Carroll (1995, 1998) condenses minimalism into four principles
and 11 "heuristics";
Provide error information when actions are error prone or when correction is difficult.
The main problem with minimalism is that there are no specific application guidelines for how to
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
9
implement the principles and heuristics. Hallgren (1992) criticized Carroll’s book on minimalism. The
Xurnherg Funnel, for "providing no direct help for me to perform my work” (p. 16). Indeed, Carroll was
There is no deductive theory o f minimalist instruction; that is, given a set o f minimalist
principles, we cannot just crank out a training manual. Design never works in this way.
What is important is the key idea o f minimizing the extent to which instructional materials
obstruct learning and o f refocusing the designing o f training materials on the goal of
Without specific "how to” guidelines, practitioners are left on their own to implement the
minimalist principles. Draper (1996) taught students how to design a "graded series” o f training manuals
using the minimalist principles. Farkas (1993) suggested that balloon help, especially with a "layering” or
"filtering" option for providing different levels o f balloon help for different needs, is an implementation o f
minimalism. Hackos and Stevens (1997) highlighted five "tips" based on the 11 heuristics. Brockman
( 1990b) included adopting the minimalist philosophy (using five tips and four case studies) as part o f the
tirst step in the documentation process. Boggan et al. (1996) integrated what they believe are the applicable
minimalist principals throughout their guidelines. Practitioners on the Techwriters (1997) ListServe suggest
Another problem with minimalism is that it is targeted toward one type o f learner and may not
prov ide the flexibility necessary to reach a spectrum o f learners with varying degrees o f experience.
Minimalism is good for the holistic, independent explorer type o f learner using a simple, noncritical
application, but is not applicable for all types o f learners in all types o f situations (Carlson. 1992; Horn,
1992; Jansen. 1994; Reddish, 1988). Jansen (1994) found that “manual-oriented subjects were often
confused by the deliberate incompleteness o f the instructions and by the absence o f introductory sections.
They missed the step-by-step approach and sometimes felt abandoned” (p. 236). In fact. Carroll reported
that some of his experimental subjects complained that they wanted the structure o f a self-instruction
manual (Carroll, 1990; Reddish, 1988). “Three participants became frustrated enough to ask to be excused
from the experiment and were replaced” (Carroll, 1990, p. 167). In another study. Carroll had to provide his
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
10
experimental subjects with a typical ISD-type o f reference manual to complete the experim ent (Horn,
1992). These findings are supported by other studies o f guided exploration hypermedia that indicate that
users lacking sufficient prerequisite knowledge were frustrated because they could not decide what
information was needed or where to find it (Cline. 1991; Lawless & Brown, 1997).
Brockman (1990a. 1990b) identified other problems with the minimalist approach (which I will
simply summarize here in paragraph format). Minimalism is based on guided exploration, which is a self-
discovery type o f learning. Self-discovery learning is less predictable and may result in shallow learning.
Minimalism assumes a motivated audience that can choose effective and attainable goals. Instructional
design research indicates that this is an erroneous assumption. Users are not motivated to explore the
interface. They need specific directions to complete a task and get back to work. Furthermore, they lack the
prerequisite knowledge about the system and their own learning to be able to set effective learning goals.
There is also a potential for documentation to become cryptic as technical communicators "slash the
verbiage" instead o f rewriting the content. Moreover, technical communicators are uncomfortable slashing
the verbiage.
Rosenbaum's (1998) survey o f technical communicators who had taken a minimalist course
revealed that practitioners were uncomfortable with the minimalist principle to specify information
incompletely. This stemmed in large part from the fact that most o f them had neither the time nor the budget
to implement iterative testing so had to rely on their own heuristics for what information users needed. They
also had difficulty deciding what kind and size o f tasks to modularize, especially for com plex applications.
Some o f them "implemented” minimalism into one medium (e.g., the training manual) by shifting all the
In another study. Draper (1996) identified conflicting purposes o f minimalism: task completion
(job aid) and learning (tutorial). Slashing the verbiage to create a job aid necessarily means eliminating
explanations and other details that learners need in a tutorial. The lack o f detail could prove disastrous to
users who lack the prerequisite knowledge. Another principle that conflicts with “slash the verbiage” is
"provide on-the-spot error information” which, by definition, means providing redundant verbiage because
no one can anticipate where and when the user may make an error. Users spend between 25% and 50% o f
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
II
the time correcting errors (Van der Meij & Carroll, 1995) and need to have the error recovery information
immediately accessible. However, providing access to additional details through hypertext links in online
This review o f the research is comprised o f those studies that were published. A detailed table o f
the studies and their findings is located in Appendix A. Other research has been conducted in the
commercial sector but has either not been reported or has been reported anecdotally.
Black et al. ( 1987) designed four different manuals with varying amounts o f verbiage in each and
then compared the time it took subjects to complete the specified tasks. The manuals they created were
skeletal (brief, task-based commands), elaborative (commands with explanations, feedback and
summaries), inferential (partial commands with "Try this on your own" prompts for self exploration), and
The current study was a modified replication o f the Black et al. study. It differed from the original
study in the following ways: (a) the principles o f minimalism were applied to online help rather than to
training manuals, (b) the rehearsal implementation was dropped, and (c) the content was more advanced.
Online help was the more appropriate medium since so much documentation is being converted that way.
The rehearsal manual was not an appropriate implementation o f online help. The use o f intermediate and
advanced tasks was more appropriate given today's higher level o f experience with word processing
applications.
The tasks the subjects were required to perform in the Black et al. (1987) study were easy to
perform. The simple learning test was to insert a letter or word; the command sequence task was to center or
move text: the realistic task was to create or revise a letter or memo. The authors admitted that they did not
know how well the inferential manual would teach advanced tasks. The tasks the subjects were required to
perform in the current study were rated as intermediate- to advanced-level tasks by the Kelly Temporary
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
12
Sen/ices training outline (Appendix B). They included creating and manipulating text boxes and tables,
inserting and formatting text and graphics, and creating a mail merge document and database.
Black et al. (1987) concluded that the inferential manual, emphasizing guided exploration, was
superior to the other types o f manuals. The results o f this study indicate that subjects using any o f the three
implementations o f minimalism to online help were able to achieve equivalent completion and accuracy
scores for the given tasks. Black et al. also suggested that the study o f alternative manuals for teaching
advanced topics was an area for future research. The results o f this study indicate that minimalism is an
Lazonder (1994) conducted a series o f four tests on the principles o f minimalism. His study is
especially relevant to the current study because he conducted his studies as part o f his dissertation work
under the guidance o f Van der Meij who. with Carroll, refined and published a more definitive set o f
Lazonder's first experiment was a replication o f Carroll. Smith-Kerker. Ford, and Mazur-Rimetz's
(1987) research comparing minimal and traditional training manuals. Overall, his findings supported
minimalism as an effective instructional design strategy for software applications, but only during the
training phase o f the experiment. In fact, the mean task-completion score o f subjects using traditional
manuals was higher than the mean task-completion scores o f subjects using the minimal manual during the
retention and transfer phases o f the experiment, but not significantly so. Lazonder also found that novice
users required more time and were less successful, less efficient, less capable o f recovering from errors, and
Lazonder's second experiment was designed to assess the effectiveness o f error information for
detecting, diagnosing, and correcting errors. He found no statistically significant differences, so he followed
up this study with a qualitative study designed to see if new users made enough errors in learning the
software to warrant further study o f error information. His findings provided support for continued study o f
the integration o f error information in instructional materials: 25% o f users' actions were in error.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
13
The fourth experiment Lazonder conducted was a modified replication o f his second (error
information). He removed error-prone tasks, provided more distinctive headings, faded error information
out over the course o f the manual and placed error information next to steps that had some sort o f visual cue
on the screen. Thirty-five percent o f the minimal manual was error information. He also eliminated the
delayed testing situation and relied only on the training, practice, and immediate performance scores.
Although there were some statistically significant differences in speed and quantity o f detected and
corrected errors during the training and practice phases, there were no statistically significant differences
between manual types during the test. Lazonder’s concluded that overall "subjects from both conditions
were equally skilled at detecting and correcting their own errors” (p. 111).
Lazonder chose to focus his study on novice users performing very basic tasks. Given the prolific
use o f computers and word processing applications, the current study was comprised o f users who already
had a working knowledge o f a word processing application and were interested in learning additional skills
to become more productive in their work. They represent a more typical target audience. The tasks chosen
to meet the needs of this target audience were more complex (e.g.. creating a data source and performing a
mail merge) than those used by Carroll (1990) and Lazonder (1994). The ability o f minimalist online help
systems to teach subjects complex tasks to subjects with intermediate skills addresses the criticism by
Lazonder (1994) and Horn (1992) that minimalism is only an effective strategy to use to teach basic tasks to
novice users.
Testing was one problem with Lazonder’s first experiment: he let the subjects use their training
manuals during the assessment phase. He inadvertently tested the ability to complete the basic tasks using a
minimal manual rather than the ability to learn tasks with it. The current study used an approach sim ilar to
the one used by Palmiter, Elkerton, and Baggett (1 9 9 1): training with instruction followed by immediate
and delayed testing without instruction. Subjects in each o f the different treatm ent groups o f the current
study maintained their level o f task completion and accuracy while decreasing the number o f errors they
Another problem with Lazonder’s study was his inconsistent time o f assessments. There was a 1-
week delay between assessments in Experiment I, a 3-day delay for Experiment 2 and no delayed
R e p ro d u c e d with perm ission of th e copyright owner. F urther reproduction prohibited without permission.
14
assessments for Experiment 4 (Experiment 3 was a qualitative, exploratory test). Immediate performance
tests assess the ability o f users to recall and apply procedural information from short term memory, but do
not assess the more practical issue o f retention beyond the initial training phase (which is more important in
a workplace environment). The current study used a I-week delayed performance assessment. A I-week
delay introduces a possible history effect— subjects could potentially practice the newly learned tasks
during the course o f their everyday work efforts. However, this type o f practice would be typical o f this
target audience and its effects distributed equally across the sample. Moreover, there were no statistically
significant differences in scores between the immediate and delayed posttest to suggest any possible threat
o f history .
Lazonder removed those tasks that were error prone (Carroll blocked them in his use o f the
Training Wheels simulated software) and provided extra error information for the remaining tasks (35% o f
the manual). The current study neither blocked nor removed from the instruction error-prone tasks. Rather,
sub jects were allowed to flounder in frustration (just as they would in the real world) if they made an error.
Subjects were, however, provided with error prevention/recovery information in Experiment 2. Again,
subjects were able to maintain their level o f task completion and accuracy while decreasing the number o f
errors they committed during the posttest and delayed posttest. However, as in Lazonder's (1994) study,
findings from the current research indicated that subjects exposed to minimalist instruction with variations
o f error information will be "equally skilled at detecting and correcting their own errors" (p. III).
Reznich (1993) tested the efFect o f minimalist design principles in reducing computer anxiety for
users learning word processing tasks. He implemented six minimalist principles in a “getting started" type
o f document to teach basic w ord processing skills in W ordPerfect. He found no statistically significant
differences between the experimental and control groups for performance achievement (time and success to
complete tasks) or decrease in com puter anxiety. He concluded that well-designed instruction (regardless o f
the selected instructional design model) could teach new users word processing tasks and that computer
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
15
There were a few problems associated with this study. Reznich (1993) had a 30% differential
mortality rate, ceiling effects o f performance tests, and testing problems— all of which could have impacted
the outcome. All the students who achieved an initial decrease in computer anxiety stayed with the
experiment: the experimental group was administered the very same questionnaire four different times:
almost all the subjects in both groups completed the word processing tasks within the given time. Reznich
In an attempt to avoid those problems in the current study, the following controls were
implemented: (a) to prevent differential mortality, temporary employee subjects were not paid unless they
attended both sessions, (b) to prevent any ceiling effects, subjects were told to complete tasks that had been
identified by expens as moderately difficult, and (c) to prevent subjects from learning the test, the pretest
and posttests were administered only once, with different content for each. Even with these controls, there
was differential mortality in this experiment (12 o f the 22 subjects who left were from the inferential group).
1louever. the inclusion o f their scores would not have changed the results o f the experiment.
Warner (1989) selected two minimalist principles to study: slash the verbiage and force the
coordination o f the system and the training (implemented via Teaming activities). He created four different
training manuals to study high versus low verbiage and activities at end o f lesson versus activities
throughout the lesson. His dependent variables were time to complete tasks, written and performance
achievement o f tasks, attitude towards learning, and level o f anxiety. As he expected, he found that subjects
using the training manuals with low verbiage scored higher on both the written and performance
achievement tests, completed their tasks faster, and had less anxiety. Subjects using the training manuals
with learning activities distributed throughout the lesson scored higher on the performance achievement test
Contrary to his expectations, he found that subjects using the training manuals with high verbiage
and with activities at the end o f the lesson had a more positive attitude toward the learning experience than
did their counterparts using a low-verbiage version with activities distributed throughout the lesson. Those
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
16
using the manuals with the activities at the end o f the lesson also finished almost 12 minutes fa ste r than
Vanderlinden. Cocklin, and McKita (1988) wanted to design more effective tutorials for Hewlett-
Packard's CAD system. They were intrigued by the guided exploration approach Carroll. Mack, Lewis.
Grischowskv. and Robertson (1985) had used to teach word-processing applications. However. HP’s CAD
s\stem was far more complex than a word processing application, so they conducted a case study o f 16
users (62% o f which had previous CAD experience) to determine if the guided exploration would work for
their situation. They were also interested in whether prior computer experience would have an influence on
users' preference for tutorial type (guided exploration versus self study).
To answer these questions, they designed two types o f tutorials: a guided exploration (plus
procedures, when necessary) and a typical self-study tutorial. They found that subjects using the guided
exploration tutorial took 10% longer to work through the tutorial, but completed the post-tutorial test in one
half the time and with far fewer errors than those subjects using the self-study tutorial. In fact, those using
the self-study tutorial had to refer back to the tutorial four times more often than those using the guided
exploration tutorial. Subjects using the guided exploration tutorials reported that the guided exploration
tutorial u'as more vague than what they were accustomed to. They also found that prior computer
Although this was an exploratory case study, their findings lend support to the use o f the guided
exploration approach in the development o f software application documentation. Further, the fact that prior
computer experience made no difference in learning indicated that the current study could eliminate prior
computer experience as a critical variable and focus instead on learning styles (e.g., visual vs. verbal).
However, the results o f the current study were in direct contrast to the Vanderlinden et al. results. Prior
computer experience was statistically significantly correlated with task completion and accuracy, and
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
17
Frese et al. (1988) were also interested in the guided exploration and active learning approaches
espoused by Carroll et al. (1985). They conducted a study comparing the effectiveness o f three different
types of instruction to teach word processing skills: sequential steps to complete unrelated tasks (e.g., copy,
delete, print): sequential steps for tasks that were integrated using a diagram depicting their relationships to
one another and to the entire word processing system plus explanatory elaborations: no written assistance,
but the guidance o f Socratic dialogue by the researcher. In other words, they were comparing steps, steps
with diagrams, and full (but guided) exploration. Overall, they concluded that guided exploration was a
more effective means o f teaching word processing tasks than the use o f unrelated task instruction that is
All but one o f the statistically significant differences they found were between the steps and the
guided exploration. They found that subjects who explored the software interface as the researcher guided
them (via questions) to develop hypotheses and a mental model of the system could use the commands
better in the performance test than those subjects who learned the software by completing instruction on
specific steps. Those who learned the system via exploration could also complete transfer tasks better and in
less time and use fewer keystrokes to complete a task than those who learned via standard instruction. All
the subjects made errors, but those in the guided exploration group and the steps-with-diagram groups made
fewer errors and corrected them faster than those who learned via standard instruction. Subjects in the
guided exploration group also recalled more commands than those in the steps-with-diagrams group in a
free-reeall session on the second day. However, there were no statistically significant differences in recall
There were a few problems with the Frese et al. study that could have accounted for some o f the
outcome. The most significant problem was that the researchers were actively involved in all three
conditions. When the subjects made errors, the researchers would either tell them how to correct the error or
use questions to guide them in the error correction. Frese et al. noticed that subjects o f a different gender
than the researchers performed better. Also, the variation in researcher personality and teaching expertise is
a variable they did not take into account. In the current study, only lab consultants were able to interact with
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
18
the subjects. The lab consultants were, in turn, monitored by the researcher to make sure they provided only
Other empirical support for minimalism comes from studies conducted by Carroll and his group
(Carroll. 1984, 1990; see also Van der Meij & Carroll. 1995. 1998). Carroll found that experimental
subjects using his Minimal Manual completed tasks 40% faster and scored higher on achievement tests (by
a factor o f 10) than subjects using traditional manuals. Although they did not directly reference minimalism.
Rogers and Brown (1993) reported that the use o f minimalist-type written instructions resulted in a greater
quantity o f responses and more timely responses. Wenger and Spyridakis (1993) combined the minimalist
approach with findings from reading research for an increased performance in user reading, comprehension,
and recall. However. Jansen (1994) indicated that minimalism is not an effective approach for all types o f
users. Nowaczyk and James (1993) reported that their findings supported the “slash the verbiage" principle
but that screen captures (recommended by the minimalist approach) interfered with tasks. These studies
addressed the application o f minimalism to training manuals, not online help systems. Minimalism is a
training and instructional methodology. The application o f minimalist principles to online help is a new area
o f research.
The hypertext characteristics o f online help may make it a better medium for the application o f
minimalist principles than a print-based medium. Carroll (1987) stated that he saw “no useful distinction
between help and training” (p. 264). He suggested that the hypertextual nature o f online help made it an
effective media by which to provide different levels o f detail for different users. He focused his research on
intelligent online help but conceded that “only very fragile and limited examples o f intelligent help now
exist and none has been demonstrated to be successful” (p. 267). Microsoft’s w izard is good representation
o f the progress made in intelligent help since Carroll’s assessment, but there is still not yet the technology
and enough knowledge o f different learning behaviors to develop a robust, effective intelligent help system.
R e p ro d u c e d with perm ission of the copyright owner. Fu rther reproduction prohibited without permission.
19
Farkas (1993) suggested that balloon help with "layering” was an application o f minimalism. This
suggestion met with mixed reviews (Price. 1993; Talburt, 1993) but merits further investigation. Layering is
a means o f providing users with a minimum amount (and different types) o f information and access to
vary ing levels o f more detailed information. Providing different layers o f information is a possible solution
to the problem o f using minimalism for users with differing levels o f prerequisite knowledge and different
task needs.
Farkas and his colleagues (Boggan et al.. 1996) proposed the use o f layering as one solution to the
problem o f different learning sty les. Carlson (1992) also sees layering as a modification o f minimalism to
address different types o f learners, referring to it as "intelligent hypertext.” She suggested placing the full
text online but adjusting the delivery according to the circumstances and the user. The interrelationships
among the intelligent nodes could be "sorted and filtered for specific combinations” o f user tasks and needs
(p. 25). Nichols’s (1994) research group developed three types o f online help panels (general, procedural
and conceptual) and two layers o f help (novice and advanced) to accommodate the different levels o f
Layering o f information is based on the hypertext structure and navigation research which is
directly applicable to the "guided exploration” philosophy behind minimalism. There are mixed findings in
regard to the most effective hypertext structure for learning and information retrieval. McKnight. Dillon,
and Richardson (1996) conducted a review o f the literature and found that overall subjects performed better
in a nonlinear environment and were able to adapt the material to their own learning style. The hypermedia
environment encouraged exploration and enabled users to see subtasks as part o f a whole task. Jonassen and
W ang's research (1993) supported these findings for field independent learners. However. Hailey and
Hailey (1997) found that a more linear structure o f the information was more effective. The very nature o f
hypermedia introduced a frustrating level o f confusion to some users. Nichols (1994) found that people do
not yet have a good schema for processing information so it was better to provide all users with structural
guidance. In their opinion paper. Grice and Ridgway (1995) suggested that "minimal information,
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
20
set o f principles for writing documentation. These principles were derived for application to training
manuals as "self instruction” for users learning new software applications. However, many o f the minimalist
principles are directly applicable to online help. Because the application o f minimalist principles to online
help has not been thoroughly investigated, the research on print-based media and hypertext/hypermedia was
evaluated for research guidelines. The resulting research was based, in part, on the above studies.
Because several different researchers (Carlson. 1992; Horn. 1992; Jansen, 1994; Reddish. 1988)
suggested that minimalism was not applicable for all types o f learners in all types o f situations, it seemed
prudent to incorporate at least a simple measure o f learning style into this research. The tool selected for use
in this research was the Verbalizer-Visualizer Questionnaire (VVQ). This 20-question assessment tool was
designed by Richardson (1977 as cited in Kirby. Moore, & Schoefield. 1988) and later modified by Kirby et
al. ( 1988). Their tests o f the modified VVQ showed that the two learning constructs (visual and verbal)
possessed good construct validity in the principal component analysis and adequate reliability (the alpha
coefficients). They also found that verbal and visual learning styles could be measured and that there was a
moderate correlation between visual and verbal learning styles and their associated abilities, but that the
learning styles explained even more o f the variance in scores than did ability.
Several dissertation studies which used the VVQ were located. They had mixed results. Blair
(1980). Kini (1993). Kuchler (1983). and Spiegel (1985) all obtained nonsignificant differences associated
with scores from the VVQ. Blair’s (1980) study was an attempt at increasing subjects’ ability to create
visual images. Subjects’ scores on the VVQ did not change after the visual imagery training. Kini’s (1993)
study compared subjects’ cognitive style, field independent (FI) and field dependent (FD) scores, and scores
from the VVQ with performance outcome on a concept-learning task. The FI-FD and VVQ dimensions
were independent, but subjects who received instruction that matched their preferred learning style did not
improve their learning performance. Kuchler (1983) administered a whole battery o f tests— including the
VVQ— in an attempt to determine the relationship between cerebral “hemispheric style” and the traditional
"cognitive style.” She obtained very low correlations (.00 to -.54) and concluded that there was
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
21
considerable diversity in cognitive style and that there is a need for further test development to measure
such constructs. Spiegel (1985) also found no significant differences in scores from two models o f math
Although two other researchers also obtained no significant differences, they did obtain some
interesting findings. Fishel (1984) obtained no significant difference in her study o f creativity and preferred
learning style. However, she did find that the mean scores for creative ability by females with a preferred
verbal learning style and males with a preferred visual learning style were slightly higher than the same
scores by their counterparts. In another study, Parzivand (1984) obtained similar results in his study o f
creativity and "m odes o f thinking,” as he called the scores from the VVQ. Although not significant, he
found that subjects with higher verbal-leaming scores tended to have higher creative production scores than
However, some researchers did obtain statistical significance in association with VVQ scores.
Ackerman-Efron (1985) found a significant positive correlation between the proportion o f upward eye
movements and stares (in response to modally ambiguous statements) and higher visual scores on the VVQ.
Dern (1982) found a significant positive correlation between higher visual scores on the VVQ and higher
scores on anxiety and depression measures. In experiments more closely related to this research. Garro
(1982) and Pantin (1982) both obtained statistical significance in the association o f scores from the VVQ
and performance outcomes. Garro (1982) found that subjects with a visual preferred learning style scored
better in both recall and recognition o f visual stimuli. This is important as it relates to the display o f content
in either graphic or textual format. Pantin (1982) found that the use o f visual imagery in text is also
important. Subjects rated higher those excerpts o f text that matched their own visual/verbal preference (i.e.,
subjects who had a higher visual score on the VVQ preferred more visual imagery in the written word).
The scores from the Verbalizer-Visualizer assessment tool have not been normed. Kirby (personal
communication. June 5. 1998) has suggested that although the constructs are independent, the range o f
scores for each construct would fall along a continuum, requiring a multiple regression analysis. Kirby
suggested that the comparison o f an individual’s score to the scores o f the related group (in this case,
secretarial employees) would provide the m ost useful information. The VVQ scores indicate the learner’s
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
22
preference for style o f learning (either verbal or visual) and were classified as an attribute variable (Borg &
Gall, 1989) and used to test any interaction between preference for learning style and type o f online help.
Subjects' scores from the VVQ were compared to their ability to learn and complete tasks using different
types o f online help systems which varied in the amount o f verbal and visual information provided.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
23
CHAPTER III
TO ONLINE HELP
The first goal o f this research was to test the application o f the general principles o f minimalism to
an online help system. The first general hypothesis was that users required (and preferred) less instructional
support to complete tasks, but more instructional support to learn tasks. The second general hypothesis was
that users who were prompted to interact with the application would spend longer amounts o f time learning
tasks, but, once learned, complete tasks faster without instructional support.
The second goal o f this research was to test the application o f Principle 3 o f minimalism
(supporting error recognition and recovery) to an online help system. The general hypothesis was that users
will be more effective and efficient at identifying and correcting errors if they have direct access to error
information.
The third goal o f this research was to determine the most effective combination o f information
types (e.g.. procedures, explanations, descriptions) for different preferences for learning style. The general
hypothesis was that users using an online help system that was matched to their learning style would
complete software application procedural skills more quickly and with fewer errors than subjects using a
Five different online help systems were developed in an attempt to implement selected principles
o f minimalism and test the above hypotheses. The first three online help systems followed those described
by Carroll (1990) and Black et al. (1987) in their experiments with the Minimal Manual. Those three help
systems are referred to as skeletal, inferential, and elaborative. Two additional help systems were developed
(using the Inferential help system as a base) to evaluate the addition and location o f error identification and
recovery information. These two help systems are referred to as inferential-proximal and inferential-distant.
All five help systems contained some identical characteristics so that variance could be associated
with their purposed differences. This section identifies those characters that were both common and unique
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
24
Care was taken to ensure that all the help systems were designed and developed in a similar
manner so that any differences among them would be more easily attributable to their purposed differences
in design. The attributes that each had in common included the same tasks in the same software application,
style guidelines, window format, table o f contents, title bars, action text, and use o f caution notes.
All the online help systems represented a tiny slice o f the MS Word software application. They
each contained instructions on how to complete the following general tasks: inserting and formatting text,
graphics, and tables: creating, formatting, and merging a mail merge data source; accessing, customizing,
and navigating the online help system. However, please note that although the help systems contained the
procedures for all the tasks listed above, the subjects used only those procedures associated with text for
Experiment I and then used only those procedures associated with graphics, tables, and mail merge for
Experiment 2. The procedures for text, graphics, tables, and mail merge were customized according to the
Stvle Guidelines
Each help system also followed the same style guideline, as specified by Boggan et al. (1996): use
o f gerund headings for Level 1 and Level 2 conceptual topics (e.g.. Inserting and Formatting Text), use o f
root headings for Level 3 headings that could be further subdivided into procedural topics (e.g.. Create a
Text Box), and use o f infinitive headings for Levels 3 and 4 procedural topics (e.g.. To Create a Text Box).
Window Format
The other constants across all five online help systems were type, color, size, and placement o f the
online help windows. Two windows were used to denote the difference between conceptual and/or
Figure 1 shows a main window containing an introductory topic for creating text boxes. Main
windows were formatted for landscape display and had a soft blue title bar over a soft yellow background.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
25
Uvimj MS Woril
File £dit Bookmark Options
Contents Index a^ck
Use text boxes to flow tex t automatically from one page to another or to create w ate rm arks
R elated T asks:
To create a text box
To create an A u to S h a o e text box
~o insert text into a text box
To link to another tex t box
To link to an .AutoShaoe text box
"o change the a p p e a ra n c e of a text box
These colors were chosen because they were softer and more aesthetically pleasing to the eye. The main
windows were larger in size (predominantly because some o f them contained much conceptual text), and
were positioned in the lower right com er o f the screen. Main windows had a default o f "Not On Top"
display (that is, they would disappear from view when the subject clicked anywhere outside the main help
window, thus allow ing the user to view more o f the task-application screen). They were set to an absolute
size with automatic scrollbars. Main windows generally did not contain any procedural steps (there was one
In contrast. Figure 2 shows a procedural window containing the procedures necessary for creating
text boxes. Procedural windows were formatted for portrait display and had a bright blue title bar over a
white background. T hese colors were brighter, drawing attention and providing a stark contrast o f the
emphasis formatting (e.g., red and green colors, bolded/italicized text). Procedural windows were smaller in
size— even the title b ar was changed to a narrow height— to allow the user to see more o f the screen while
simultaneously viewing the steps in the help system. Procedural windows were positioned in the upper right
comer o f the screen with a default o f “On Top” display (that is, they would remain visible on top of the
task-application screen until the subject closed them). They were set to relative size (that is. they expanded
vertically as necessary to accommodate the text) with automatic scrollbars when the amount o f text
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
26
In sert t e x t into a t e x t bo x
Link to a n o t h e r t e x t b o x
Link to a n A u t o S h a o e t e x t bo x
C h a n g e its a o o e a r a n c e
U s e it to c r e a t e a w a t e r m a r k
C h a n g e t h e d ir e c tio n of t h e te x t in s id e a te x t b o x
surpassed the height o f the computer window. Procedural windows contained the procedural steps with
Forward/backward chevrons ( » « ) were placed on the procedure windows, but not on the main
windows. Thus, subjects were able to view conceptual topics in the smaller, stay-on-top procedural window,
but not procedural topics in the larger, not-on-top window. Both windows contained a “ Back” button which
took subjects to the previously viewed screens. Subjects could access the Index directly from the Main
window, but had to first click on a “ Help Topics” button to access Index or Find from the procedural
window. This decision was made in an attempt to keep the procedural window simple with a focus on the
procedures.
Table o f Contents
The second principle o f minimalism is to anchor the tool in the task domain (Van der Meij &
Carroll. 1995). The two heuristics to support this principle are (a) select or design instructional activities
that are real tasks and (b) make the components o f the instruction reflect the task structure. Based on that
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
27
principle and those heuristics, all the online help systems were designed to assist subjects in completing
specific tasks rather than describe the functionality o f each dialog box and button in MS Word (the selected
software application). Figures 3 and 4 show an example o f the skeletal table o f contents at Level 2 and
Level 4. respectively. The tasks chosen represent typical end-user tasks in this software application.
With the exception o f the inferential-distant help system, all the help systems were identical down
to Level 2 o f the table o f contents (the former included a "Troubleshooting” section). The skeletal and
elaborative online help systems each contained the same listing o f procedural topics in Levels 3 and 4. The
inferential online help system displayed less procedural tasks (see Figure 5) in order to force the subject to
infer related tasks from the tasks given (compare the elaborative and inferential online help systems).
Title Bars
The title bars for the table o f contents and the main windows in all help systems displayed the
"Help Topics: Using MS Word” text. The title bars for the procedural windows in all help systems
Click a book, and then click Open. Ot dick another tab. such a t Index.
Prnt.. Cancel
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without perm ission.
28
Click a topic, and then click Display. Or cSck another tab. such a t Index.
Click a tope, and then click Display. Or cSck another tab. such as Index.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
29
Action Text
supporting heuristic is that we should provide an immediate opportunity to act (Van der Meij & Carroll.
1995). This principle and heuristic is closely related to Principle 4 (support reading to do, study, and locate)
and its supporting heuristic (be brief, don’t spell out everything). The implementation o f the first principle
and heuristic meant placing everything in active voice and designing topics around the associated tasks that
a given user may be attempting to complete (see, for example. Figure 6).
The implementation o f Principle 4 and its supporting heuristic meant eliminating some
information. Van der Meij and Carroll (1995 refer to two types o f screen information: the content and th
location. They suggest that “ incomplete screen information can be created by omitting one o f the two and
by making the information less explicit” (1995. pp. 256-257). Being brief, while simultaneously supporting
the user in his or her activity, meant eliminating the obvious (e.g.. "Click OK to close the dialog box."), but
it also meant going one step further and eliminating the steps leading to the location o f the information. For
.r jn jx j
Help Xopici fiack Print Options <<
>>
R elated Tasks:
To insert text into a text box
To link to another text box
To link to an AutoShape text box
To use a text box to create a watermark
To change the direction of the text inside a text box
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
30
example, in the illustration in Figure 6, Step 3 is to “Select a solid Fill C olor or a special Fill Effect,” but
the user is not told where to find the fill color effect in the resulting dialog box (see Figure 7). The user
must infer that the "Fill Color” option would be located under the "Colors and Lines” tab. They must then
infer that they must then click on the down arrow to select one o f the displayed colors in the color palette, or
click on "Fill Effect” and select a predefined fill effect or create a unique fill effect. The net effect o f being
brief while supporting the user’s activity is that the user is then forced to interact m ore with the interface
(and hopefully, therefore, learn and remember how to complete the given tasks in that application).
Caution Notes
Boggan et al. (1996) differentiated among the different types o f warning notes: danger (injury to
persons), warning (damage to software or hardware), and caution (potential loss o f data). The only type o f
warning note used in all the help systems is the caution note (Figure 8). The frequency o f the caution note
differs, however, among the help systems. All the help systems have at least the caution note illustrated in
Figure 8. The inferential help systems (including its revisions used in Experiment 2) include additional
Fill ------
Line — - NoFS
Color: ■■■■■■■■
Cashed:
■■■■■■■■
Arrows —
□ c c □ □ □ □ (□
More Colors...
fi* Effects...
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
31
mmm. - |D |X |
Help Topict
>>
Sack Prr* l
Qpfora ln
1 Create 3 data so urce.
2 Click on Tools I Mail Merge I S etu p ....
3 S e t the desired print and label options or envelope
options and d ic k OK.
J Click on the Insert Merge Field button and select
the layout of the fields you want to merge.
5 Click OK.
The first three help systems (used in Experiment 1) were based on the skeletal help system, but
then had content and instructional features either added or removed. The three help systems were skeletal,
The skeletal online help system included all the functions using terse, but explicit definitions and
procedural steps. Subjects had to click on hypertext to get definitions. There were no conceptual or
explanatory sections included in the procedural topics. Subjects learned by doing. Figure 6 is an illustration
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
□
X
i
Help Topics £ack £nnt Options <<
>>
On the page
1 Click and hold the mouse over the graphic.
2 Move the graphic to the desired position; release
mouse.
Within layers
1 Select the graphic
2 Click on Draw | Order.
3 Select one of the layering options.
O nYourO wn...
Related Tasks:
To in s e rt a oraohic
o f a procedural topic in the skeletal online help system. Figures 10 - 12 illustrate examples o f what the
subject would see if he or she clicked on some o f the definition hyperlinks in this window.
The elaborative online help system is based on the skeletal version o f help, but includes the i
explanation o f what the task does and why subjects might want to perform it. This explanation was
displayed below the heading and immediately above the procedural steps.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
33
T ext Box
A co ntain er for text. Word h a s several different s h a p e s for
text b ox es. You can resize and position text box es
anywhere on the document. You c a n also link text box es
tog eth er s o th at text flows automatically from one text box
to another.
Fill E ffect
Special effects for text boxes. There are four different ty p es
of fill effects: Gradient, .Texture, Pattern and .Picture,
G radient
A special fill-effect for drawing objects. There are three
Gradient options: (1) one solid color, (2) two colors in a
variety of shading options, (3) p re s e t gradient color th e m e s
{for example. Chrome, Ocean).
Notes. Elaborative help topics included supplemental information in the form o f notes. Although
not critical to complete the task, notes provided users with helpful information. When the information was
relevant to the entire task, then the note was placed below the steps; however, when the note was relevant
only to a step within the task (see Figure 13), then the note was placed immediately after that step (Boggan
et al.. 1996).
Graphic feedback. Elaborative help topics also included two types o f instructional, graphic
feedback: (a) "Click here for a graphic o f . . . ” text (see Figure 13), and (b) "W hat you should see” hypertext
located either at the end o f the sequence o f the tasks or (in the case o f a complex task) at appropriate
divisions within the task (see Figure 13). Figure 14 illustrates what a subject would see if they clicked on
the “Click h e re ...” text pictured in Figure 13; Figure 15 illustrates what they would see if they clicked on
the "W hat you should see” text. The graphical feedback confirmed to the users that they were or were not in
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
34
L?jCHa IBgHi'f
Help lopics
>>
Back Print Options <<
r
Select the text, cell, or box for which you want to
change the default Vi pt. black line border.
2 Click on £ormat | Borders and Shading.
(C.lick here for a graphic of the Borders and
Shading dialog box.)
' Al)
• Shadow
• 3-D
■ find
« C u sto m
the right place to complete the step o r that they had completed the step correctly. The main difference
between the two types o f graphical feedback is that the ‘‘What you should see” graphics included a b rief
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
35
B o ld e r: an d S h ad in g
r fitpne
Bos
Cleft on diagram bebw or use
buttons to apply borders
ID
Al
Color:
6
Gri£
&dth:
'A p t App(y to:
asm
C^tofli 3 | Table
C p tijr * ...
Showloobar OK Cancel
B o ld e r: an d S h ad in g
[□ Bos
[s' ®
[^Auto
Grig
i
w Mfldtht1 -
'Apt Apptttor
Qfftom
, p s m
Opffcnt.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
36
Hints. The hints in the elaborative online help system differ from those found in the inferential
online help systems. Elaborative hints provided additional information to help the user complete the given
task (see Figure 16). while the inferential hints encouraged the user to reason out how to com plete a similar
The inferential online help system was used in both experiments. In Experiment 1 it was used in
direct contrast to the skeletal and elaborative online help systems. In Experiment 2 it provided the base for
the error-recovery information added to the inferential-distant and inferential-proximal online help systems.
It was also used in its original state (i.e., without any error information included) for comparison purposes,
resulting in three online help systems for Experiment 2: inferential without, inferential-distant, and
inferential-proximal.
■ ■ ■ I
Hdpjopics Back Print Options <<
>>
1 Right-click on th e table.
2 S e lec t Bordeis and S hading.
R elated Tasks:
To insert a table
To change the alignm ent o f the table text
To chanoe the direction o f the table text
To change table borders
To shade or pattern c e lls
To meroe c e lls togeth er
To divide c ells
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
37
Help Topics
OnYoiirQyrn...
The inferential online help system used the skeletal version o f help as a base, but with less
information and more encouragement to "reason and improvise” the application’s functions (Carroll. 1990).
For example, where the subject had the option to select special fill effects for a text box. the procedural
steps were included for only one effect (see Figure 18) and the subject was encouraged (via “On Your
Own" text) to explore the rest (see Figure 19). The inferential online help system required the learner to
Another, more significant way that the inferential online help system forced users to reason and
improvise with the software application was to provide the directions for completing related tasks within
"Hints" located at the end o f the procedural steps for a base task. For example. Figure 17 shows the steps
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
38
Options
On Your Own...
Related Tasks:
To create a te xt box
To linn to a n o t h e n e x t box
Figure 18. Inferential procedural topic with "On your own” prompt.
for the base procedure o f creating a text box. However, notice the inclusion o f four related tasks (creating
an Autoshape text box. inserting text into a text box. using a text box as a watermark, and changing the
direction o f text inside a text box) located under “Hints” in the same window. Notice also that there are only
two related tasks as hypertext at the bottom o f the window. Compare this with the same procedural task
from the elaborative online help system (see Figure 20). By providing related tasks as hints in the same
window as the base procedure, users can apply the same knowledge to different, but similar, tasks and see
how those tasks are related to one another within an overall goal.
Based largely on the inferential online help system, this online help (used only in Experiment 2),
was designed to test the effectiveness o f error-recovery information placed in a ‘Troubleshooting” section
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
39
i i s e rt t e x t .n to a te x t b c <
j - n k tn - m o t h e r S e t t h o *
I 'in k tn o n A .u tT S h a o e h fix
i ■ tn e a p p e a ra n c e
I
I J -se it to c r e a te a w a te r m a rk
| > . n - e tn e e r e c t io n o f th e te x t in s id e a t e x t b o x
at the end o f the help system. Users had two ways o f accessing the error-recovery information: (a) through a
"Troubleshooting" button located on each procedural window and some main windows, and (b) through the
table of contents. Figure 21 shows the table o f contents for the inferential-distant online help system. It was
the only help system that difFered in the table o f contents at Level 2.
Most subjects, however, used the 'Troubleshooting” button (Figure 22) to access error-recovery
information. Clicking on that button resulted in the display o f the general “Troubleshooting” main window
(Figure 23). Subjects could then select the area in which they were having troubles and then access the
Again based largely on the inferential online help system, the inferential-proximal online help
(used only in Experiment 2), was designed to test the effectiveness o f error-recovery information placed
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
40
Click a book, and then dick Opea Or dick another tab. such as Index.
S s liiP ih ’:'"
r
_ -!□ ! x |
Helplopics fiack Print Options L<
>>
Troubleshooting
1. Click on Io o ls | Mail Mecge | Query Options;...
2. In Field, select the field you want lo use as a filter.
3 In Comparison, select a filter.
4 Enter the value of the filter in the Compare to field.
5 Click OK.
Related Tasks:
Editing/Formatting the Layout
Merging a List with a Main Document
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
41
Contents
119 I
Help Topics Back Print Options <<
>>
next to where users need it most (Lazonder, 1994: Van der Meij & Carroll, 1995). In line with the iterative-
testing approach to minimalism (Carroll, 1990), two pilot studies were conducted for this research to
determine the error-prone steps. Basic procedural steps were revised for clarity o f understanding. Error-
recovery information (see Figure 25) was then placed in the procedural topics at a frequency o f about once
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
42
Help Topic*
for every three to five actions (Van der Meij & Carroll [1995] suggested inserting error information at a rate
Subjects could view the error information related to the task or a substep o f the given task ju st by
clicking on the "error information" hypertext link. Unfamiliar words in the error information were
highlighted as hypertext pop-up definitions for further understanding. Notice that the error information for
both the inferential-proximal (Figure 26) and the inferential-distant (Figure 24) online help systems
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
43
CHAPTER IV
All subjects for this empirical research had to qualify for entrance based on their scores on a Kelly
Services Skills Testing o f MS Word 97 tasks. Once qualified, they completed a background-information
survey and a Verbalizer-Visualizer survey. Kelly Services personnel then scheduled them for two sessions,
one week apart, at the Business Information Systems and Education computer lab.
During the first session subjects used MS Word 97 to create three similar-looking newsletters as a
form o f a pretest, treatment, and posttest. During the second session, a week later, subjects created a final
newsletter. Subjects evaluated one o f the two online help systems they had used the week before.
This chapter identifies and describes the experimental materials that the subjects received. It also
describes the treatment design and specific hypotheses tested, the pilot studies, the external review o f
treatments, the population sample selection and assignment, the use o f lab consultants, the actual
experimental procedures, the procedure for coding data, and the procedures for statistical analyses.
Presession Materials
Subjects signed an informed consent form, completed questionnaires, and took a qualification test
Subjects were required to sign a letter o f informed consent (Appendix C) prior to being admitted to
the study. The letter o f informed consent specified that the subjects would be paid only upon completion o f
both experimental sessions, one week apart. The letter also specified the prerequisite qualifications for
participating in the study (basic word-processing expertise) and suggested that subjects could become more
An eight-question assessment tool (Appendix D) was designed specifically for this study to obtain
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
44
simple demographic information. The results o f this tool provided data on whether gender, age, level o f
general com puter experience, and experience with previous computer applications had any impact on how
Verbalizer-Visualizer Questionnaire
by Richardson (1977 as cited in Kirby et al., 1988) and later modified by Kirby et al. (1988) to assess a
person's preference for a verbal or visual learning style. All subjects completed this questionnaire before
being admitted to the study. The visual scores o f the experimental subjects were then used to stratify their
Subjects used a Kelly Services' proprietary, computer-based assessment tool to complete a variety
o f tasks in a simulated version o f MS Word 97. The skills-testing simulation recorded the time it took for
subjects to complete tasks using their choice o f method (e.g.. menu or keyboard). Subjects for this study
completed the basic version o f this test, which assessed mastery o f skills such as opening, closing, and
saving tiles: inserting, copying, moving, and deleting text (see Appendix B). Subjects had to receive a "Q"
(qualify) in order to participate in the study. Subjects could take the Kelly’s training in order to pass the
test. Originally subjects were required to complete correctly 80% o f the tasks in this test in 18 minutes in
order to receive a "Q .” However, some subjects were still struggling so much over basic Windows
operations (e.g.. open, close, save) that were not part o f the research study, that the minimum qualification
Experimental Materials
Subjects went through the same procedures and performed the same tasks throughout the course o f
this experiment. The only thing that differed among the subjects was the type o f online help instruction they
received. Everything that was the same for each subject is described in this section: the four newsletters the
subjects created, the software they used to create the newsletters, and the tasks that comprised the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
45
newsletters. Also described in this section are the instructions on how to use the online help systems, the
names o f the five different online help systems, and the evaluation instrument for the online help systems.
All subjects created (or at least attempted to create) four newsletters (Figures 27, 28, 29. and 30)
as a pretest, treatment, posttest, and delayed posttest assessment o f their expertise with intermediate and
advanced MS Ward tasks. Each assessment newsletter was comprised o f five main steps:
3. Insert and format a graphic (i.e., resize, position, apply word wrap)
4. Create and format a table (i.e., apply borders and shading; rotate text)
Each o f the five tasks was timed for the pretest and posttests. For the pretest, subjects were allotted
the amount o f time it would take for an expert to complete the same task. For the immediate and delayed
posttests, subjects were given a total o f 45 minutes (divided proportionately among the five tasks) to
complete all five tasks. The resulting time for each task was approximately one and one half to two times
the amount o f time it would take an expen to complete the same task. Subjects were not timed during the
completion o f the treatment tasks, but were encouraged to move ahead without completing the tasks after a
specified time (they had approximately 2 hours to complete the five tasks— a lab m onitor encouraged them
to move to the next task after each task’s proportion o f the 2 hours had elapsed).
Each subject also received directions (Figure 31) on what was required to com plete the newsletters
and where to find the text and graphic resources. The complete packet o f instructions received by each
experimental subject for Session I (pretest, treatment, and posttest) is located in Appendix F. The complete
Software
Carroll (1984) advocated the use o f commercial applications for research, suggesting that the use
o f noncommercial applications is analogous to creating “toy” systems that have no real application to the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
45
newsletters. Also described in this section are the instructions on how to use the online help systems, the
names o f the five different online help systems, and the evaluation instrument for the online help systems.
All subjects created (or at least attempted to create) four newsletters (Figures 27, 28, 29, and 30)
as a pretest, treatment, posttest, and delayed posttest assessment o f their expertise with intermediate and
advanced MS Word tasks. Each assessment newsletter was comprised o f five main steps:
3. Insert and format a graphic (i.e., resize, position, apply word wrap)
4. Create and format a table (i.e., apply borders and shading; rotate text)
Each o f the five tasks was timed for the pretest and posttests. For the pretest, subjects were allotted
the amount o f time it would take for an expert to complete the same task. For the immediate and delayed
posttests, subjects were given a total o f 45 minutes (divided proportionately among the five tasks) to
complete all five tasks. The resulting time for each task was approximately one and one half to two times
the amount o f time it would take an expert to complete the same task. Subjects were not timed during the
completion o f the treatment tasks, but were encouraged to move ahead without completing the tasks after a
specified time (they had approximately 2 hours to complete the five tasks— a lab monitor encouraged them
to move to the next task after each task’s proportion o f the 2 hours had elapsed).
Each subject also received directions (Figure 3 1) on what was required to complete the newsletters
and where to find the text and graphic resources. The complete packet o f instructions received by each
experimental subject for Session 1 (pretest, treatment, and posttest) is located in Appendix F. The complete
Software
Carroll (1984) advocated the use o f commercial applications for research, suggesting that the use
o f noncommercial applications is analogous to creating '‘toy” systems that have no real application to the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
46
;n»
! B«<*
01
0» ____
A 1i
ML
i StstTadcl
1 BstTadcl: >
! StatTn*k2 •
I BtiTukZ:
| StatTukJ
1
o Sales Report
i BbtTadcJ: u
Jg
j StartTuka.
i BtdTtdc* 9
| StetTukf
! BstT«*lc5
r
£
*
doddle
Food
3
2
5
4
4
4
B8 U
U
H B n iM f “ I T 16 14 r r ™ ....
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
47
ID# 123
Date: 60899
S tsrtlfck I: 1*5:36:54
Bui T « k l: 16*139
Start T*k2: 1*5:4155
Bui T * k2 :165041
Start Tfck3: 1*5:3>48
Adventure
Bui T«k3 465348
Start Ifckd: 16:56:43 T ravel
B ulT «k4:170959
Start B dc5:17:10:14
Bui Tadc5:17:1045
R«iew:
Xanadu Resort
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
48
ID# 133
Dd*: 4/1&09W
Hope
Stilt T u k l: 10:40*10
Bi4T*ikl: 10:4435
StntT «ik2:10:44:40
ai4T *jk2:10:40 35
StartTaikl: 10:4*00
BnlT«ik3:10:5225
StartT uk4:10:5230
Oil T «k 4:10:3805
StartTaikS:10:58:10
Bid T*ik5:11:1425
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
49
ED# 123
D dc:6A & t9»
S tart T ta k l: 1040:00
B id T aak l: 10.4432
S tart I * k 2 : 10:44:40
B u lT ad c2 :104335
S tart 1 fc lc 3 .10.40:00
B td T « k 3 :105235
S ta rtT U c 4 :105230
f i l l T « k 4 :105305
S tart T W c5:105010
B ui T * k £ : 11:1425
V U n t a t iM n n M ^ r h y r t m ilH iiu i
SchaMtaf fesptdfan
City Sections
North W art East South
1 fen Fab Bfir /p H
% 2 M*y tan Mr Aug
p
o Stp. 0<t Mar Dtc
3
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
50
Start:
Position your cursor after"Start Task 1” and. insert the time from Invert | Date and Time.
Use the very last time format (14:30:25).
I. Create a text box spanning from, the harder o f the dateAime text box to the right margin and
as long as the date/time text box (i.e., align the battomborders o f the two text boxes).
6. Apply to the text box atight text wrap with a 0.1 external margin.
9. Place the star over the “Sales Report” text box as shown.
10. Make the 'W e're #1" text white, Arial, 14 point font, bold, centered.
12. Apply the same texture and border as the “Sales Report” text box.
Stop:
Position your cursor after“End T ask 1” and insert the time from Insert | Date and Time.
Use the very fort time format (14:30:25).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
51
real world and real problems. M S Word 97 was selected as the software application for this experiment
because it is a commercially distributed and widely used word-processing application. Its prolific use will
increase the generalizability o f any findings to a large target audience. The use o f M S Word also increases
the available sample size (i.e.. many people are interested in extending their expertise in a word-processing
application).
Carroll (1990) criticized the use o f artificial tasks in research as being nonrelevant to the subject.
The tasks chosen for this research were identified as either intermediate or advanced skills in the Kelly
Testing & Training Management System (see Appendix B for a list o f basic, intermediate, and advanced
skills). They represent functions o f the software application that are not normally used by the typical end
user, but are requested by employers and would increase the end user’s productivity. The subjects in this
research were advised that learning these tasks would make them more employable and effective in their
work settings.
1. Insert and format a graphic (i.e.. resize, position, apply word wrap)
2. Create and format a table (i.e.. apply borders and shading; rotate text)
It is important to note that all five tasks were completed for each newsletter activity; the subjects
were unaware that they were transitioning from one experiment to another within the context o f the same
newsletter.
It should also be noted here that Mirel (1998) suggested that some tasks that technical
communicators define as complex (she specifically refers to creating and applying print merges) are not
complex tasks because they are not open ended, situational, and contingent She defined complex tasks as
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
52
those requiring problem solving or the creation o f new perspectives for novel purposes. For the purposes o f
this study. I chose to define complex tasks as complicated procedures that require an intermediate or
The lab consultants provided instruction and demonstration on how to navigate through and use the
online help system. Subjects were provided a copy o f the instructions (Appendix H) to which they could
Subjects had access to one o f three different online help systems to complete Tasks 1 and 2 o f the
treatment newsletter (Experiment 1). The three different online help systems were the skeletal, elaborative.
and inferential.
Subjects had access to another o f three different online help systems to complete Tasks 3 .4 . and 5
of the treatment newsletter (Experiment 2). The three different online help systems were inferential-without.
inferential-distant, and inferential-proximal. (The inferential-without system for Experiment 2 was identical
to the inferential system from Experiment I .) The online help instructions were the only instructions
After completing the second session, subjects used the Help Design Evaluation Questionnaire
(HDEQ: Duffey et al„ 1992) to evaluate one o f the two online help systems they had used the previous
week. This eight-category assessment tool (Appendix I) was designed, developed, and tested by Duffy et al.
(1992) to measure the usability o f online help systems. It measures an online help system's support for
problem representation, access to help, topic selection, ease o f scanning help, appropriateness o f the type o f
help information presented, comprehensibility o f help content, ease o f navigation, and ease o f transferring
information from help to the application. The HDEQ allows the omission o f specific items that are not
relevant to the given online help system. The “Access” section was omitted since the research had no access
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
to Microsoft’s proprietary code for Word and was therefore unable to create context-sensitive help for the
application. Scores from the HDEQ were used to measure subjects’ evaluation o f the various online help
systems. They were also used to compare subjects’ evaluation o f the online help systems with ability to
perform tasks in those systems. The purpose o f the HDEQ was to assess which style o f help users preferred
(as compared to measures indicating which style o f help was more effective).
Treatment Design
The experimental design selected for this research was the pretest-posttest, 2 x 4 factorial design.
1. Inferential help (I) will be more effective in teaching software application procedural skills
than skeletal (S). which will be more effective in teaching procedural skills than elaborative (E) online help
2. While learning procedural tasks, subjects who use the skeletal help (S) will complete tasks
more quickly than subjects using elaborative help (E). who will complete tasks more quickly than subjects
3. Subjects who used the inferential help (1) to learn tasks will complete those tasks more quickly
than subjects using skeletal help (S), who will complete tasks more quickly than subjects using elaborative
4. Inferential-proximal (P) online help will be more effective in helping learners identify and
correct errors than inferential-distant (T) online help, which will be more effective than inferential-without
5. Subjects who prefer a verbal style o f learning will complete tasks more accurately and
6. Subjects who prefer a visual style o f learning will complete tasks more accurately and
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
54
Pilot Studies
One usability and two pilot studies were conducted to test the treatments and procedures. The
usability study was conducted over a year prior to the actual experiment. The researcher and two assistants
each monitored a different subject, drawn from the sampling pool, as they completed the five tasks selected
for the experiment. Entrance qualification criteria and treatment modifications were based on the findings
The first pilot study was conducted at the same time the experimental treatments were sent out for
an external review by Laurie Kantner from Tec-Ed (see Appendix J). The pilot study was administered to
11 subjects under the actual test conditions for this study. Modifications to the lab-assistant procedures,
subject experimental materials, and treatments were based on the findings from the first pilot study.
The second pilot study was conducted immediately prior to the experiment. The pilot study was
administered to 6 subjects under the actual test conditions for this study. There were no problems. Data
Laurie Kantner, o f Tec-Ed, completed iterative expert reviews o f the treatments and the subject
experimental materials. Kantner and colleague Stephanie Rosenbaum teach workshops on the
implementation o f minimalism.
Carroll (1984) criticized the use o f undergraduates as experimental subjects, suggesting they are
compliant and lack real-world experience. The 1 11 subjects for this research were obtained from the clerical
pool of a local temporary employment agency (Kelly Temporary Services). These subjects were very
ty pical o f the target audience. Clerical employees are predominant users o f word-processing applications.
They may have an intrinsic motivation to gain expertise in M S W ord in order to function m ore effectively in
their work.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
55
Selection
Kelly Temporary Services (KTS) routinely screens all applicants according to their expertise with
word-processing applications. Subjects from the clerical pool received monetary compensation from the
employment agency (who in turn billed the researcher). Subjects were paid only at the completion o f both
experimental sessions.
Subjects from the pool who did not qualify at the basic level o f word-processing expertise and
subjects who did not speak English were eliminated from the sample. Subjects had to be familiar with a
word-processing software (either M S Word or WordPerfect), but not be an expert programmer or hacker.
This was based on research conducted by Roper Starch Worldwide (Snapshot. 1998) that states that the
majority o f computer users (58%) think that computer technology is important to know and somewhat
confusing. This is the target benefactor o f these research findings. They are neither the upper 15% that are
programmers nor are they the lower 15% that are technophobes.
All subjects were assigned a Kelly Service ID number upon selection for participation. They used
that number on all testing materials throughout the study in order to protect their anonymity. Subjects for
each Kelly-scheduled session were randomly assigned (by the researcher) to both the first and second
experimental conditions based on their visual score from the VVQ. Subjects were ranked from highest to
lowest based on their visual score. Three different coins were then used to randomly assign the top three
subjects to each o f the first three experimental conditions (skeletal, inferential, elaborative) and then again
proximal).
The subjects’ KTS ID numbers were placed in a box and shaken. A single coin flip and a draw
from the box were then used to select which online help system each subject would evaluate using the
HDEQ (Appendix I). For example, if three subjects were using the combined skeletal/inferential-proximal
online help systems, then their ID numbers were tossed in the box. The coin flip determined if the first ID
drawn would evaluate the skeletal or the inferential-proximal online help system. The next ID drawn would
R e p ro d u c e d with perm ission of the copyright owner. F urther reproduction prohibited without permission.
56
evaluate the other online help system. The remaining ID was paired up with another user o f either the
skeletal or the inferential-proximal online help system, and the procedure was repeated.
Subject Attrition
Subjects who did not complete a full session or who failed to show up for the second session were
eliminated from the study and were not paid for any time they had already invested. Fifty-one subjects went
through the preliminary, 2-hour qualification, registration, and scheduling process (unpaid) at Kelly
Temporary Services, but never showed up for the experiment. Twenty-two subjects completed all or part o f
the first session but either left before completing the first session or did not show up for the second session.
Four subjects were so frustrated that they were excused from the study. The data from 33 subjects were
accidentally erased by the computer lab manager. The data from the remaining 111 subjects was used for
Table 1
^ •%
51 13 9 4 JJ
The same four, paid lab consultants were used for the entire data-collection phase o f the
experiment (which lasted 6 months). Three o f the lab consultants were selected specifically for their
troubleshooting expertise in the microcomputer labs: the fourth was selected for her speaking ability (she
welcomed the subjects and read all the directions) and troubleshooting expertise.
AH four lab consultants were trained. They each completed the same activities required o f the
subjects, with the exception that the researcher was there to guide them if they ran into problems. The lab
consultants were blind to the hypotheses o f the experiments and the differences among the help systems.
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
57
For the most pan the lab consultants, not the researcher, interacted with the subjects (the few
exceptions were when there were subjects who had to be excused from the experiment because o f
belligerence or total frustration and inability to continue). The lab consultants monitored the subjects’
activities and intervened when the subjects had technical difficulties or general Windows questions; the
researcher monitored the lab consultants to make sure they were not providing instructional support to the
subjects (the researcher’s windowed office was in the back o f the lab).
Experimental Procedures
Subjects, in groups o f 9 to 15, were schedule for two experimental sessions, one week apart. This
Presession Activities
All sessions for this research were conducted in the same College o f Business microcomputer lab
at Utah State University. The lab consisted o f 27 Dell. Pentium II computers connected to a LAN.
Prior to the beginning o f the data collection phase o f this research, all computers in the designated
lab were "ghosted" with an image that included the online help systems (the treatments) and access to those
help systems from the S ta rt | P rogram s pop-up menu. “Ghosting” refers to the process o f configuring one
computer and then copying an image o f that computer (including all hard drive contents and pointers to
server applications) via the server to all the computers in that lab.
computer. Camcorder was used to record subjects’ mouse moves and keyboard strokes as they completed
each of the experimental activities. The resulting data were saved as an .avi file.
Subjects completed the qualification testing, the background survey, and the W Q at the Kelly
Temporary Services (KTS) facility. A KTS person would either bring that data to the researcher, or she
would pick it up. The researcher ranked the subjects according to their visual score and then randomly
(using pennies) assigned them to one o f the three experimental conditions for each experiment. The
researcher assigned each subject to a com puter (in order to space the subjects out) and then created an
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
58
assignment sheet (Figure 32) displaying each subject’s assignment to a computer, lab assistant, and
experimental conditions.
Just prior to the arrival o f the subjects for each session, the researcher removed the access to
IYard's online help (FI and the Help menu) and disabled the Help wizard. The text and graphic resources
for the session were downloaded from the server onto each computer: Camcorder was launched and placed
in the upper right com er o f the screen: and Word 97 was launched, set to Page Layout, and set to 80% view
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
59
The researcher arranged the materials for each subject onto tables for fast access by the lab consultants. The
materials included a general description o f the session activities, samples o f the newsletters, and instructions
The researcher then wrote the following general instructions on the front white board:
■ The lab consultants are not allowed to help you complete your tasks. Please d on’t ask them.
■ All text and graphic resources are located in the C:\Newsletter folder.
= Undo
■ A = Font color
Session 1 Activities
The researcher greeted the subjects in the hallway outside the computer lab. checked them in
according to their KTS ID number, had them sign the KTS time sheet, and organized them into groups
according to their assigned lab consultant. The researcher then introduced the subjects (by ID number) to
their assigned lab consultant and turned them over to the lab consultant for seating.
One lab consultant greeted the subjects, read the "Welcome/Overview” materials (Figure 33), and
guided them through the initial open/save/time procedures (see the complete Packets in Appendices F and
G). Another subject demonstrated the open/save/time procedures on the demonstration computer, while the
When all subjects were ready, the lab consultants distributed Task 1, set the timer, and told the
subjects to begin. At the completion o f Task 1. the lab consultants picked up the Task 1 instructions and
replaced them with instructions for completing Task 2. This procedure was repeated until all five pretest
tasks had been completed. This portion o f the experiment took approximately 45 to 50 minutes.
At the completion o f the five tasks, the subjects were excused from the lab while the lab
consultants launched the assigned online help system for each subject and distributed the directions for
navigating the help systems. The subjects, meanwhile, were treated to some sort o f refreshment (e.g., sweet
breads or cookies).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
60
Overview
Welcome>Thank you fo r participating m this study.
This orientation packet is designed to guide you through the activities for this session. We will
preview the session activities as a group before you start You may refer back at any time to the
instructions included in this packet
Session Activities
During today’s session you will complete three very similar activities using different content, but
comprised of the same five tasks:
1 Create and format regular and AutoShape text boxes
2 Insert text; format it into columns
3 Insert graphics; wrap the text around them
4 Create a table, format it with borders, shading, etc.
5 Use mail merge to create mailing labels
All three activities you complete today resemble partially-completed newsletters. A description of
each activity follows:
» Create a pardon of a sales report (see enclosed sample): m e mail men*e to create mailing
labels. (35 minutes)
The intent of this first activity is to introduce you to intermediate- and advanced-level skills m
MS Word that will make you more employable You will attempt to complete five tasks in M S
Word without any help. You will have 35 minutes to complete this newsletter/sales report
Note: Although each task is timed, feel free to work at your own pace Note: It is not expected
that you know how to complete each task—these are intermediate and advanced-level tasks m
M S Word It's OK if you do not know how to complete them now—you will be able to use the
online help system to complete these same tasks in the second portion of this session. Just do the
best you can.
* Create a newsletter; use mail merge to create mailing labels. (approximately 2 hours)
________This second activity is to enable you to learn the same MS Word skills using an online help_____
After the subjects were again seated, one lab consultant read the online help navigational
instructions (Appendix H), another demonstrated the procedures, and the other two assisted subjects.
When all subjects were ready, the lab consultants distributed Tasks I and 2 for the treatment
newsletter and told the subjects to begin. The subjects were told they had approximately 2 hours to
complete the five tasks. The time was divided proportionately among the five tasks. One lab consultant was
in charge o f the time and, at the end o f the identified time for each task, would encourage the subjects to
move to the next task even if they had not yet completed the task on which they had been working.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
61
When a subject completed Task 2, he or she was instructed to raise his or her hand and notify a lab
consultant. The lab consultant then stopped Camcorder and saved the resulting .avi file, exchanged Tasks I
and 2 for Tasks 3, 4. and 5. closed the first online help system, and launched the second o f the two assigned
online help systems. At the completion o f Task 5, the subject was given the opportunity to review any o f the
procedures again using the online help system. When subjects felt comfortable enough to complete the
procedures without the assistance o f an online help system, they again notified a lab consultant. The lab
consultant stopped Camcorder, saved the resulting .avi file, and picked up Tasks 3, 4, and 5 from the
treatment newsletter.
The lab consultant then, on a subject-by-subject basis, distributed Task 1 o f the posttest newsletter
to each subject, recorded the start time for that subject, and told the subject to begin. When the subject
completed Task 5 o f the posttest newsletter, he or she was thanked for participating, reminded that payment
was contingent on completion o f the next week's activities, and reminded to sign out on the KTS time sheet.
At the end o f Session 1, the researcher moved all the .doc and .avi files generated by the subjects
to the server, then moved them once again to the demo computer, where they were then burned onto CD-
ROMs. It generally took one CD-ROM per subject (see Figure 34).
Session 2 Activities
The researcher completed all the same presession procedures for Session 2 as she did for Session
I : picked up the qualification testing scores and surveys, ranked and assigned subjects, prepared computers
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
62
for the session, arranged subject materials, wrote reminders on the white board, and checked in subjects
The lab consultants seated the subjects at the same computers they had used the previous week,
demonstrated the open/save/time procedures once again, distributed Task 1, set the timer, and told the
subjects to begin. At the completion o f Task 1, the lab consultants picked up the Task I instructions and
replaced them with instructions for completing Task 2. This procedure was repeated until all five delayed
At the completion o f Task 5, the lab consultants stopped Camcorder and saved the .avi file,
launched the online help system the subject was assigned to evaluate, and distributed the Help Design
Evaluation Questionnaire (Appendix I). After completing the HDEQ, each subject was thanked for his or
At the end o f Session 2. the researcher again moved all the .doc and .avi files generated by the
subjects to the server, then moved them once again to the demo computer, where they were then burned
onto CD-ROMs.
The A VI Player was used to watch the .avi video files on the computer. Five different checklists
were used to code each subject’s data. A research assistant was trained to assist the researcher in the data
coding.
AVI Plaver
Dr. Thomas Hardy, Utah State University, created the A VI Player, the tool used to play the
resulting .avi subject data files. Camcorder produces unique .avi files that even the Windows Media Player
would not play. Camcorder has only Stop and Play buttons, requiring the user to start over from the very
beginning if it were necessary to stop for a phone call or meeting. With approximately five hours o f
recorded data per subject, that option was unacceptable. The A VI Player had regular VCR-like controls plus
a frame-by-frame forward/backward option, and an invaluable option for starting at any given frame.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
63
The starting and stopping times (Figure 35) for each task were recorded onto the Dissertation Data
Sheet. The online help information (e.g., features accessed and available) and the subjects Verbalizer-
Task completeness was measured using the Activity Tasks Checklist. The same five tasks and their
associated steps were listed for each activity (Figure 37). There were 28 steps total to complete each task.
Subject received a check next to the step if it was completed, even if the subject did not use the correct or
Accuracy was measured by comparing the basic steps for completing each o f the tasks (as
identified in the online help systems) to the steps the subject took to complete the same tasks. The
procedures from the online help systems were compiled into a separate Help Tasks Checklist document.
Where a different way o f completing the task was actually more efficient but was not consistent with the
PRETEST
A c tk itg h n m i f a k i A j k l i A ccuaK r
T m ik F tm t S ta r t Stop T o to i 96 C onpL Pom Co XT % Extoes / * C o st R oc. T i m
C reate a b a n n e r h e a d in g ( te x t b o x es)
A d d a n d fo rm at text
A d d a n d fo rm at p ictu re
A d d an d fo rm at table
C reate a n d p n n t m ailin g lab els
To tale t
T R EA TM EN T S
A cthdty 2 1N e w le tr e d la W b Ac c u b k t
T ask Fnm S to rt Stop T o ta l % CuiopL Pom C o rr % E rx o c * / % C a r R oc. T i m
C reate a logo (te x tb o x e s )
A d d a n d fo rm at tex t
A d d a n d fo rm at p ictu re
A d d a n d fo rm a! tab le
C reate a n d p n n t m ailin g lab els
f o ta k t
R rrin rt
M A ST ER Y PO STT EST
ill liiipi 1\ N n w h ltir f f olw li i | A c c u to tr
Tuk ham S tu t Stop T o to l I 91 CmopL Pom C o rr % E x x o rs / % C o e r R oc. T n m
. - £ » t o a o .h i n u o r hoobim ftffoH hnT or) L _— ...
Figure 35. Top part o f dissertation data sheet.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
64
Fttfiw:
daCmiiai H-Om
I>Dkfiraiian. 7*Troahkal»<rt(»cBtttim
E*Emr In&eynetim N-Note W-Poo-u#cipMt(vA at milumld—..*)
cnpiacCcEdthsia*»** ) OOm YourOwi
Accm:
C iC o ttn * W iukx R>RikfedT«)« ♦»
! Activity 1: NewsletterA.aDeis I
i i Task 1: Create a banner heading Task 2: Add and format ted Task 3: Add and format picture |
I I 1 Create a regular textbox l Insert text file l Insert a p tc ttse 1
i 2 Create an au to sh g re textbox i Format first h ae into heading; form at type 2 A pp h rftg h f text w rap
ZZJ 3
4
E nter an d form at text
Rotate text m text box
3
4
Form at text m to three colum ns; 2 .2
Form at lest Ime into heading; form at type.
3. Edit w rap around d a n k
4 P o sitio n picture
5 A pply eith er a texture o r p a r e m M
1 16 A pply a special b order
| 7 A pply a fight text text uoqa
I 3 P ositio n text b o ats
__ ! Task 4: Add and format table Task 5: Create the mailing labels
l Create table I Create moil m erge docum ent
2 Form at h o rd e s 2. Create d a ta source |
3. F oim at shading 3. S etup d o cu m eg !
4 M erge cells. 4 S ort the labels a asctndm g order.
5 R otate text i 3 M erer th e labels to a r a r Document.
6 Align text vertically
7 Align text h crtro n taily
main approach used throughout the help system, then both methods were listed (see Figure 38) and the
Error-commitment and error-recovery information was recorded on the lines below each task. The
number o f errors the subject successfully resolved were divided into the total number o f errors committed.
The amount o f time subjects spent on error recovery was recorded to the nearest 30-second interval.
Research Assistant
The data from three different subjects (selected because they increased in level o f data coding
judgment) were used to train a research assistant. The researcher first coded the data and then had the
research assistant code it. The researcher and research assistant discussed the differences and agreed on a
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
65
coding rubric. This procedure was repeated for each o f the next two subjects. Thereafter, the research
assistant and researcher discussed any new coding situations and added them to the master coding sheet
The two statistical analyses used in this research were a multiple multivariate analyses o f
covariance followed by analyses o f variance, simple effects and pairwise comparisons for statistically
significant findings. The multivariate analysis o f covariance was selected in order to provide a benchmark
comparison point when conducting a more complete and detailed description o f differences among the
groups, especially with correlated variables (Stevens. 1996). The analysis o f variance is the appropriate
methodology for “describing the relationship between a continuous independent variable and one or more
dependent variables” (Kleinbaum, Kupper. & Muller, 1988. p. 12). The Bonferroni approach (Green,
Salkind. &. Akey. 1997) was used to prevent Type I error. Wilks lambda A was the statistic used to
perspective o f the data. The repeated measures methodology prescribed by Pedhazur and Schmelkin (1991)
was used as the guide for this analysis. The exploratory analysis was conducted to determine if any learning
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
66
1 2 3 4
To create a textbox
t Click on Insect I Text Box or.
Click on the test box icon
2 Click, hold ana drag the crossbars la me desired test bos size.
Rl
Errors
a Count as 3 steps irS creates a rectangular AutoSnape ana m en changes it tc a lest bos
j
; 3 can form at ane test box ana then cocv ana cnange :t w ith out error (prevents having to ‘armat twice)
t ; - n o r if S clicks anc men arous test aos reserved after S resizes it
|
i
I To c r e a t e an AutoShape textbox
1 Click on me Draw icon on me Standard toolbar to display the Drawing toolbar tor insert;
I 2 Click on me A utoShapes menu on me Drawing toolbar car Picture » AutoShaoesj
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
67
CHAPTER V
RESULTS
The general purpose o f this research was to define ways in which minimalism, an instructional
methodology designed for print-based training materials, could be effectively modified and applied to
online help systems. Two separate experiments (using different sets o f online help systems) were designed
to evaluate different principles o f minimalism. The first experiment focused on comparing variations o f
minimalist instruction for increased learning. The second experiment focused on identifying the effect that
presence and location oferror-recovery information had on subject errors. General error-
eommissiorv'recovery data were also obtained for Experiment 1; instructional-effectiveness data were also
obtained for experiment 2. Post hoc analyses were conducted for only significant findings from the
multivariate analyses.
software application would increase if his or her preference for learning style would be matched to an
instructional approach. The two hypotheses (with associated data analysis) for this research question follow
The data analysis follows a deductive approach: description o f the sample population, help
systems, and data set are first described followed by a multivariate analysis o f specific variables by
research hypotheses. The general support (or no support) for the experimental hypotheses are presented
The 111 experimental subjects used in this study were randomly assigned to the different
The majority o f the sample population was comprised o f females aged 18-25 with 1 to 10 years o f
computer experience (Tables 2 and 3). Sixty-seven percent o f the sample population indicated that they
possessed an intermediate level o f experience in Microsoft Word or Corel WordPerfect (Table 3). Nearly
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
68
Table 2
36-40 1 0.9
41-45 3 2.7
46-50 1 0.9
51-55 i 1.8
Table 3
over 10 11 9.9
Missing 3 2.7
111 100.0
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
69
90% o f the sample population was familiar with the WSkV Windows platform, and nearly 65% owned their
own computer (Table 4). The sample population used for this research was much more advanced in their
use o f computer technology than the sample population used by Carroll in his experiments in the early
1980s: however, the sample population more accurately represents the current target population. A detailed
Table 4
Table 5
Experiment I Experiment 2
N 38 36 38 36
(table continues)
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
70
Experiment I Experiment 2
Comp
Word
(table continues!
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
71
Experiment 1 Experiment 2
Note. The same subjects were randomly assigned to both Experiment 1 and Experiment 2.
All subjects in the experimental groups completed a Help Design Evaluation Questionnaire
(Duffey et al„ 1992) at the completion o f the delayed posttest. They evaluated one o f the two online help
systems used during the treatment, evaluating that system along seven measures. The means and standard
deviations for these measures across all help systems are displayed in Tables 6 and 7. T he greatest mean
differences were associated with Content. These differences approached significance (p = .07) at the .05
level o f confidence. However, a multivariate analysis o f the help systems on these measures revealed no
statistically significant differences (Table 8 ). The subject rating o f each online help system was equivalent
Description o f Data
Prior to conducting an in-depth analysis o f the data to search for statistically significant
differences among the online help systems, simple descriptive tests were conducted. The greatest mean
differences and variability in data among the treatment groups for Experiment 1 were associated with the
completeness variable during the treatment session (Table 9). The mean completion scores o f subjects in
the elaborative group were 8% higher than the scores o f their counterparts in the skeletal group. This
finding is contrary to what was anticipated. Other interesting mean differences from T able 9 are associated
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
72
Table 6
Help system M SD M SD M SD M SD
Table 7
Help system M SD M SD M SD M SD
with the accuracy variable. Accuracy scores o f subjects in the inferential group were 7% higher than those
o f subjects in the skeletal group during the treatment and 5% higher than those o f subjects in both the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
73
Table 8
Table 9
Measure Exp. 1 M SD M SD M SD M SD
Time (secs) Skeletal 803.95 70.53 2577.97 1466.65 840.53 162.37 861.21 98.20
Complete Skeletal 0.41 0.18 0.79 0.18 0.77 0.21 0.76 0.24
Accuracy Skeletal 0.64 0.20 0.81 0.13 0.80 0.15 0.81 0.13
skeletal and elaborative groups during the immediate posttest. The only other initially interesting finding
was that the subjects in the elaborative group corrected 5 to 7% more errors than the subjects in the skeletal
group during the pretest, treatment, and immediate posttest (Table 10).
The greatest mean differences and variability in data among the treatment groups for Experiment 2
were associated with the completeness variable (Table 11). Subjects in the inferential-without group
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
74
Table 10
Measure Exp. 1 M SD M SD M SD M SD
Errors Skeletal 6.63 4.45 8.29 6.00 4.42 3.13 3.89 2.70
"o Correct Skeletal 0.69 0.21 0.86 0.18 0.63 0.32 0.79 0.27
Rec time Skeletal 194.21 142.73 515.53 426.94 131.84 138.19 139.74 117.19
completed 10% more tasks than did their counterparts in the inferential-proximal group during the delayed
posttest: subjects in the inferential-distant group completed 9% more tasks than did their counterparts in the
inferential-proximal group during the immediate posttest. These findings are contrary to what was expected.
There were also some unexpected findings associated with the percentage o f errors corrected for
Experiment 2 (Table 12). Subjects in the inferential-distant group corrected 16% more errors than did their
counterparts in the inferential-without group during the pretest. They also corrected 12% and 8% more than
those in the inferential-proximal group in the posttest and delayed posttest, respectively.
In addition, m ore subjects preferred a visual (versus verbal) learning style (Table 13). The
complete breakdown o f visual and verbal preferences for learning are displayed in Tables 14 and 15. The
distribution o f visual and verbal learners is equivalent across all treatment groups.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
75
Table 11
Measure Exp. 2 M SD M SD M SD M SD
Time (secs) Without 1207.58 106.63 4447.16 1223.60 1527.79 222.15 1600.87 234.94
Complete Without 0.22 0.14 0.77 0.24 0.73 0.26 0.79 0.24
Accuracy Without 0.72 0.17 0.90 0.13 0.92 0.11 0.92 0.09
Hypothesis I stated that “ inferential help (I) will be more effective in teaching software
application procedural skills than skeletal (S), which will be more effective in teaching procedural skills
than elaborative (E) online help. “Effective” refers to the combination o f the following measures:
2. Accuracy: the exactness by which the subject’s method for completing a given task matches
3. Errors committed: the number o f errors the subject made while completing a task
4. Percentage o f errors corrected: the number o f corrected errors divided by the total number o f
errors committed
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
76
5. Error recovery time: the total time spent correcting errors per experiment and activity.
Analysis o f covariance was the statistical test selected in order to best determine the magnitude o f
differences among the groups after exposure the testing and treatment materials. Therefore, the interaction
between the group and the performance measures was assessed to ensure that the group regression lines
Table 12
Measure Exp. 2 M SD M SD M SD M SD
Error Without 9.13 3.29 15.84 9.02 6.39 6.39 7.76 6.33
11o Correct Without 0.57 0.25 0.79 0.15 0.75 0.29 0.75 0.25
Rec time Without 292.18 197.37 959.21 681.06 211.66 217.15 294.47 212.34
Table 13
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
77
Table 14
Table 15
were parallel to the overall regression lines (Table 16). The analysis o f covariance procedure adjusts
dependent variable means for each group along the slope o f the overall regression line. Accordingly,
straightforward interpretation o f analysis o f covariance results requires the assumption that regression lines
for each group be parallel. The absence o f an interaction between the independent variable and the
covariates means that regression lines are parallel, that the adjustment employed by the analysis o f
covariance procedure is appropriate, and that analysis o f covariance results may be interpreted
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
78
straightforwardly. Since none o f the interaction terms was statistically significant, this assumption is met to
a reasonable degree, and use o f analysis o f covariance is appropriate for the data.
Three different multivariate analyses o f covariance were conducted to evaluate the relationship
between types o f online minimalist instruction and specific performance measures. The pretest scores on
the above five measures were used as the covariate to measure any differences in subjects' performance
Table 17 represents the combined output o f the multivariate analyses o f the five performance
measures listed above applied to Experiment 1 (skeletal, inferential, and elaborative) for the treatment,
posttest, and delayed posttest using the pretest measures as covariates. Only treatment was significant at the
multivariate level (F=1.92, p = .04). An intermediate effect size ( q - = .09) with a strong power (1- |3 = .86)
indicated that the magnitude o f the variance obtained was related to the treatment effects and the
probability o f obtaining such a difference by accident would be less than 15%. (Stevens [1996] lists q - =
.01 as a small, q - = .06 as a medium, and q - = . 14 as a large effect size for social science research.)
Separate univariate analyses o f covariances were conducted (Table 18) for each o f the five
measures to determine which measures contributed to the overall level o f significance for the multivariate
analysis o f covariance during the treatment (Table 18). Both the completeness and accuracy variables were
significant at the .05 level o f confidence (F = 3.37. p = 0.04; F = 3.36. p = 0.04. respectively). A medium
effect size ( q - = .06) and level o f power (1- P = .62) were also obtained for both measures.
An additional pairwise comparison (using the Bonferroni approach) o f the completeness measure
Table 16
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
79
Table 17
Table 18
revealed a statistically significant difference (e = .05) between subjects in the elaborative and skeletal
groups (Table 19). The mean completion scores for subjects using elaborative online help (M = .87. SD =
.14) was statistically significantly greater at the .05 level of confidence than the completion scores for
subjects using skeletal online help (M = -79, SD = .18). A standardized m ean difference effect size
(SM DES) o f .49 indicated that subjects in the elaborative group completed about 7% more tasks (.49 x .14)
than did subjects in the skeletal group. Figures 40 and 41 illustrate the m agnitude o f difference between the
two groups.
Although not statistically significant, the standardized mean difference effect size (0.49) for the
accuracy measures between the subjects in the inferential and skeletal groups (Table 19) tended toward
significance (p = .0.07), which contributed to the overall significance o f the accuracy measure during
R e p ro d u c e d with perm ission of the copyright owner. Furth er reproduction prohibited without permission.
80
Table 19
SMDES
Measure (I) Exp. I (J) Exp. 1 SE Sig. (I—J/SD)
Completeness Inferential Skeletal 0.03 0.15 0.40
treatment (Table 17). This effect size indicates that subjects in the inferential group were 5% more accurate
(0.40 x 0 .1 1) in completing their tasks during the treatment than were their counterparts in the skeletal
group.
The hypotheses for this research study compared different implementations o f minimalism during
different vertical "time slices'”: pretest, treatment, posttest, delayed posttest. O f additional significance for
practitioners and educators is the horizontal perspective afforded by analyzing each online help system
across all four testing times. For Hypothesis 1, such an analysis would address the question o f whether or
not any learning occurred given any o f the different implementations o f minimalism.
A multivariate repeated measure analysis was conducted to evaluate the differences in subjects'
performance outcome scores (completeness, accuracy, errors committed, percentage o f errors corrected,
error recovery time) over four different points in time (pretest, treatment, posttest, delayed posttest).
Statistically significant differences (F = 46.84, g = 0.000) were obtained for a combination o f all
An additional pairwise comparison (using the Bonferroni approach) o f the performance measures
across time revealed several statistically significant differences. As shown in Table 21, the greatest effect
sizes were associated with the differences in performance measures between the pretest and the treatment;
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
81
20
20
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
82
Table 20
the next greatest effect sizes were associated with the differences in performance measures between the
A trend analysis (Table 22) was conducted to focus the analysis and mathematically model the
form o f the subjects' performance scores across the different time periods. There were strong statistically
significant differences (Table 22) along each o f the trend lines. Figures 42 through 46 illustrate the
statistically significant differences within the performance scores over time. As suggested by the data in
Table 2 1 and illustrated in Figures 42. 4 3,44, and 46. the sharp differences in scores between the pretest
and treatment and pretest and posttests and then the nearly negligible differences in scores between the
posttest and delayed posttest suggest that the data are best represented by linear and quadratic trend models.
The cubic trend model better represents Figure 45, percentage o f errors corrected.
Hypothesis 2 stated that "While learning procedural tasks, subjects who use skeletal help (S) will
complete tasks more quickly than subjects using elaborative help (E), who will complete tasks more
quickly than subjects using inferential help (I).” Hypothesis not supported at the 0.05 level o f confidence.
Hypothesis 3 stated that "Subjects who used inferential help (I) to learn tasks will complete those
tasks more quickly than subjects using skeletal help (S), who will complete tasks more quickly than
subjects using elaborative help (E).” Hypothesis not supported at the 0.05 level o f confidence.
"M ore quickly’ (for both hypotheses) was m easured by tracking the start and stop time (as entered
by the subjects) for each task in the pretest, treatment, posttest, and delayed posttest.
First, the interaction between the group and the time measure was assessed to ensure that the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
83
Table 21
(table continues)
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
84
.90
.8 0 -
.7 0 -
-O
<U
0) .60-
Q.
f—
C
o
O
CO .5 0 - Help 1
-SC
O)
CO
h- Skeletal
° .40-
c Inferential
0)
au .
0
CL 30 Elaborative
Pretest Treatment Posttest Delayed Posttest
TEST
Figure 42. Trend: Percentage o f tasks completed.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
85
Table 22
group regression lines were parallel to the overall regression lines (Table 23). There was a statistically
significant interaction (F = 4.50. g = 0.01) during the treatment, which suggested that time measure during
the pretest differed enough from the time measure during the treatment to cause biased results. However, a
statistically significant difference in time between the treatment and the pre- and posttests is expected since
the pre- and posttests were time controlled, and the treatment was not. Therefore, the use o f analysis o f
Three univariate analyses o f variance were then conducted to evaluate the relationship between
types o f online minimalist instruction and time. The pretest score on the time measure was used as the
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
86
90
80
.70
>.
o
CD
Help 1
3
O
U
< Skeletal
4—
o
c Inferential
(D
O
q! 50 Elaborative
P retest Treatment Posttest D elayed Posttest
TEST
Figure 43. Trend: Percentage o f accuracy in task completion.
12
10
8
■o
0)
E
E
o 6
O
C/5
u. Help 1
o
Skeletal
V *—
o 4
<D Inferential
n
E
3
2 ____ Elaborative
Pretest Treatment Posttest D elayed Posttest
TEST
Fiuure 44. Trend: Num ber o f errors committed.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
87
1.00
Help 1
ow
w .70
LU Skeletal
o
c Inferential
d)
u\_
(U
0. Elaborative
P re test T reatm ent P o sttest D elayed P o stte st
TEST
Figure 45. Trend: Percentage o f errors corrected.
700
600
500
(A
400
</)
(D
E
I - 300 Help 1
&
<D
> Skeletal
0
flj 200
cr1 Inferential
O
u.
UJ 100 E laborative
P re test T reatm en t P o sttest D elayed P o stte st
TEST
Figure 46. Trend: Error recovery time.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
88
Table 23
eovariate to measure any differences in subjects’ performance after exposure to the testing materials. Table
24 represents the combined output o f the univariate analyses o f time as applied to Experiment 1 (skeletal,
inferential, and elaborative) for the treatment, posttest, and delayed posttest. There were no statistically
significant differences among the groups at the 0.05 level o f confidence in time to complete the tasks.
The variability o f the time measure was constrained during the time-controlled pretest and
posttests. Subjects who completed the tasks prior to the allotted time continued to tweak their work until
stopped by the lab consultant: subjects who did not complete the task were also stopped at the designated
time, resulting in similar completion times for all subjects. The data coders considered using the observed
ending time (rather than the ending time as entered by the subject) for the time measure, but decided
The hypotheses for this research study compared different implementations o f minimalism during
different vertical "time slices”: pretest, treatment, posttest, delayed posttest. O f additional significance for
practitioners and educators is the horizontal perspective afforded by analyzing each online help system
across all four testing times. For Hypotheses 2 and 3, such an analysis would address the question o f
whether or not subjects could decrease the time taken to complete tasks given any o f the different
implementations o f minimalism.
A multivariate repeated measure analysis was conducted to evaluate the differences in subjects'
performance outcome scores (time) over four different points in time (pretest, treatment, posttest, delayed
posttest). A statistically significant differences (F = 112.66, p = 0.000) was obtained (Table 25), which was
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
89
Table 24
Table 25
An additional pairwise comparison (using the Bonferroni approach) o f the time tc complete tasks
revealed several statistically significant differences (Table 26). The findings that are relevant to this
research are the differences (or lack thereof) among the timed tests: pretest, posttest, and delayed posttest.
There was no statistically significant difference between the posttest and pretest even though subjects had
nearly 14 more minutes in the posttest to complete tasks. Instead, they were able to complete tasks during
the posttest within about 24 seconds o f the time (0.22 x 65.9 / 60) they took to complete the same tasks in
the pretest. Thus, a nonsignificant finding here actually is important. The other finding o f interest is the
statistically significant difference (g = 0.03; MDES =0.71) in time to complete tasks between the posttest
and delayed posttest. After a I-week delay, subjects could complete the tasks within just over a minute o f
the time it took to complete the same tasks the previous week. This finding, coupled with the fact that
subjects completed about the same amount o f tasks with nearly the same amount o f accuracy, is relevant.
A trend analysis (Table 27) was conducted to focus the analysis and mathematically model the
form o f the time it took for subjects to complete tasks across the different time periods. There were strong
statistically significant differences (Table 27) along each o f the trend lines; however, the data from the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
90
Table 26
Table 27
Hypothesis 4 stated that "Inferential-proximal (P) online help will be more effective in helping learners
identity and correct errors than inferential-distant (T) online help, which will be more effective than
inferential-without (W) online help (i.e., P > T > W).” Hypothesis not supported at the .05 level o f
1. Errors committed: the number o f errors the subject made while com pleting a task
2. Percentage o f errors corrected: the number o f corrected errors divided by the total number o f
errors committed
3. Error recovery time: the total time spent correcting errors per experim ent and activity.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
91
3000
20 0 0 -
tn
o
(D
w
<n
-S£
(/)
ro
TEST
Fitmre 47. Trend: Time to complete tasks.
Analysis o f covariance was the statistical test selected in order to best determine the magnitude o f
differences among the groups after exposure the testing and treatment materials. Therefore, the interaction
between the group and the performance measures was assessed to ensure that the group regression lines
were parallel to the overall regression lines (Table 28). The absence o f an interaction between the
independent variable and the covariates means that regression lines are parallel, that the adjustment
employed by the analysis o f covariance procedure is appropriate, and that analysis o f covariance results
may be interpreted straightforwardly. Since none o f the interaction terms was statistically significant, this
assumption was met to a reasonable degree, and use o f analysis o f covariance was appropriate for the data.
Three different multivariate analyses o f variance were then conducted to evaluate the relationship
between types o f online minimalist instruction and specific, error-related performance measures. The
pretest scores on the above three measures were used as the covariate to measure any differences in
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
92
Table 28
Table 29 represents the combined output o f the multivariate analyses o f the three measures listed
treatment, posttest, and delayed posttest. No statistically significant differences were obtained (at the 0.05
level o f confidence) regarding the presence or location o f error information and the effect o f that error
information on decreasing the number o f errors subjects committed, increasing the percentage o f errors
they corrected, or decreasing the amount o f time they spent correcting errors.
The hypotheses for this research study compared different implementations o f minimalism during
different vertical "tim e slices” : pretest, treatment, posttest, delayed posttest. O f additional significance for
practitioners and educators is the horizontal perspective afforded by analyzing each online help system
across all four testing times. For Hypothesis 4. such an analysis would address the question o f whether or
not subjects could decrease the number o f errors they committed, increase the percentage o f errors they
corrected, or decrease the amount o f time they spent correcting errors given any o f the different
implementations o f minimalism.
A multivariate repeated measure analysis was conducted to evaluate the differences in subjects'
performance outcome scores (errors committed, percentage corrected, error recovery time) over four
different points in time (pretest, treatment, posttest, delayed posttest). A statistically significant difference
An additional pairwise comparison (using the Bonferroni approach) o f the error performance
measures revealed several statistically significant differences (Table 31). The findings that are relevant to
this research are the differences and effect sizes between the pretest and each o f the posttests. Subjects
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
93
Table 29
Table 30
committed approximately 4 fewer errors (0.62 x 5.95) during the posttest (p = 0.001. SMDES = -0.62) and
approximately 2.25 fewer errors (0.42 x 5.33) during the delayed posttest (p = 0.031. SMDES = -0.42) than
the> did during the pretest. There were no statistically significant differences in the amount o f time taken to
Although subjects corrected approximately 12% more errors (0.50 x 0.23) during the delayed
posttest (p = 0.031, SMDES = -0.42) than they did during the pretest, the statistically significant difference
among the three groups during the pretest (see Figure 48) suggests sampling error and therefore makes any
comparison differences suspect. However, the scores from the subjects in the inferential-without group (the
most divergent during the pretest) were comparable to the scores from the other groups during the
A trend analysis (Table 32) was conducted to focus the analysis and mathematically model the
form o f the time it took for subjects to complete tasks across the different time periods. There were strong
statistically significant differences (Table 32) along each o f the trend lines; however, the data from the
timed tests best fits a cubic trend model (Figures 4 8 ,4 9 , and 50). O f real interest here also is the fact that
the subjects in the proximal group corrected less errors overall than did the subjects in the other group
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
94
Table 31
(Figure 48) although their correction rate nearly parallels that o f subjects in the inferential-without group.
In other words, the subjects who had no error information at all corrected more errors than did the subjects
w ho had error information located right next to the error-prone steps. In fact, they corrected even fewer
errors during the posttest than did their counterparts in the inferential-without group.
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
95
.90
.80-
L-
L_ Ir
o
O
(/)
L_
Help 2
o
i5 60 ■ Without
o
c ^ Distant
<u
o
<u
0. Proximal
P retest Treatment Posttest Delayed Posttest
TEST
Fitzure 48. Trend: Percentage o f errors corrected over time.
Table 32
R e p ro d u c e d with perm ission of the copyright owner. Furth er reproduction prohibited without permission.
96
20
18
16
-o 14
aj
1
£ 12
o
O
(/)
W. Help 2
o1
I—
— 10
UJ
.«— Without
o
k-
(U 8
-Q ^ Distant
E
2
6 ___ Proximal
Pretest Treatment Posttest Delayed Posttest
TEST
Figure 49. Trend: Number o f errors committed over time.
1000
800-
600-
(0
o
CD
U)
^ 400-
i- Help 2
^ f
<u [
Without
g 200-
<u
DC Distant
Proximal
Pretest Treatment P osttest Delayed P osttest
TEST
Figure 50. Trend: Error recovery time over time.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
97
Hypothesis 5 stated that "Subjects who prefer a verba! style o f learning will complete tasks more
accurately and efficiently using skeletal online help.” An attempt was made to associate an instructional
style with verbal aptitude. "A ccurately and efficiently” refer to the combination o f the following measures:
1. Time: the start and stop time for each task as entered by the subject
2. Completeness: the ability to complete the given task (see Figure 37 for a sample o f the data
coding sheet listing the five tasks and the steps to complete them)
3. Accuracy: the exactness by which the subject’s method for completing a given task matches
the method outlined in the online help systems (see Figure 38 for a sample o f the related coding sheet)
First, univariate analyses o f variance were conducted for each outcome measure (time,
completeness, accuracy) for the pretest, treatment, and posttests to evaluate any aptitude treatment
interaction between verbal learning scores and type o f online instruction. The fixed factors were the
different Experiment I treatment groups: skeletal, inferential, and elaborative. The covariate was the
subjects' verbal learning scores. The main interaction effects for the outcome measures between verbal-
learning score and online help system for each o f the testing phases are identified in Table 33.
Simple scatterplots (Figures 51 and 52) were created to illustrate the magnitude o f the interactions
relative to each treatment group. Linear regressions were then conducted for each group to generate the
intercept and the slope coefficients for the group regression lines. The slope and intercept data for each
A statistically significant interaction between time, preference for verbal learning, and type o f
online help was obtained (F = 4.28: p = .02) during the pretest (Table 33). A medium effect size ( q - = .08)
and intermediate level o f power (1 - P = 0.74) associated with the interaction suggest that the occurrence o f
this interaction was probably due to sampling error. That is, the groups differed from the target population
along some characteristics (even though they were randomly assigned to the various groups). However, the
obtained significant interaction with tim e was not maintained across the treatment or posttests.
R e p ro d u c e d with perm ission of the copyright owner. Furth er reproduction prohibited without permission.
98
Table 33
Time (mins) Pretest 597.06 298.53 2.00 4.28 0.02 0.08 0.74
Figure 51 illustrates the relative magnitude and direction o f interaction between subjects' verbal-
learning scores and time to complete tasks that contributed to the main interaction effects that occurred
during the pretest. There was an inverse relationship between time and verbal learning preference
associated with the skeletal group and a positive correlation between time and verbal learning preference
associated with the elaborative and inferential groups. That is, subjects in the skeletal group who scored
higher on the verbal learning measure took less time to complete tasks during the pretest, whereas subjects
in the elaborative and inferential groups who scored higher on the verbal learning measure took more time
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
99
1100
^ 1000 -
c/3
o
0
C/3
0 900-
E
K
re
£ 800- ' ^>4 Help 1
XX
C/3
0 * Elaborative
0
700- Y = 7 77.27 + 3.32X
CL
Inferential
c
| 600- Y = 7 89.57 + 3.56X
o3 * Skeletal
Q.
QJ 500. Y = 825.08 - 3.51X
-10 0 10 20
c
0
CJ
<D
Q.
<U
O
O
CO
>N ■ Help 1
o
03
w
3 * Elaborative
O
u
< Y = 0.91 -0 .0 1 X
* Inferential
c
<D Y = 0.90 - 0.01X
E
3 ■ ■ ■ i
0 * Skeletal
Q.
X
0 Y = 0.76 + 0.01X
-10 0 10 20
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
100
As shown in Table 33, a statistically significant interaction between accuracy, verbal learning
preference, and type o f online help was obtained during treatment (F = 6.68, g = 0.002). A large effect size
(H~ = 0 .11) and level o f power ( I - (3 = 0 .9 1) indicate that over 10% o f the subject’s ability to complete a
task accurately was associated with his or her preference for a verbal learning style. In addition, there was
less than a 10% chance that the interaction obtained from this sample would have occurred by chance.
Figure 52 illustrates the relative magnitude and direction o f interaction between subjects’ verbal-
learning scores and accuracy scores that contributed to the main interaction effects that occurred during the
treatment. There was a positive relationship between accuracy scores and verbal learning preference
associated with the skeletal group and an inverse relationship between accuracy scores and verbal learning
preference associated with the elaborative and inferential groups. That is, subjects in the skeletal group who
scored higher on the verbal learning measure were more accurate in task completion during the treatment,
whereas subjects in the elaborative and inferential groups who scored higher on the verbal learning measure
Because there was a statistically significant interaction between the groups during the pretest, a
further exploration o f the demographic makeup o f the groups was warranted. The focus o f the investigation
was on the skeletal group and the ways in which those subjects differed from the subjects in the inferential
and elaborative groups. As shown in Table 34, subjects in the skeletal group tended to differ from the other
subjects along the characteristics o f years o f computer experience and level o f specific experience in Word
and WordPerfect. However, there was no real trend in the differences. For example. Figures 53, 54. and 55
illustrate that although some o f the subjects in the skeletal group had less general and specific computer
knowledge than did the subjects in the other two groups, others in the skeletal group either matched or
exceeded the top level o f general and specific computer experience possessed by subjects in the other two
groups. There did not appear to be any statistically significant differences among the groups in regard to
their entrance characteristics. However, a multivariate analysis o f variance was conducted (Table 35) to
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
101
Table 34
Experiment I groups
Hypothesis 6 stated that “Subjects who prefer a visual style o f learning will complete tasks more
accurately and efficiently using Elaborated online help.” In other words, an attempt was made to associate
an instructional style with visual aptitude. “Accurately and efficiently” refer to the combination o f the
following measures:
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
102
14-
12-
10-
Help 1
S keletal
Inferential
E laborative
Missing o 0-1 1-2 2-5 5-10 o v e r 10
20-
Help 1
Skeletal
o
<D
•Q Inferential
E
3
2 Elaborative
None Beg Med Adv
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
103
30
Skeletal
Inferential
Elaborative
None Beg Med Adv
1. Time: the start and stop time for each task as entered by the subject
2. Completeness: the ability to complete the given task (see Figure 37 for a sample o f the data
coding sheet listing the five tasks and the steps to complete them)
3. Accuracy: the exactness by which the subject’s method for completing a given task matches
the method outlined in the online help systems (see Figure 38 for a sample o f the related coding sheet)
Univariate analyses o f variance were conducted for each outcome measure (time, completeness,
accuracy) for the pretest, treatment, and posttests to evaluate any aptitude treatment interaction between
visual learning scores and type o f online instruction. The fixed factors were the different Experiment 1
treatment groups: skeletal, inferential, and elaborative. The covariate was the subjects' visual learning
scores. The main interaction effects for the outcome measures between visual-learning score and online
help system for each o f the testing phases are identified in Table 36. There were no statistically significant
interactions at the .05 level o f confidence between preferred visual learning style, online help system, or
performance measures.
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
104
Table 35
-)
Test df F Sig- n- Power
Table 36
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
105
CHAPTER VI
The general purpose o f this research was to define ways in which minimalism, an instructional
methodology designed for print-based training materials, could be effectively modified and applied to
online help systems. Two separate experiments (using different sets o f online help systems) were designed
to evaluate different principles o f minimalism. The first experiment was focused on comparing variations
o f minimalist instruction for increased learning. The second experiment was focused on identifying the
effect that presence and location o f error-recovery information had on subject error commission and
recovery’. A related purpose o f this research was to determine if a subject’s performance in learning a
software application would increase if his or her preference for learning style would be matched to an
instructional approach. The summary o f this study, conclusions drawn from findings, and further
recommendations based on findings from this research are presented in this chapter.
This summary section includes the statement o f the problem, statement o f purpose, research
Effective, well-written, helpful documentation strongly influences the perceived effectiveness o f the
software by users (Greer, 1984; Rosenbaum, 1998) to the point that documentation can “make or break a
software application" (Greer, 1984, p. 7). Inadequate or poorly written online help results in user frustration
with and possible rejection o f the software (Kearsley, 1985b; Lieberman, 1987) and possible loss o f
business (Rosenbaum, 1998). Technical communicators are starting to apply the principles o f minimalism,
an instructional design approach used to develop training manuals, to the developm ent o f online help
systems. The implementation o f minimalist principles to online help instruction could result in increased
learning o f the subject matter content and acceptance o f the software application; however, the application
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
106
o f minimalism to training materials has met with mixed reviews, and its application to online help has not
been evaluated thoroughly. Research on the application o f the principles o f minimalism to online help is
needed to assist technical communicators in the design and development o f effective online help
instruction: the results o f this research could also be extended to the development o f any online instruction.
The purpose o f this research was to define ways in which minimalism, an instructional
methodology designed for print-based training materials, could be effectively modified and applied to
online help systems, which are computer-based and typically informational and referential. Two separate
experiments (using the same sample population) were conducted with the following general hypotheses:
1. Users require (and prefer) less instructional support to complete and learn software
2. Users who are prompted to interact with the application spend longer amounts o f time
learning tasks, but, once learned, complete tasks faster without instructional support.
3. Users will be more effective and efficient at identifying and correcting errors if they have
4. Users using an online help system that is matched to their learning style will complete
software application procedural skills more quickly and with fewer errors than subjects using a nonmatched
Research Procedures
A sample o f 170 subjects for this research was obtained from the clerical pool o f Kelly Temporary
Services (KTS). where they completed prequalification testing in M S Word 97 tasks, a background survey,
and a learning preference survey. Subjects were randomly assigned to two different experimental
conditions. The data from 111 subjects were used for this research.
All sessions for this research were conducted in the same College o f Business microcomputer lab
at Utah State University. Loaded onto each computer were two sets o f online help systems for M S Word 97
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
107
( for a total o f five different online help systems), a screen-capture application, and text and graphic
Subjects created four different newsletters (pretest, treatment, posttest, and delayed posttest) which
were comprised o f five tasks identified as intermediate or advanced in difficulty by the KTS training guide.
Experiment 1 consisted o f the first two tasks (create a banner heading; insert and format text into columns);
Experiment 2 consisted o f the last three tasks (insert and format a graphic, create and format a table, create
a data source and mailing labels for a mail merge). Online help instruction was available only during the
treatment newsletter. Subjects used one online help system to complete the first two tasks and a different
online help system to create the remaining three tasks: lab consultants switched the online help systems for
the subjects. After completing the fourth newsletter (the second week), subjects evaluated one o f the two
Data Analysis
MS Cam corder£>, a Microsoft computerized screen capture software, was used to record subjects’
mouse moves and keyboard strokes as they completed each o f the experimental activities. The resulting
data were saved as an .avi file and burned onto CD-ROMs. Coding checklists were used to record all
SPSS 10.0 was used to analyze the data. Multivariate analyses o f covariance (followed by
univariate analyses o f variance and pairwise comparisons for statistically significant findings) were used to
determine if one online help system was more effective than another in assisting subjects in both learning
and remembering how to complete software application procedural tasks. Bivariate correlations were used
to identity any relationships among subjects’ preferred learning style, type o f online help system used, and
Univariate analyses o f variance with subjects’ preferred visual or verbal learning style were
conducted to assess the effect on performance outcomes associated with an interaction between preferred
learning style and type o f online instruction. Linear regressions were used to plot the slope and intercept for
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
108
Summary o f Results
There were four general findings associated with the four general research hypotheses. Each
finding and general hypothesis is listed below followed by the summary o f results for each hypotheses.
Finding 1
It was generally hypothesized that users would require (and prefer) less instructional support to
complete and learn software application procedural skills. Specifically, it was predicted that subjects who
used the inferential online help system (least instructional support) would have better completion and
accuracy scores than subjects using either the elaborative or skeletal online help systems. The findings were
mixed. Although not quite statistically significant at the 0.05 level o f confidence, the mean accuracy scores
for subjects using inferential online help were greater during the teaching'treatment session than the
accuracy scores for subjects using skeletal online help. Instead, the mean completion scores for subjects
using elaborative online help (most instructional support) were statistically significantly greater during the
teaching treatment session than the completion scores for subjects using skeletal online help. This finding
between the pretest and posttest on the following performance outcome measures: completeness, accuracy,
number o f errors committed, and amount o f time taken to recover from errors. In other words, subjects
completed more tasks, completed them more accurately, committed fewer errors, and corrected the errors in
less time after exposure to any o f the three minimalist online help systems.
F inding 2
It was generally hypothesized that users who are prompted to interact with the application would
spend longer amounts o f time learning tasks but. once learned, would complete tasks faster without
instructional support. Specifically, it was predicted (Hypothesis 2) that while learning procedural tasks (i.e.,
during the treatment), subjects who used skeletal help would complete tasks m ore quickly than subjects
using elaborative help, who would, in turn, complete tasks more quickly than subjects using inferential
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
help. It was also predicted (Hypothesis 3) that subjects who used inferential help during the treatment to
learn tasks would complete those tasks more quickly during the posttests than subjects who used skeletal
help, who would, in turn, complete tasks more quickly than subjects who used elaborative help. Instead,
this was not the case. There were no statistically significant differences among the treatm ent groups.
between the posttest and pretest, which means that even though subjects had nearly 14 more minutes in the
posttest to com plete tasks, they were able to complete tasks during the posttest in the same amount o f time
they took to complete the same tasks in the pretest. Thus, a nonsignificant finding here actually is
important. The other finding o f interest is that after a I-week delay, subjects could complete the tasks
within ju st over a minute o f the time it took to complete the same tasks the previous week. This finding,
coupled with the fact that subjects completed about the same amount o f tasks with nearly the same amount
o f accuracy, is relevant.
FitnJinu 3
It was generally hypothesized that users would be more effective and efficient at identifying and
correcting errors if they had direct access to error information. Three online help systems were developed:
one without any error information (without), one with error information located in a '"Troubleshooting"
section (distant), and one with error information located right next to the error-prone steps (proximal). It
was predicted (Hypothesis 4) that subjects in the proximal group would outperform subjects in the other
two groups. Instead, the presence and location o f error information did not seem to effect the number o f
errors subjects committed, the percentage they corrected, or the time they spent recovering from errors.
between the pretest and posttests on the number o f errors committed. In other words, subjects committed
fewer errors after exposure to one o f the three minimalist online help systems. O f special interest was the
fact that the subjects in the proximal group corrected less errors overall than did the subjects in the other
groups even though they had error information located right next to the error-prone steps. Their rate o f
errors corrected paralleled that o f subjects in the inferential-without group (without any error information)
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
110
Finding 4
It was generally hypothesized that users who used an online help system that was matched to their
learning style would complete software application procedural skills more quickly and with fewer errors
than subjects who used a nonmatched type o f online help. Specifically, it was predicted that subjects who
preferred a verbal style o f learning would complete tasks more accurately and efficiently using skeletal
online help (Hypothesis 5), and subjects who preferred a visual style o f learning would complete tasks
more accurately and efficiently using elaborated online help (Hypothesis 6).
An interaction between verbal learning score, time to complete tasks, and online help system was
obtained during the pretest (that is, before the subjects were actually exposed to any online help system),
which indicated that groups differed along this attribute prior to exposure to the different treatments.
Subjects in the skeletal group were different from the subjects in the other two groups: there was inverse
correlation between learning score and time to complete tasks for subjects in the skeletal group. This
difference among the groups was also evident in the interaction between accuracy and verbal-learning
measure as assessed during the treatment: there was a positive correlation between learning score and
accuracy o f task completion for subjects in the skeletal group. However, because the skeletal group differed
in the pretest, the differences in their scores on the accuracy measure during the treatment are suspect.
There were no interactions between visual learning score, the performance measures, and online
Conclusions
Based on the findings from the research hypotheses, the following conclusions were made:
1. As evidenced by the equivalent posttest scores related to the completion and accuracy o f tasks
among the three groups, it is possible to say that, based on the way the subjects used the different help
systems, the different applications o f minimalism to online instruction were fairly equivalent in their overall
effect on learning. That is, although the online help systems were different in the amount and type o f
instructional support provided, the subjects’ use o f the features which were consistent across the systems
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
Ill
and nonuse o f the features which varied across the systems may have resulted in the systems appearing
In addition, based on the increase in scores from the pretest to the posttests related to the
completion and accuracy o f tasks and decrease in scores related to the number o f errors committed and time
taken to recover from errors, it is possible to say that learning did occur. That is, given a strong base o f
minimalism, the slight design differences preferred by different instructional designers (e.g., providing
more or less explanations, feedback, graphic support) will make no real difference to the end users: they
2. As evidenced by the lack o f any statistically significant findings associated with the time it
took for subjects in the treatment groups (who were intermediate-level users) to learn or complete tasks
( Finding 2) during the untimed testing period, it is possible to say that time is not a differentiating factor
among more experienced users using different implementations o f minimalist online help.
3. As evidenced by equivalent pretest and posttests times to complete tasks with increased
completion and accuracy rates, it is possible to say that using minimalist instruction increases the speed
4. As evidenced by the lack o f any statistically significant findings associated with the presence
and location o f error information in online instruction (Finding 3). it is possible to say that the placement o f
error information (either next to the error-prone step(s) or in a distant location) may not be beneficial in
reducing the number o f errors users commit or helping them recover from errors they do commit.
However, lack o f use o f error information may actually be an issue o f access rather than o f
minimalist content. Getting subjects to access error information by clicking on a button or hypertext link
appeared to be a problem. Therefore, one may conclude that providing separate error information via button
or hypertext link may be ineffective in assisting subjects in preventing or recovering from errors.
and Experiment 2 (Finding 1 and Finding 3), it is possible to say that subjects who are exposed to a solid
base o f minimalist instruction will leam to complete more tasks with fewer errors.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
112
6. As evidenced by lack o f interaction between time, completeness, and accuracy measures and
both verbal- and visual-learning preference scores for subjects in the two treatment groups that did not
differ in the pretest assessment (Finding 4), it is possible to say that a subject’s preference for a verbal- or
Discussion
research may have accounted for the lack o f statistically significant findings in support o f one
Taking a more minimalist approach to this study may have been beneficial. This research effort
was an attempt to complete on one grand scale in a condensed time period what Carroll (1990) and Black et
al. (1987) completed iteratively on a smaller scale over different time periods. Carroll (1990) used 19
subjects in three 8-hour work days for Test 1. used 32 subjects in a 5-hour time period for Test 2. joined
with Black et al. (1987) and used 20 subjects in about a 2-hour time period for Test 3. and used six subjects
in three 8-hour workdays for Test 4. Carroll revised and refined his experimental conditions between each
experimental session to test a different principle o f minimalism or to refine the testing o f the same
principle. The current research used the data from 111 subjects (which, with a 2.5% unemployment rate in
the valley, took 6 months to obtain), gathered in one 4- to 5-hour time period at one session and another 1-
In addition, where Carroll focused on one set o f principles for each test, this research was an
attempt to split the experimental session into two different experiments. The first two tasks the subjects
completed were associated with the application o f Carroll’s (1990) and Black and others’ (1987) skeletal,
elaborative, and inferential manuals to online instruction. The last three tasks o f the same activity were
associated with the Principle 3, support error recognition and recovery, as described by Van der Meij and
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
113
Carroll (1995) and tested by Lazonder (1994). It may have been more beneficial to run a series o f smaller
The sample population for this research is typical o f today’s intended target audience, but is quite
different from the sample population used by Carroll (1990), Black et al. (1987), and Lazonder ( 1994).
Whereas the early experiments testing the different implementations o f the principles o f minimalism used
subjects with little or no computer experience (i.e.. nearly novice users), this research indicated that the
typical users o f online instruction to learn intermediate- to advanced-level tasks in an office setting would
be 18 to 25 years o f age, have up to 10 years o f computer experience, possess at least an intermediate level
o f experience in the given application, and own their own Windows platform computer. This great
difference in sample populations may have accounted for much o f the lack o f statistical differences.
Slight differences in the design o f good instruction make little difference to a target audience with
intermediate to advanced computer skills: their ability to leam and complete computer-based tasks
outweighs slight variances in the instruction. The fact that subjects felt confident enough in their own
ability' to explore and leam from the interface without (as Carroll would say) letting the instruction get in
the way o f learning, lends support to the theory o f minimalism. Subjects used the base o f minimalist
instruction in the different online help systems and tended to ignore the different instructional components
that varied among the systems. Instead, they interacted directly with the system to leam how to complete a
given task.
Samplinu Error
Although great care was taken to randomly assign subjects to the different experimental groups,
there was a statistically significant difference among the groups associated with the percentage o f errors
corrected (inferential-without group was different). This difference in groups was not maintained across the
treatment or posttests. The difference during the pretest could have been attributed to either sampling error
or coding error (although no coding errors were located after a cursory investigation). The statistically
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
114
significant differences here caused the statistically significant differences on this measure between the
Another sampling-error limitation was associated with the statistically significant interactions
related to subjects’ scores on the verbal-learning measure. Although there were no statistically significant
differences among the groups on visual- or verbal-leaming scores, there were statistically significant
differences among the groups during the pretest. Subjects in the skeletal group differed from the other two
groups on the interaction between verbal-leaming score and completion rates: subjects in the inferential
group differed from the other two groups on the interaction between verbal-leaming score and accuracy
rates. It is possible that the groups differed along some other characteristic that was not identified by any of
Time Factor
Related to the first limitation, overall experimental design, the time factor may have been another
limitation o f this study. For the pretest, subjects were given the amount o f time it took an expert (about 32
minutes total) to complete all five tasks. Subjects then took a short refreshment break before going back
into the lab for Activity 2, which lasted between 2 and 3 hours. Subjects generally took a quick break about
one to one and one-half hours into Activity 2 while the lab consulted saved the first video and loaded the
second help system. Subjects then completed the last three tasks, which took approximately one to one and
one half hours. By the time subjects began Activity 3 (the immediate posttest), they had been in a learning
situation for approximately two and one half to three hours with little nutritional support (the sessions
started at 6 p.m.: many subjects came directly from work). Subjects made mistakes during Activity 3 on
tasks that they had executed correctly in the previous activity, which suggests that perhaps the internal
validity' threat o f testing effect (i.e., subjects were getting tired) had an effect on performance outcomes.
Another time-related limitation was the amount o f time allocated for the posttests (which was
determined in large part on the budget: an additional 15 minutes per subject multiplied by all the subjects
was very' expensive). The posttests were each 45 minutes long, divided proportionally among the five tasks.
The resulting time per task was one and one half the time it would take an expert to complete the same
task. This may have created an artificial ceiling for some. Others completed the tasks well within the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
115
allotted time, but then spent the remainder o f the time tweaking their work. The data coders discussed the
option o f ending the time upon observed completion o f task (which may have resulted in statistically
significant differences for time: Hypothesis 2), but decided against that option because it would introduce
too much o f an instrumentation threat to the internal validity o f the study (Campbell & Stanley, 1963).
Instead the stopping time for each timed task was the time the subject entered as the stopping time.
Experimental Mortality
Campbell and Stanley (1963) defined experimental mortality as the "differential loss o f
respondents from the comparison groups” (p. 5). A quick analysis o f the 22 subjects who forfeited their pay
for the first session by either not showing up for the second session or leaving the first session before
completing the tasks revealed that a disproportionate amount o f these subjects (12) were in the inferential
group for Experiment 1. However, it is highly unlikely that the inclusion o f their scores would have
changed the direction o f the outcome. That is, the overall scores o f the inferential group would not have
increased with the inclusion o f the score o f those who left the experiment.
In light o f research ethics, the researcher did not provide as much emotional support as perhaps
was possible. Carroll (1990) mentioned that three subjects became too frustrated with the study and were
excused. A similar situation occurred with at least three subjects in this research (presumably more if one
takes into account those that did not return the following week). O f the three known subjects, two were
struggling to the point o f tears (one just sat staring at her computer, unable to even move) and the
researcher quietly excused them from the study. The third subject broke into hysterical tears after the
pretest and would not even reenter the computer lab to retrieve her purse. All attempts to console her
If subjects were frustrated enough to leave or not return (even when it meant forfeiting their
payment), then it could be assumed that they did not leam the software application tasks. Ethically, the
researcher should have provided an alternative method o f instruction to ensure that subjects learned how to
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
116
The following recommendations were based on the findings and conclusions o f this study in
1. Since there was a lack o f statistically significant differences (supporting the hypotheses)
among the treatment groups, then future research is recommended to test the different implementations o f
minimalism on a larger scale. Subjects were only required to complete two tasks in the current study
t Experiment I ). Future research may require that subjects complete all five tasks using either the skeletal,
2. Again, since there was a lack o f statistically significant differences (supporting the
hypotheses) among the treatment groups, then future research is recommended with an emphasis on the
quantity and type o f learning that occurs. That is, given any type o f minimalist online instruction, will users
be prompted to explore and leam their software application better? Will they leam how to perform known
3. Since the presence and location o f error information made no difference to performance
outcome (Finding 3. Conclusion 3), then future research is recommended to identify better ways to get
subjects to access the error information. The conflict arises between placing the error information directly
on the screen (where the subject could see it immediately) and keeping the contents o f the screen brief (by
having the subject access the error information via a hypertext link).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
117
REFERENCES
Ackerman-Efron, J. (1985). States and reflective eye movements in the vertical dimension as an index o f
cognitive function in a verbal or visual modality. [On-line], Dissertation Abstracts Item. 46-06B. 2053.
Black. J. B., Carroll. J. M.. & McGuigan, S. M. (1987). What kind o f minimal instruction manual is the
most effective? In H.J. Bullinger, B. Shackel & K. Komwachs (Eds.), Proceedings o f the second IF1P
Blair. J. D. (1980). Effects o f a self-administered audio-taped program on image awareness, use, vividness
and control as reported by college students. [On-line]. Dissertation Abstracts Item. 41-03 A. 0940.
Boggan. S., Farkas, D.. & Welinske, J. (1996). Developing online help for Windows 95. London:
Borg. W.R., & Ball. M.D. (1989). Educational research (5th ed.). New York: Longman.
Brockman. J. R. (1990a). The why, where and how o f minimalism. SIGDOC Asterisk. J_4(4), 11 l- l 19.
Brockman, J. R. (1990b). Writing better computer user documentation: From paper to hypertext. New
Campbell, D. T„ & Stanley, J. C. (1963). Experimental and auasi-experimental designs for research.
Carlson. P. A. (1992). From document to knowledge base: Intelligent hypertext as minimalist instruction.
Carroll, J. M. (1987). Five gambits for the advisory interface dilemma. In M. Frese, E. Ulich, & W. Dzida
(Eds.), Psychological issues o f human computer interaction in the work place (pp. 257-274). North
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
118
Carroll, J. M. (1990). The Numberg funnel: Designing minimalist instruction for practical computer skill.
Carroll. J. M„ Mack. R. L., Lewis. C. H.. Grischkowsky, N. L., & Robertson, S. R. (1985). Exploring a
Carroll. J. M.. Smith-Kerker. P. L., Ford. J. R„ & Mazur-Rimetz, S. A. (1987). The minimal manual.
Cline. J. A. (1991). Cognitive stvle in system design (Report No. IR 053 868). Unpublished master’s thesis.
Kent State University, Kent, OH. (ERIC Document Reproduction Service No. ED 343 586)
Dem. R. W. (1982). Learning instructions and cognitive coding in depressed and nondepressed college
students. [On-line]. Dissertation Abstracts Item. 43-05B. 1610. Abstract from: DAI Accession No.:
AAG8223271
DutYy. T. M.. Palmer. J. E.. & Mehlenbacher, B. (1992). Online help design and evaluation. Norwood, NJ:
Ablex.
Farkas. D. K. (1993). The role o f balloon help. The Journal o f Computer Documentation. 17(2), 3-19.
Fishel. A. K. (1984). Clinical effects and personality correlates o f being a creative person and participating
in creative activity. rOn-linel. Dissertation Abstracts Item. 45-08B. 2685. Abstract from: DAI
Frese. M.. Albrecht, K.. Altmann, J. L., Papstein, P. V., Peyerl, R„ Prumper, J.. Schulte-Gocking, H.,
Wankmuller, I.. & Wendel, R. (1988). The effects o f an active development o f the mental model in the
training process: Experimental results in a word processing system. Behaviour and Information
Gagne'. R. M. (1985). The conditions o f learning and theory o f instruction. Chicago: Holt, Rinehart &
Winston.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
119
Garro, L. C. (1982). Imagery’s role in cognition: Integrating imagery theory with an individual differences
approach. [On-line]. Dissertation Abstracts Item. 43-07B. 2371. Abstract from: DAI Accession No.:
AAG8227832
Green. S. B.. Salkind, N. J.. & Akey, T. M. (1997). Using SPSS for Windows: Analyzing and
Greer. M. (1984). A trainer’s guide to software documentation. Performance and Instruction Journal. 23(3),
6-7.
Grice. R. A., & Ridgway. L. S. (1995). Presenting technical information in hypermedia format: Benefits and
Hackos. J. T„ & Stevens. D. M. (1997). Standards for online communication. New York: Wiley.
Hailey. D. E. Jr.. & Hailey, C. E. (1997). Hypermedia, multimedia, and reader cognition: An empirical
Mallgren. C. (1992). Numberg funnel: A minimal collection. The Journal o f Computer Documentation.
16(1). 11-16.
Horn. R. E. (1992). Commentary on the Numberg funnel. The Journal o f Computer Documentation. 16(1).
3-11.
Horton. W. K. ( I990). Designing and writing online documentation: Help files to hypertext. New York:
Wiley.
Horton. W. K. (1994). Designing and writing online documentation: Hypermedia for self-supporting
44(2). 234-239.
Jonassen, D. H.. & Wang, S. (1993). Acquiring structural knowledge from semantically structured
Kearslev. G. (1985a, May). Helps, simulations, and intelligent tutors: An overview o f embedded training.
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
120
Kearsley. G. (1985b). Embedded training: The new look o f computer-based instruction. Machine-Mediated
Kini. A. S. (1993). Effects o f cognitive learning style and verbal and visual presentation modes on concept
Dissertation Abstracts Item. 54-08A. 2954. Abstract from: DAI Accession No.: AAG9403528
Kirby. J. R., Moore, P.J.. & Schoefield, N.J. (1988). Verbal and visual learning styles. Contemporary
Kleinbaum. D. G.. Kupper, L. L.. & Muller. K. E. (1988). Applied regression analysis and other
Kuchler. C. J. (1983). Hemispheric laterality and cognitive style. [On-line], Dissertation Abstracts Item. 44-
Lawless. K. A.. & Brown. S. W. (1997). Multimedia learning environments: Issues o f learner control and
Lazonder. A. W. (1994). Minimalist computer documentation: A study o f constructive and corrective skills
Liebenman. J. (1987). Implication o f adult learning characteristics and learning styles for the design o f
McKnight. C.. Dillon, A.. & Richardson. J. (1996). User-centered design o f hypertext/hypermedia for
education. In D. Jonassen (Ed.). Handbook o f research for educational communications and technology
Mirel. B. (1998). Minimalism for complex tasks. In J. M. Carroll (Ed.), Minimalism bevond the N um bers
Nichols. M. C. (1994). Using style guidelines to create consistent online information. Technical
Nowaczyk. R. H.. & James, E. C. (1993). Applying minimal manual principles for documentation o f
graphical user interfaces. Journal o f Technical Writing and Communication. 23(4). 379-388.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
121
Palmiter, S. Elkerton, J., & Baggett, P. (1991). Animated demonstrations vs. written instructions for
34*687-701.
Pantin. H. M. (1982). The relationship between subjects’ predominant sensory predicate use, their preferred
representational system and self-reported attitudes towards similar versus different therapist-patient
dyads. [On-line], Dissertation Abstracts Item. 43-07B. 2350. A bstract from: DAI Accession No.:
AAG8229208
Parzivand, A. (1984). Effects o f habitual modes o f thinking and stress on production o f creative responses.
[On-line], Dissertation Abstracts Item. 46-02B. 0678. Abstract from: DAI Accession No.:
AAG8507881
Pedhazur. E. J.. & Schmelkin, L. P. (1991). Measurement, design, and analysis. Hillsdale. NJ: Erlbaum.
Pratt, J. A. (1998). Where is the instruction in online help systems? Technical Communication. 45(1). 33-
37.
Price. J. (1993). Comments on balloon help. The Journal o f Computer Documentation. 17(2). 21-22.
Reddish. J. C. (1988). Reading to leam to do. The Technical Writing Teacher. 15(3), 223-233.
Reznich. C. B. (1993). Minimalist design principles applied to the problem o f computer anxiety (Doctoral
dissertation. Michigan State University, 1993). Dissertation Abstracts International. 54(10 A), A AC
9406544.
Rogers. H. G.. & Brown, F. W. (1993). The impact o f writing style on compliance with instructions.
Rosenbaum, S. (1998). Follow-up on training in minimalism: How are technical communicators using
minimalism? In J. M. Carroll (Ed.), Minimalism beyond the N um berg funnel (pp. 119-147).
Snapshot. (1998. Spring). Stages: The Fidelity Investments Magazine o f Personal Finance, p. 3.
means o f CAI (algebra, mathtiles, manipulatives, visualizer). [On-line]. Dissertation Abstracts Item. 46-
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without perm ission.
122
Stevens. J. (1996). Applied multivariate statistics for the social sciences (3"* ed.). Mahwah, NJ: Erlbaum.
Talburt, J. R. (1993). Comments on "The role o f balloon help” by David Farkas. The Journal o f Computer
Vanderlinden. G., Cocklin. T.G., McKita, M. (1988.). Testing and developing minimalist tutorials: A case
history. Paper presented at the 35th International Technical Communications Conference. Washington.
DC.
Van der Meij, H.. & Carroll. J. M. (1995). Principles and heuristics for designing minimalist instruction.
Van der Meij, H.. &. Carroll. J. M. (1998). Principles and heuristics for designing minimalist instruction. In
J. M. Carroll (Ed.), Minimalism bevond the Numberg tunnel (pp. 19-53). Cambridge. MA: M IT Press.
Warner, J. W. (1989). The effectiveness on adult learners of minimalism design theory on learning, attitude,
anxiety, and time-on-task for self-study learning environments (Doctoral dissertation. Michigan State
Wenger, M. J.. & Spyridakis. J. H. (1993). Reduced text structure at two text levels: impacts on the
performance o f technical readers. Journal o f Technical Writing and Communication. 23(4). 333-352.
http://www.documentation.com/winheIp
R e p ro d u c e d with perm ission of the copyright owner. Fu rther reproduction prohibited without permission.
123
APPENDICES
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
124
Appendix A
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
125
Table X
Black. J. B„ Compared four Inferential (I) and Skeletal groups N/A, but the
Carroll. J. M.. different types o f completed the overall learning task 24 findings are in
& McGuigan, manuals: inferential, and 18 minutes, respectively, faster than minutes for ease
letter task.
(table continues)
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without perm ission.
126
Frese et al. steps alone vs. steps Performance: Guided exploration (GE)
(1988) with mental model could use the commands taught in the
diagrams vs. guided course better than Steps (S) (A = 1.5) 1.5
basic word processing better and in less time than S (A = 1.2) 1.2
Lazonder la. minimal manual MM took less time to train than SSM -1.10
(table continues)
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
127
quickly .5
procedural skills
training/practice .51/.8
testing .59
testing 67
(table continues)
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
128
knowledge scores.
tasks
tasks
computer skills
(table continues)
R e p ro d u c e d with perm ission of the copyright owner. Furth er reproduction prohibited without permission.
129
Cocklin. & (plus procedures, when (GE+P) spent 10% more time working
(1988) study tutorials to leam GE+P had less problems and required
Performance Achievement:
(table continues)
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
130
Attitude:
Anxiety:
.36 36
.69 69
Time on Task:
1.45
b. VL faster than VH. A = 1.45
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
131
Appendix B
Training Outline
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
132
PinPoint
The Power o f Learning at Work
Word 97
Training Outline
R 1 2 /9 7 Word 97 Training
© 1997 Kdly S e r v i c e s . Inc
R e p ro d u c e d with perm ission of the copyright owner. Fu rther reproduction prohibited without permission.
Appendix C
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
p 't o r - 'i £ d
Instruction in Microbunts:
OEPAitTstENrc?iNsrsLcncH^L TcCM,sct.c<:T^e Study o f Minimalist Principles
College o» Educmon Applied to Online Help
logon. uunM J” .:jJU
ty«pl'Qnei3i>l I
fixiaon.~ir-:si]
6/5/91
Dear Participant:
Utah S ate University a iavalved la a research study to improve die eSedyesess of anBee help systems. You were
selected lor this study because of your registration in die derieal pool o f the Kedy Services temporary employment agency.
Your participation in this study is based on the assumptions chat you are (a) familiar with a word processing application
(e.g., MS Ward or WordPerfect) and (b) not an expert in all the advanced funeioas of.US Ward You may be ineligible to
participate in this study if you are either x.ocperienced with computers and ’word processors or demcnsoare a high level o f
expertise.
Your involvement in this study is comprised of completing some iatbnaationai questionnaires, learning and completing
some new word processing sksils, and evaluating an online help system. You will csmplem these acdvities over two
sessions, scheduled one week apart, at a Utah State University computer lab. The first session is scheduled to last
between four and five hours (although you may be able to complete it -in three hours): the second session is scheduled to
last approximately one and one-half hours. You m ust complete each session in order :o paid:
You may withdraw from this study at any time without ronsequer.ee I f you are a Utah State University student, then
withdrawal will not arTrc: your grades in any way. However, please keep in mind Sat payment for Sis temporary
employment Srough Kelly Services is contingent upon your completion o f hath sessions, a week apart.
Your concdendaiicy in this study 'will be maintained in the following manner
♦ You will use a special number instead o f your name on ail dccumemanon. This number will be assigned to you
by Kelly Services and will be similar to OCO-OQ-OlOl Tae research personnel wi3 idem!?/ you by that number
only and your responses to anything in the study will be associated with that number only The only association
between your name and your assigned number will be in the Kelly Services: secured database tor payment
purposes. .Vote: you must use your assigned number when checking in at bed: sessions o f the srody in order to
be paid by Kelly Services.
♦ Ad scores, responses and data collected from your participation in tae study wiE be kept :n a locked ale cabinet
at the researcher's home omce. Only the researcher will have access to the ale cabinet All informittcn collected
at Keily Services wiil be secured In their employee ale cabinets until retrieved by the researcher
♦ Payment to you for mis temporary employment will be made directly through Keily Services.
B enefit to you. We appreciate your paructpiticn In this study 1: is our hope h at you will beaesc mom It by learning
some word-processing skills that will make you more marketable. There is no risk to you associated wtth this study
Flense sign both eopies and keep one for your records.
Agreem ent fo r participation. 1 foteae er-.it same hcrei ... have read the above
research descnpr.cn and consent to participate Ia both sessions o f this srody of oniine heip systems. I further agree to
allow all data generated by me and collected by computer or surveys to be used in the research sctdy
Signature of Participant
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
135
Appendix D
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
136
1. G ender: M F
2. Age: 18-25 26-30 31-35 36-40 41-45 46-50 51-55 56-60 61-65 o v e r6 5
3. C o m p u ter Experience: 6-12 months 1-2 years 2-5 years 5-10 years over 10 years
4. W ord processing applications used (please check all that apply and circle level o f expertise):
G rap h ics/d raw in g applications (please check all that apply and circle level o f expertist
6. O th e r types o f com puter applications (please name and circle level o f expertise):
7. C o m p u ter platfo rm with w hich I am most fam iliar: Macintosh PC/IBM clone
R e p ro d u c e d with perm ission of the copyright owner. Furth er reproduction prohibited without permission.
137
Appendix E
Verbaiizer-Visuaiizer Questionnaire
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
138
Agree — Disagree
1. I enjoy doing work that requires the use o f words. 4 5
2. 1 don't believe that anyone can think in terms o f mental
pictures.
3. I enjoy learning new words. 4 3
4. I find illustrations or diagrams help me when I'm reading. 4 5
1 have a hard time making mental pictures o f a place that 4 5
I've only been a few times.
6. 1 can easily think o f synonyms for words. 4 5
7. I read rather slowly. 4 5
8. I seldom use diagrams to explain things. 4 5
9. I like newspaper articles that have graphs. 4 5
10. 1 prefer to read instructions about how to do something 4 5
rather than have someone show me.
I don't like maps or diagrams in books. j 4 5
12. I have a better than average fluency in using words. 2 3 4 5
13. 1 spend little time attempting to increase my vocabulary. 2 j 4 5
14. When 1 read books with maps in them, 2 3 4 5
I refer to the maps a lot.
15. I dislike word games like crossword puzzles. 2 3 4 5
16. The old saying "A picture is worth a thousand words" is 2 3 4 5
certainly true for me.
17. 1 dislike looking words up in the dictionary. 2 3 4 5
18. 1 have always disliked jigsaw puzzles. 2 3 4 5
19. I find maps helpful in finding my way around a new city. 2 3 4 5
20 . I have a hard time remembering the words to songs. 7 j 4 5
Note: I will put your learning preference score in your file for you in case you are curious.
Thanks for your participation.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
Appendix F
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
140
O verview
WelcomeI Thank you fo r participating in this study.
This orientation packet is designed to guide you through the activities for this session. We will preview the
session activities as a group before you start. You may refer back at any time to the instructions included in
this packet.
S e s s io n A ctivities
During today's session you will complete three very similar activities using different content, but comprised
o f the same five tasks:
All three activities you complete today resemble partially-completed newsletters. A description o f each
activity follows:
• Create a portion o f a sales report (see enclosed sample): use mail merge to create mailing labels. (35
minutes)
The intent o f this first activity is to introduce you to intermediate- and advanced-level skills in MS
Word that will make you more employable. You will attempt to complete five tasks in MS Word
without any help. You will have 35 minutes to complete this newsletter/sales report.
Note: Although each task is timed, feel free to work at your own pace. Note: it is not expected that you
know how to complete each task— these are intermediate and advanced-level tasks in MS Word. It's
OK if you do not know how to complete them now— you will be able to use the online help system to
complete these same tasks in the second portion o f this session. Just do the best you can.
* Create a newsletter; use mail merge to create mailing labels, (approximately 2 hours)
This second activity is to enable you to leam the same MS Word skills using an online help system.
Note: this is the only time you will have any instruction to leam these new skills. Everything you need
to know is included in the online help system. The online help was developed specifically for this study
so is limited to just the help you need. Using the online help system will prevent you from experiencing
a lot o f frustration as you attempt to leam these skills.
You have two hours to complete all five tasks in this second activity. Please pace yourself so that you can
complete all five tasks and have time to review the skills prior to the third activity. You will not have any
online help available to you during the third activity, so you will want to spend some time reviewing the
skills prior to starting the third activity. Activity 3 takes about 50 minutes to complete. Leave yourself
enough time to complete it before the time you are scheduled to leave.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
141
■ Create a portion o f a newsletter; use m ail merge to create m ailing labels. (SO m inutes)
This third activity provides you with an opportunity to demonstrate your new knowledge by
completing the same MS Word skills without the aid o f an online help system. N o online help will
be available for this activity.
4. Most o f the text and all the graphic files for the newsletter are located in the Newsletter folder on the
hard drive. The file names are printed next to the corresponding text and graphics in the sample
newsletter. T he Newsletter folder is on the C:\ drive!
5. Please follow the instructions for each o f the activities. Make sure you have completed all the tasks
before you notify a lab consultant that you are through.
6. You will be directed to insert the time periodically throughout this ID# 123
session. D ate: 6/16/1999
S ta rt T ask 1: 13:59:03
a. Place your cursor next to the corresponding task number in the text End T ask 1: 14:04:25
box in the upper left comer.
S ta rt T ask 2: 14:04:45
b. Click on Insert | Date and Time. End T a sk 2: 14:08:03
S ta rt T ask 3: 14:08:25
c. Make sure there is no ch eckm ark in the Update A utom atically
End T a sk 3: 14:13:41
box!
S ta rt T ask 4: 14:13:54
d. Click on the very last time option (for example, 13:00:05) and click End T a sk 4: 14:26:57
O K . You will end up with something like this box in the upper left S ta rt T ask 5: 14:27:13
com er o f your screen. End T a sk 5: 14:35:35
7. There is a Stop button in the upper right comer o f your screen. You may click on the title bar and move
the Stop button, but do not click on th a t button. If for some reason the computer closes the recording
application, then notify a lab consultant immediately.
8. You will be prompted to save your w ork as you go.
It is very important that you include y o u r ID# (e.g. 15) in the filename o f all documents— use a /, 2, or 3 on
different documents to prevent overwriting earlier documents by the same name (e.g., I5listl.doc,
I5list2.doc. lSlist3.doc).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
142
S a v e a ll d o c u m e n t s in t h e C : \N e w s le tte r fo ld e r .
h. The lab consultants can only help you with computer problems. The online help system has the answers
to everything you need to do: it was developed specifically for this study— p le a se u se th e
online help system. In the first and third activities (those without online help), ju st do the best
you can to figure out or remember how to complete the given task.
10. We will be taking two five-minute breaks today: one between each o f the activities. Your lab consultant
will direct you to the restrooms and water fountains.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
143
Creating a portion o f a newsletter and some mailing labels to mail it is comprised o f five main tasks. There
is a certain amount o f time allotted to complete each task. Just work at your own pace to complete each
task. If you do not know how to complete a particular task, think about how you might complete it and try
that. However, don't spend too long trying to figure out how to do any one part o f a task. If you can't figure
out how to do something, then ju st complete the task as best as you can.
I sample copy o f the partial sales report is enclosed in this packet. You may use it as a guide.
G etting S ta rte d :
l. Open the start.doc document (located in the Newsletter folder).
c. Scroll down (or over) until you find the Newsletter folder.
d. Double-click on the Newsletter folder to open it.
e. Find and double-click on the start.doc file to open it.
2. Save the document as 000sales.doc in the C:\Newsletter folder (where 000 IS YOUR ID#).
a. In Word, click on File | Save As ...
b. Click on the broken up-arrow folder untiiy cu see the B115 (C :) drive.
c. Scroll down (or over) until you find the Newsletter folder.
d. Type " o o O s a l e s . d o c " (where 000 IS YOUR ID#) in the File Name field.
3. Type your ID# (e.g., #54, no* your social security number) in the text box at the top o f the page.
4. Position your icon next to “D ate:” in the text box and insert the date from Insert | D ate and Time.
Use the very first date format (6/18/1999).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
144
I. Create a text box spanning from the border o f the date/time text box to the right margin and as
long as the date/time text box (i.e., align the bottom borders o f the two text boxes).
6. Apply to the text box a tight text wrap with a 0.1 external margin.
9. Place the star over the "Sales Report” text box as shown.
10. Make the "W e're # 1 ” text white, Arial, 14 point font, bold, centered.
12. Apply the same texture and border as the “Sales Report” text box.
14. Position text box over “Sales Report” text box as shown.
Stop:
Position your cursor after “ E nd T ask I” and insert the time from Insert | Date and Time.
Use the very last time format (14:30:25).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
145
Position your cursor after ’‘S tart Task 2” and insert the time from Insert | Date and Time. Use the
very last time format (14:30:25).
1. Bring in some existing text (use the M yD rink.doc file in the Newsletter folder); place it below the
text boxes.
5. Place the remainder o f the text into three columns using the following dimensions:
6. Make the last line o f text ("Table o f Quarterly Sales") and the remaining blank lines into a
one-column heading.
S. Place the cursor at the beginning o f the "Table o f Quarterly Sales" heading and insert a blank
line.
Stop:
Position your cursor after “ End Task 2” and insert the time from Insert | Date and Time. Use the very
last time format (14:30:25).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
146
Position your cursor after “S ta rt T a sk 3” and insert the time from Insert | D ate an d Time. Use the
very last time format (14:30:25).
1. Position your cursor anywhere in the "Soft Drink Market" section o f the text.
3. Resize the graphic so that it fits better in the text (use the sample as a guide).
4. Apply a tight text wrap to the graphic; edit the wrap points so the text contours the graphic.
5. Position the "drink" graphic so that the text wraps uniformly around it.
Stop:
Position your cursor after “ E nd T ask 3” and insert the time from Insert | D ate and Time. Use the
* SAVE y o u r d o c u m e n t
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
147
Position your cursor after “S tart Task 4” and insert the time from Insert | Date and Time. Use the
very last time format (14:30:25).
1. Insert a few blank lines below the "Table o f Quarterly Sales" heading text.
2. Create the following table below the "Quarterly Sales" heading— pay attention to
I Vi point external b o rd er,'/: point internal border. 'A point double-lined internal border
cell merges
10% shading
Stop:
Position your cursor after “ End Task 4” and insert the time from Insert | Date and Time. Use the very
last time format (14:30:25).
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
148
Start:
Position your cursor after "S ta rt T ask 5” and insert the time from Insert | Date and Time. Use the
very last time format (14:30:25).
1. Use the Mail Merge Helper to create a list o f addresses for mailing labels in a new main
docum ent.
2. Save the mailing list as "000listl.doc" (WHERE 000 IS YOUR ID#) in the C:\Newsletter folder.
3. Format the mailing list as you would like it to be printed on 2160 Mini-address Avery Standard
labels.
5. Merge the labels to a new docum ent: save as "000m aill.doc" (where 000 IS YOUR ID#).
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
149
c. Scroll down (or over) until you find the N ewsletter folder.
d. Type " 0 0 0 m a i l l . d o c " (where 000 IS YOUR ID#) in the File N am e field.
7. Save the new main document (Document/V) as 000m aitil.doc (where 000 IS YOUR ID#);
close it.
Stop:
Position your cursor after ” End T ask 5” and insert the time from Insert | D ate an d Xime. Use the very
last time format (14:30:25).
******PLEA SE r a is e y o u r h a n d t o s u m m o n a l a b c o n s u l t a n t *******
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
150
N ew sletter S c e n a rio
You have been hired temporarily (through Kelly Services) to work for a small, locally owned and operated
travel agency. The duties you normally perform in situations like this typically include invoicing customers,
paying suppliers, and preparing company correspondence. In this situation, however, the ow ner o f the
company wants you to publish a promotional newsletter for a January vacation get-away. Publishing a
newsletter involves many o f the skills that you could also use to complete other tasks (e.g., using mail
merge to prepare the form letters and mailing labels for customer invoicing).
You have about two hours to complete this small newsletter. Work at your own pace.
* use th e o n lin e help system to leam new skills to be more efficient in performing other tasks.
Remember...
■ the text and graphics for this newsletter are contained in the Newsletter folder on the hard drive
* use the enclosed checklist to make sure you complete all tasks
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
151
Getting Started
l. Open the newsletter.doc file in the Newsletter folder on the hard drive.
2. Save the document as 000news.doc in the C:\Newsletter folder (where 000 IS YOUR ID#).
d. Type " 0 0 0 n e w s .d o c " (where 000 IS YOUR ID#) in the File Name field.
3. Type your ID# in the text box at the top o f the page.
4. Position your icon after “ D ate:” and insert the date from Insert | Date and Time.
Start:
Position your cursor after “S ta rt T ask 1” and insert the time from Insert | D ate and Tim e. Use
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
152
b. "Travel" text box = Arial, bold, 28 point, centered, black on white, 2.25 pt. border
c. "January 1999" text box = Arial, bold, 10 point, all caps, rotated text; 3 pt. double
lined border
d. "Special Issue" AutoShape text box = Arial, bold, 12 point, centered, AutoShape
Stop:
Position your cursor after “ E nd T ask 1” and insert the time from Insert | Date an d Tim e. Use the
* SAVE y o u r docum ent— rem em ber to save often while you work!
* Proceed to Task 2.
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
153
Start:
Position your cursor after “S ta rt T ask 2” and insert the time from Insert | D ate an d X>me. Use
2. Bring in first the fre e p o rtd o c and then the dolphins.doc text files
a. Create a one-column heading from the first line o f text “Freeport - Island o f Action
and Relaxation”
b. Change the heading font type and size to Times New Roman, bold, italic,18 point
c. Place the rest o f the text in three columns (2.17, 2.4, and 2.37) with .27 and .29
spacing
d. Delete the blank line before the first paragraph o f the "Freeport" article
e. Make the last line o f text ("Xanadu Resort") into a one-column heading.
Stop:
Position your cursor after “ End T ask 2” and insert the time from Insert | D ate a n d Time. Use the
* SAVE y o u r docum ent— rem em ber to save often w hile you work!
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
154
Start:
Position your cursor after “S ta rt T ask 3” and insert the time from In sert | D ate and Time. Use
b. Position "dolphins.bm p” behind the "Travel” box: align the waterline and bottom box
border
d. Edit the wrap on palm .bm p so that it goes in between each palm leaf
Stop:
Position your cursor after “ End T a sk 3” and insert the time from In sert | D ate and Jim e. Use the
* Proceed to Task 4.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
155
Task 4: Add and format table (replicate the table in the sample)
Start:
Position your cursor after “S ta rt T ask 4” and insert the time from Insert | Date and Tim e. Use
a. Format the text as shown (i.e., 12 point. Times New Roman and Arial, bold headings)
b. Left-align text in the left column; center text in the right column; center the titles
f. Make the internal ceil borders '/: pt. single and double-lines
Stop:
Position your cursor after “ End T ask 4” and insert the time from In sert | D ate and Tim e. Use the
* SAVE your d o c u m e n t
* Proceed to Task 5.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
156
Start:
Position your cursor after " S ta rt T ask 5” and insert the time from In se rt | Date and Tim e. Use
1. Use the Mail Merge Helper to create a list o f addresses for mailing labels in a new m ain
docum ent.
Gerald Trunnell
4352 W 5200 N
Richmond, UT 84333
Serena Hoffman
1814 E 1200N
Providence, UT 84332
Ailene Gregory
434 N. Main St.
Cedar Valley, UT 84013
2. When prompted, save the mailing list as 000list2.doc (where 000 IS YOUR ID#) in the
C:\Newsletter folder.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
157
3. Format the mailing list as you would like it to be printed on 2160 Mini-address Avery Standard
labels.
4. Sort the mailing list in descending order by postal code
5. Merge the labels to a new document; save as "000mail2.doc" (where 000 IS YOUR ID#).
d. Type " 0 0 0 m a il2 .d o c " (where 000 IS YOUR ID#) in the File Name field.
6. Close 000mail2.doc (the mailing labels) and return to the main document (looks like this:)
«FirstName» «LastName»
«Address I»
«City». «State» «PostalCode»
7. Save the new main document (Document#) as 000main2.doc (where 000 IS YOUR ID#); close
c. Scroll down (or over) until you find the Newsletter folder.
d. Type " 0 0 0 m a in 2 .d o c " (where 000 IS YOUR ID#) in the File Name field.
Stop:
Position your cursor after "E n d T a sk 5” and insert the time from In sert | D ate an d Tim e. Use the very
last time format (14:30:25).
* SAVE y o u r d o c u m e n t
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
158
If NO. then please review the procedures in online help until you feel confident o f your ability.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
159
You now have the opportunity to demonstrate the skills you learned by creating a small portion o f a
newsletter and some mailing labels without the use o f an online help system. W ork at your own pace to
complete each task. If you do not know how to complete a particular task, think about how you might
complete it and try that. However, don't spend too long trying to figure out how to do any one part o f a
task. If you can't figure out how to do something, then just complete the task as best as you can.
.1 sam ple copy o f th e partial newsletter (taken fr o m Utah State University's H ope H ealth Letter) is
enclosed in this packet. You m ay use it as a guide.
Getting Started:
b. Click on the broken up-arrow folder until you see the B115 (C :) drive.
=D B115 (C:)
c. Scroll down (or over) until you find the Newsletter folder.
d. Double-click on the Newsletter folder to open it.
e. Find and double-click on the end.doc file to open it.
2. Save the document as 000hope.doc (where 000 IS YOUR ID#) in the C :\N ew sletter folder.
d. Type " o o o h o p e . d o c " (where 000 IS YOUR ID#) in the File N am e field.
3. Type your ID# in the text box at the top o f the page.
4. Position your icon after “ D ate:” and insert the date from Insert | Date a n d T im e.
5. Raise your hand to let the lab consultant know you are ready to start Task I .
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
160
Start:
Position your cursor after “S ta rt T ask 1” and insert the time from In sert | D ate and Time. Use
1. Create a text box spanning from the border o f the date/time text box to the right margin.
6. Create a smaller text box with the same solid green fill, but with no border.
9. Apply a tight text wrap with a 0.1 external margin to the "Hope" text box
10. Create the “News for You!” AutoShape text box: Arial, 12 point, centered.
Stop:
Position your cursor after “ End T ask 1” and insert the time from In sert | D ate and lim e . Use the
* SA V E y o u r d o c u m e n t
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
161
Start:
Position your cursor after “S tart Task 2” and insert the time from Insert | Date and Time. Use
2. Bring in some existing text. (Use the ltealth.doc file in the Newsletter folder.)
3. Use the first line o f text I" Research News... You Can Use!") as a one-column heading.
5. Format the remainder o f the text into three columns using the following dimensions:
a. Columns 1 and 3: 2.5 inches wide with a 0.25-inch space between columns
6. Make the last line o f text (“Analysis o f Popular Diets...") it into a one-column heading.
Stop:
Position your cursor after “End Task 2” and insert the time from Insert | Date and Time. Use the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
162
Start:
Position your cursor after "S tart Task 3” and insert the time from Insert | Date and Time. Use
4. Position the picture in the text so that it overlaps any two columns.
5. Apply a tight text wrap; edit the wrap so that the text wraps in a circle around the picture.
Stop:
Position your cursor after “End Task 3” and insert the time from Insert | Date and Time. Use the
* SAVE your do c u m e n t
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
163
Start:
Position your cursor after “Start Task 4” and insert the time from Insert | Date and Time. Use
Merged cells
Text direction
Shading: 15% for title. 10% for "Bad" category, 5% for "OK” category
Stop:
Position your cursor after “End Task 4” and insert the time from Insert | Date and Time. Use the
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
164
Start:
Position your cursor after '‘S ta r t T a sk 5” and insert the time from In sert | D ate and Time. Use the
very last time format (14:30:25).
I. Use the Mail Merge Helper to create a list of addresses for mailing labels in a new main
docum ent.
3. Format the mailing list as you would like it to be printed on 2 160 M ini-address Avery Standard
labels.
4. Sort the labels in descending order by Postal-code.
5. Merge the labels to a new docum ent: save the mailing labels as "000mail3.doc" (where 000 IS
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
165
7. Save the new main document (DocumentAO as 000main3.doc (where 000 IS YOUR ID#); close it.
Stop:
Position your cursor after "E n d T ask 5” and insert the time from Insert | Date and Tim e. Use the
Thank you very much for participating in the first half o f this study.
The second half o f the study will be conducted in one week.
Please check with your Kelly Services representative for your scheduled time.
Note: you will be paid only after participation in both th is an d next w eek's
se s sio n .
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
166
Appendix G
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
167
O verview
Welcome back to the second session o f this study.
This orientation packet is designed to guide you through the activities for this session. We will preview the
session activities as a group before you start. You may refer back at any time to the instructions included in
this packet.
S e s s io n A ctivities
During today's session you will have an opportunity to demonstrate your employable knowledge o f
intermediate and advanced word-processing skills in Microsoft Word. You will create a portion o f a
newsletter, completing the sam e five tasks you completed last week while learning advanced skills in Word.
After you finish the partial newsletter, you will have an opportunity to evaluate the online help system that
you used last week. You w ill have access to that online help system to assist you in this evaluation. A more
detailed description o f today's activities follows:
a. Although each task is timed, feel free to work at your own pace. Try to remember what you learned
from the previous session o f MS Word using an online help system.
b. You will be com pleting MS Word tasks without the use o f an online help system.
c. Wait to start the first task until a lab consultant directs you to begin.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
168
I. You will be directed to insert the time periodically throughout this session
ID# 123
a. Place your cursor next to the corresponding task number in the text D ate: 6/16/1999
box in the upper left comer. S ta rt T ask 1: 13:59:03
End T ask I: 14:04:25
b. Click on Insert | D ate and Time. S ta rt T ask 2: 14:04:45
End T ask 2: 14:08:03
c. Make sure there is no checkm ark in the U pdate Automatically S ta rt T ask 3: 14:08:25
box! End T a sk 3 : 14:13:41
S ta rt T ask 4: 14:13:54
d. Click on the very last time option (for example, 13:00:05) and E nd T ask 4: 14:26:57
click O K . You will end up with something like this box in the S ta rt T ask 5: 14:27:13
upper left com er o f your screen. End T a sk S : 14:35:35
2. There is a Stop bunon in the upper right comer o f your screen. You may click on the title bar and m ove
the Stop button, but do not click on th at button. If for some reason the com puter closes the recording
application, then notify a lab consultant immediately.
It is verv important that you include your ID# (e.g. 15) in the filename o f all documents.
S a v e a ll d o c u m e n t s in t h e C : \N e w s le tte r fo ld e r .
4. The lab assistants are here to help you with com puter problems. They will not help you perform the
tasks. Please do not ask them for specific task-related assistance. Just try to remember how to complete
each o f the tasks from what you learned last week.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
169
You have another opportunity to demonstrate the skills you learned by creating a small portion o f a
newsletter and some mailing labels without the use o f an online help system. Work at your own pace to
complete each task. If you do not know how to complete a particular task, think about how you might
complete it and try that. However, don't spend too long trying to figure out how to do any one part o f a
task. If you can't figure out how to do something, then just complete the task as best as you can.
.1 sam ple copy o f the p a rtial newsletter (taken fr o m an actual newsletter) is enclosed in this p a c k e t You
m ay use it as a guide.
Getting Started:
c. Scroll down (or over) until you find the Newsletter folder.
d. Double-click on the Newsletter folder to open it.
e. Find and double-click on the fin a L d o c file to open it.
2. Save the document as 000safe.doc (where 000 IS YOUR ID#) in the C:\Newsletter folder.
b. Click on the broken up-arrow folder , ® | until you see the BUS (C:) drive.
c. Scroll down (or over) until you find the Newsletter folder.
d. Type " o O O s a f e . d o c " (where 000 IS YOUR ID#) in the File Name field.
3. Type your ID# in the text box at the top o f the page.
4. Position your icon after “ Date:” and insert the date from In sert | Date and Time.
Use the very first date format (6/18/1999).
5. Raise your hand to let the lab consultant know you are ready to start Task 1.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
170
Start:
Position your cursor after “Start Task 1" and insert the time from Insert | Date and Time. Use
1. Create a text box spanning from the border o f the date/time text box to the right margin.
2. Type and center the words "Safety News" in the text box.
4. Make the w ord ”News” Arial Black, 72 pt. font, bold, italics
5. Fill the text box with a texture: apply a special, black border to the text box.
7. Create another text box with the same texture fill and a special border.
11. Create the “#1 in Safety!” AutoShape text box: Arial, 12 point, bold, white.centered.
12. Fill the AutoShape with color or texture; apply a special border.
Stop:
Position your cursor after “End Task I” and insert the time from Insert | Date and Time. Use the
* SAVE your d o cu m en t
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
171
Start:
Position your cursor after “S tart T ask 2” and insert the time from Insert | Date and Time. Use
1. Position your cursor a few lines below the "Safety News" banner.
2. Bring in some existing text. (Use the safety.doc file in the Newsletter folder.)
3. Use the first line o f text (" Volunteers to Perform ...") as a one-column heading.
5. Format the remainder o f the text into three columns using the following dimensions:
a. Columns I and 3: 2.5 inches wide with a 0.25-inch space between columns
6. Make the last line o f text (“Schedule o f Inspections ") and remaining lines into a one-column
heading.
Stop:
Position your cursor after “End T ask 2” and insert the time from Insert | D ate and Time. Use the
* SAVE y o u r d o c u m e n t
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
172
Start:
Position your cursor after “S ta rt T ask 3” and insert the time from In se rt | D ate an d Time. Use
4. Position the graphic in the text so that it overlaps any two columns.
5. Apply a tight text wrap; edit the wrap so that the text wraps in the shape o f the graphic.
Stop:
Position your cursor after "E nd T ask 3” and insert the time from In se rt | D ate and Time. Use the
* SAVE y o u r d o c u m e n t
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
173
Start:
Position your cursor after "Start Task 4” and insert the time from Insert | Date and Time. Use
2 'A point external border with '/: point internal cell borders
Merged cells
Text direction
Stop:
Position your cursor after "End Task 4” and insert the time from Insert | Date and Time. Use the
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
174
Start:
Position your cursor after “S ta rt T ask 5” and insert the time from In sert | Date a n d Tim e. Use the
very last time format (14:30:25).
I. Use the Mail Merge Helper to create a list o f addresses for mailing labels in a new main
docum ent.
4. Merge the labels to a new docum ent: save the mailing labels as "000mail4.doc" (where 000 IS
c. Scroll down (or over) until you find the Newsletter folder.
d. Type "0 0 O m ai.14. d o c " (where 000 IS YOUR ID#) in the File Name field.
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
175
6. Save the new main document (Document/V) as Q00main4.doc (where 000 IS YOUR ID#);
close it.
Stop:
Position your cursor after “End Task 5” and insert the time from Insert | Date and Time. Use the
* SAVE y o u r d o cu m en t
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
176
Appendix H
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
177
O p e n in g Help
Your lab consultant will open the correct online help system for you. If you accidentally close your
online help system, please raise your hand and ask the lab consultant to reopen it.
Please do not reopen help on your own.
A c c e s s in g Help T o p ic s
This help system is designed to provide you with the greatest amount o f flexibility to get the
information you need in a hurry. Choose from the following methods o f accessing help to complete
any given task.
3. Double-click on any book ^ you think contains the needed information. (The book
"opens" ^ to display its "sheets" o f help information.)
4. Double-click on any help sheet to access the corresponding help information.
© Hint: not sure what to do next? Look to the Table o f C ontents for the correct order o f tasks.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
178
15inim^databa»iael(Bconwiendedl
^t~-M^piK8»earchcapab®iBt
?C ‘&itoiii»»eatdicapaMMes-1
Figure 1. Find Setup W izard: What you see after clicking on the Find tab
3. Click on Next.
Help Topics. Leominij MS Woid 9 /
zl
l4TopcsFctmd‘ Atwords.'BegstAuto:Pause
D-gptag,-.^ c [ Cancel-
Figure 2. Find W izard : W hat you see after clicking on the Next button.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
179
4. Type in the word or phrase that describes the task you want to complete.
5. Click on a word from the alphabetized list to narrow your search.
6. Double-click on the topic that most closely matches your task.
N avigating in Help
There are different ways to get around in help to find related topics. You can use any o f the most
frequently used methods:
<< >>
Browse b u tto n s =1 L — take you to the topics before and after the topic that is
currently displayed. Use this method to browse back and forth through the help system
after you access a closely related help topic.
g ack
Back button returns you in succession to previously displayed help topics.
H ypertext jum ps are green with a solid underline. They are typically preceded with a
Related Tasks: heading. Click on hypertext jum ps to go directly to related topics— use
the Back button to return.
Hypertext pop-ups are green with a dotted underline. Click on them to get additional
information.
D isplaying Help
Once your online help system is open, you can move or minimize it to provide more working space
on your screen.
^ U s i n g MS W o i d B E E
BEQE
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
180
Appendix I
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
181
I. Does the help system support different naming fo r tasks and com mands?
A. Are alternative menu systems available to reflect differing options when seeking help (for example,
alphabetical menus to support searching for a particular command, task-based menus that
correspond to real-world tasks, task-based menus to reflect computing tasks, or expert and novice
organizations)?
1. Yes, alternative representations are available, and the alternative organizations make a lot o f
sense.
2. Yes, alternative representations are available, but one or more o f the alternative organizations
is not very helpful.
3. There is only one menu system, but there are both tasks and commands on it.
4. There is only a command-based menu.
B. Does the help system enable the novice user to use familiar terminology to get to the relevant
topics?
1. An index or keyword search system (for example, a Find function) is available and it includes
an extensive list o f synonyms for both the functions inthe application program and the real-
world tasks comprised by these functions (for example, the "cut" and "paste" commands point
to the task "moving a paragraph").
2. Either the tasks or synonyms (but not both) are part o f a keyword search system or index, o r
both tasks and synonym-based access is available but very limited.
3. There is a glossary that defines terms in the application program in a manner that the novice
user can understand.
4. The help system does not contain a glossary, index, or keyword search system.
A. How many items are there on the main menu? Consider the main menu to be the first menu that has
a list o f actual help topics that the user may see. not an introductory menu offering navigational or
"branching" choices (for example, the "books” are navigational; the "pages" are main menu items).
1. 15 to 50
2. 5 1 to 70, or 8 to 14
3. More than 70, o r fewer than 8
4. There are no menus, ju st a keyword search system.
B. What is the average number o f items that are on the submenus (including embedded menus, that is,
words in the help text that act as menu choices)?
1. Between 7 and 15.
2. Fewer than 7 or more than 15.
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
182
2. No. In some menus, users must scroll or page to see all menu items - but never for more than
two screens.
3. No. In most menus, users must scroll or page to see all menu items - but never for more than
two screens.
4. No. In most menus there are more than two screens o f menu items.
Category 3: Format
III. Does the fo rm a l o f the help text facilitate searching fo r and understanding the needed
information?
A. If the help text frequently takes two or three screens per topic, but not more, how do you move
through the text?
1. Page through.
2. Scroll.
B. If the individual help texts tend to be more than three screens in length, as in an online document,
how do you move through the text?
1. Page through, and there's a visible menu available to jum p to particular subtopics.
2. Page through, but there's no menu available to jum p to particular subtopics.
3. Paging is not available; the user must scroll through the text. There is. however, a menu
available to support jum ping to particular subtopics.
4. No paging or menu o f subtopics is available; the user must scroll through the text.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
183
D. How much o f the screen appears to be given to margins and spacing between chunks o f
information? (Do not include spacing between lines in your estimate.)
1. Looks like at least 50%.
2. Looks like between 30% and 50%.
3. Looks like less than 30%.
E. Are different types o f information (command syntax, descriptions, examples, and so on) clearly
separated by spacing?
1. Frequently.
2. Sometimes.
3. Never.
F. Are headings or some form o f highlighting (such as underlining, boldface, indenting, or extra line
spacing) used to identify the different types of information in help text?
1. Almost always.
2. Frequently.
3. Occasionally.
4. Very seldom.
G. Are basic organization and formatting principles (layout, highlighting) applied consistently across
the different help texts?
1. Yes, the appearance is highly structured and consistent from topic to topic.
2. Yes. formatting principles are applied consistently, but not often (more structuring would be
helpful).
3. Formatting principles are applied, but not consistently.
4. No formatting principles were applied.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
184
Category 4: Content
C. Are there levels o f explanation (for example, quick reference and elaborated, pop-up explanations
or definitions, simplified and technical, and so on)?
1. Yes. and it's very complete and distinguishable.
2. Yes. but it's not very complete or different.
3. No.
For each o f the following questions, enter the appropriate score if the feature is present in the help system.
Score:
• Present as necessary in all help, score 1.
• Present for some help but should be in all, score 1.5.
• Absent but should be present, score 2.
H. Is a concrete example o f how a command is used presented with enough detail that you could
imitate it?
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
185
Category 5: Comprehensibility
F. Does the text refer to users with person pronouns (for example, "you") and use the imperative
("Press the Return key")?
1. Frequently.
2. Sometimes.
3 Infrequently.
G. Is the vocabulary, beyond the use o f command names and task names, unnecessarily difficult or
technical?
1. A lm ost always easy to understand.
2. Usually easy to understand.
3. Usually difficult to understand.
4. Alm ost always difficult to understand.
H. Does the help use functional graphics (functional graphics are used for informational purposes, not
just for motivational or aesthetic reasons)?
1. Yes, graphics are used consistently.
2. Yes, but only occasionally.
3. No.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
186
Category 6: Navioation
D. How long does it take to move from one help topic to another (or within the same general topic
area if the help is broken into topic areas)?
1. Less than 2 seconds.
2. Between 2 and 5 seconds.
3. More than 5 seconds.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
187
1II. Do the help system and the application program facilitate applying the information to the users’
tasks or problem s?
A. Can you transfer help information to the application program (for example, using cut and paste)?
1. Yes.
2. No.
B. Can you view the relevant parts o f the application program while in the help system? .......
1. Yes, windowing permits viewing both the help and the application program simultaneously.
2. Sometimes (there is an overlapping window with fixedplacement).
3. No, but you can switch between the help and the application screen easily.
4. No, you must leave help to see the application program.
C. Can you work in the application program while help is on the screen?
1. Yes. for all help content.
2. Yes, but only for a small portion o f the help content.
3. No.
D. Does the help use the same functions as the application program where appropriate (for example,
does it use the same movement commands or the same menu system)?
1. Yes, fully o r almost fully mimics the application program.
2. Somewhat, not consistent in all respects.
3. No.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
188
Category 8: Usefulness
1. Do you generally use the online help that comes with computer applications? Yes No
2. Did you use the online help to learn the Word tasks in this study? Yes No
3. Was the online help system used in this study helpful to you? Yes No
4. What was the most useful feature o f this online help system?
5. What was the least useful feature o f this online help feature?
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
189
Appendix J
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
190
T ec-E d , A nn A rb o r, MI
L au rie K a n tn e r has been a u sability sp ecia list, inform ation a rc h ite c t, an d p ro jec t m a n a g e r
for o v e r 20 y ears, w ith experience based on w ell o v e r 200 projects. L au rie sp ec ialize s in h e lp in g
c lie n ts d e fin e a n d c a rry out usability stu d ies and docu m en tatio n pro jects to a n sw e r c ritical
d esig n for d o c u m e n ta tio n at the U niversity o f C alifo rn ia, S anta C ru z E x te n sio n . L aurie is o n th e
U PA 2000 C o n fe re n c e .
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
191
V ITA
Jean A. Pratt
Department o f Business Information Systems jpratt@ b202.usu.edu
and Education http://www.bus.usu.edu/bise/faculty/jp/
3515 Old Main Hill Work: 435-797-0331
Utah State University Home: 435-752-4595
Louan. UT 84322-3515 Fax: 435-797-2351
EDUCATION
Ph.D. Instructional Technology, em phasis in online help, Utah State University, December 2000
M.S. Instructional Technology, emphasis in Instructional Design and Development, Utah State
University, January 1995
B.S. Organizational Communication, Summa Cum Laude, University o f Idaho. May 1991
EX PERIEN CE
Utah State University: Assistant Professor (2000-current)
Utah State University: Lecturer (1997-2000)
■ INST 581(0): Instructional Design
• BISE 5450/6450: Advanced Web Design
■ BISE 5450/6450: Computerized Business Presentations
■ ENG 5410: Interactive Media: The Development o f Online Help Systems
» BISE 2550: Business Communications
■ BISE 2400: Desktop and W eb Design
■ BISE 1400: Microcomputer Applications in Business
W ashington State University (1991-1992)
• Public Speaking
■ Small Group Communication
■ Team leader for a university-funded educational hypermedia project
ID2 Research Group, Utah State University: Project M anager, Technical Com municator
(1992-1997)
■ Managed the corporate and educational public education projects (over $500,000 in
funding)
■ Wrote a $ 133,888 grant which was awarded to develop an interactive CD-ROM for higher
education
■ Supervised and contributed to the design and development o f instructional products
■ Created context-sensitive help systems for different versions o f five instructional
development software applications
■ Created hard-copy training and reference documentation for three instructional development
software applications
Moore Business Communication Services: Training Developer (Summers of 1992 and 1993)
■ Developed and upgraded the 1099 training program (both the instructor and participant
materials)
■ Developed the Standard Operating Procedures for all facets o f processing work orders
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
192
PROFESSIONAL PUBLICATIONS
Pratt. J. A. (1998). Where is the Instruction in Online Help Systems? Technical Communication.
45(1), 33-37.
Merrill, M.D., Drake, L., Lacy, M., & Pratt, J.A. (1996). Reclaiming Instructional Design.
Educational Technology. 36(5), 5-7.
Pratt, J. A. (1994). Instructional Design Theory [Review o f Instructional Design T heory].
ETR&D. 42(41. 101-104.
Pratt, J. A. (1994). AECT chapters: Filling a niche. Tech Trends. 39(1). 61.
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
193
PROFESSIONAL MEMBERSHIPS
Association for Educational Communications and Technology
Delta Pi Epsilon
National Business Education Association
Society for Technical Communication
Utah Business and Computer Education Association
R e p ro d u c e d with perm ission of the copyright owner. Fu rther reproduction prohibited without permission.