Journal of Intellectual & Developmental Disability, September 2009; 34(3): 266–274

LITERATURE REVIEW

The Kirkpatrick model: A useful tool for evaluating training outcomes*

ANDY SMIDT1, SUSAN BALANDIN2, JEFF SIGAFOOS3 & VICKI A. REED4

1
University of Sydney, Australia, 2Molde University College, Norway, 3Victoria University of Wellington, New Zealand, and
4
James Madison University, USA

Abstract
Background Services employing staff to support people with disability usually provide training in a range of areas including
communication and managing challenging behaviour. Given that such training can be costly and time-consuming, it is
important to evaluate the evidence presented in support of such programs. Efficacy in clinical practice is measured using
evidence-based practice. However, there is currently no model that is widely used to compare and evaluate training programs
despite the large number of training programs reported each year.
Method Six studies published in the last decade that reported the outcomes of communication-based training and six that
reported on the outcomes of challenging behaviour training were evaluated using the 4-level Kirkpatrick model.
Results Comparison of the levels of evidence is made for these 12 studies.
Conclusion The Kirkpatrick model provides one technique for appraisal of the evidence for any reported training program
and could be used to evaluate whether a training program is likely to meet the needs and requirements of both the
organisation implementing the training and the staff who will participate.

Keywords: staff training, evaluation, communication, challenging behaviour

Introduction appraised using an evaluation approach developed by

Staff who support people with intellectual disability
are an important asset to the service they work in and
usually account for the greatest expenditure within
any service (Felce, 1994; Jenkins, Rose, & Lovell,
1997). Given that staffing is typically the greatest cost
to the service, many researchers have reflected on
both the dollar value and the role of paid staff within
services. Such scrutiny logically extends to staff
training; however, there is scant information on the
efficacy of training programs. Reilly (2004) pre-
sented an overview of evidence-based practice speech
pathology clinicians can use to critically appraise the
effectiveness of intervention studies. The need for
clinician appraisal is equally relevant when consider-
ing staff training programs. Critical appraisal of a
program includes consideration of validity, results,
and relevance (Grey & Hastings, 2005; Reilly, 2004).
In this paper, 12 reports on training programs are
Donald Kirkpatrick (Kirkpatrick, 1996).
The role of care staff
Researchers have considered the role and contribu-
tion made by paid staff to the quality of lives of
individuals in state or voluntary run residential
facilities (Doerner, Miltenberger, & Bakken, 1989;
Mansell, 1995; Parsons & Reid, 1993; Vollmer, Iwata,
Zarcone, Smith, & Mazaleski, 1993). A particular
focus of this research has been the interactions that
take place between staff and people with intellectual
disability. ‘‘Positive social interaction between staff
persons and residents is among the most basic and
important components of staff activity in residential
settings’’ (Baker & Feil, 2000, p. 489) and impacts on
the lives of both residents and staff alike.
The quality of life people with intellectual dis-
ability in residential care receive is partly determined

*This manuscript was accepted under the Guest Editorship of Deb Keen.
Correspondence: Dr Andy Smidt, Discipline of Speech Pathology, University of Sydney, Cumberland Campus, PO Box 170, 75 East Street, Lidcombe, NSW
1825, Australia. E-mail: andy@eye.com.au
ISSN 1366-8250 print/ISSN 1469-9532 online ª 2009 Australasian Society for the Study of Intellectual Disability Inc.
DOI: 10.1080/13668250903093125

by the support they receive from staff (Mansell,
1995). In recent years, researchers have considered
the role of paid care staff in the management of
challenging behaviour (see Ager & O’May, 2001, for
a review) and have attempted to determine the most
effective method of training and supporting carers
(both paid and unpaid) (e.g., Chatterton, 1998;
Dobson, Upadhyaya, & Stanley, 2002; Gentry,
Iceton, & Milne, 2001; Golden & Reese, 1996;
Purcell, McConkey, & Morris, 2000).
Staff training programs
Given the research findings detailing the limited
number of interactions that typically occur between
staff and residents in residential services (Felce,
Lowe, & Blackman, 1995; Felce & Perry, 1995;
Hundert, Walton-Allen, Vasdev, Cope, & Summers,
2003; Markova, Jahoda, Cattermole, & Woodward,
1992), it is not surprising to find a body of recent
research that is dedicated to changing and improving
interactions between residents and the staff who
support them. These studies include staff training
programs (Chatterton, 1999; Dobson et al., 2002;
McLeod, Houston, & Seyfort, 1996; Money, 1997),
activity scheduling, direct skill teaching to residents
(Sigafoos et al., 2004), and staff management pro-
cedures (Seys & Duker, 1993; Suda & Miltenberger,
1993).
In recent years, research and intervention with
people with intellectual disability has shifted towards
focusing not only on the person with the disability
but also on the skills of communicative partners and
modification of their interactions with those indivi-
duals (Butterfield & Arthur, 1995; Light & Binger,
1998; McConkey, Morris, & Purcell, 1999;
McNaughton & Light, 1989; Purcell et al., 2000;
Purcell, Morris, & McConkey, 1999; Seys, Kersten,
& Duker, 1990). The most common approach to
changing the interaction skills of the communication
partner is via a training program. There have been a
range of types of training programs offered (Jahr,
1998), including instructional procedures (e.g.,
lectures, discussions, reading material) and one or
more of the following:
. Role-playing (i.e., where participants act out
the part of the staff member and the client)
(Gentry et al., 2001; Schepis, Reid, Ownbey, &
Parsons, 2001).
. Direct feedback (e.g., on tasks learned during
training) (Reid & Parsons, 1996).
. Self-management (such as self-recording, self-
evaluation, and self-administered consequen-
ces) (Embregts, 2000; Morris & Ellis, 1997;
Evaluation of training 267
Richman, Riordan, Reiss, Pyles, & Bailey,
1988).
. A combination of the above (Reid et al., 2003).
Despite this focus on training, the efficacy of such
training is rarely examined. Even though staff may
report that training is useful, there is no guarantee
that the training will impact on their knowledge
levels or behaviour (Ziarnik & Bernstein, 1984).
Evaluating training programs
When considering the impact of any particular
study or training program it is necessary to evaluate
the evidence presented in support of the program.
One approach to the measurement of the impact of
training is that developed by Kirkpatrick (1996),
who presented a 4-level model to evaluate training
comprising reaction (1), learning (2), behaviour (3),
and results (4). The first level of evaluation,
reaction, typically involves trainees completing a
post-course evaluation of their impressions of the
program. Such evaluation does not measure what
participants have learned, but gauges the interest,
motivation, and attention levels of participants. The
second level, learning, involves measuring what
participants have learned in terms of both knowl-
edge and/or skills. Learning evaluation can include
trainees participating in written assessments or
role-plays to demonstrate their skills. This level of
evaluation allows participants to demonstrate their
understanding of specific skills and/or knowledge
within the learning program. The third level is
behaviour or performance. This involves assessment
of the trainees’ ability to use their newly learned
knowledge or skills in the workplace. This level of
evaluation attempts to determine whether partici-
pants (who may already have demonstrated acquisi-
tion of specific skills and/or knowledge) use their
new skills when they return to the work environ-
ment. The fourth level, described as results, is a
measure of the impact that the training has had
overall, including financial or morale impacts. This
might include improvement in, for example, staff–
resident interaction, decreased incidents of challen-
ging behaviour, and staff turnover.
In this paper we will consider the effectiveness of a
range of staff training programs for staff working with
adults with intellectual disability as evaluated using
the Kirkpatrick model (Kirkpatrick, 1996). Given
that those needing the highest level of support often
include people with severe communication impair-
ments and those with challenging behaviour, our
focus is on two types of training: (a) training to
improve staff–resident interaction, and (b) training
268 A. Smidt et al.
to improve staffs’ management of challenging beha-
viour. What follows is a critical overview of current
knowledge in staff training in these two areas to
determine which types of training are most effective
at modifying staff behaviour and, in turn, resident
behaviour.
Training to improve staff–resident interaction
Studies reporting communication-based training
were selected from a search of the PsychINFO and
ERIC databases using the terms ‘‘staff training,’’
‘‘disabilities,’’ and ‘‘communication.’’ Six studies
that evaluated the effectiveness of this training were
identified. The methods of evaluating each reported
training course are presented in Table 1 using
Kirkpatrick’s (1996) levels.
Golden and Reese (1996) reported an unusual
approach to training that involved treating direct-
care staff as professionals and teaching them to use
an assessment format. Evaluation consisted of con-
sumer satisfaction measures (Level 1 measurement
in Kirkpatrick’s model), as well as measures of
staff–resident interactions (Level 3 measurements).
The design of this study was a multiple-probe
across-group design, which involved measurements
of the behaviours of staff groups prior to and after
training. The results reported the percentage of
intervals of staff behaviour. No statistical analysis
was performed; however, the authors discussed the
graphs and compared them to a control group who
received no intervention. Results indicated that
staff participants made some increases in positive
verbal behaviours and slight improvements in posi-
tive nonverbal behaviour after training. Concurrent
improvements in resident behaviours were not
observed.
Money (1997) compared the effectiveness of
three different approaches to the delivery of speech
and language therapy; namely (a) direct individual
therapy given to the client and his/her communica-
tion partner; (b) attendance by communication
partner at a training program; and (c) a combination
of approaches where the communication partner
attended training and received individual therapy
input. Money’s training program did not result in
any significant changes in the quantity of interactions;
there was, however, a reported change in staff initia-
tions where staff used more open questions and
fewer statements. Statistically significant changes
only occurred in the combination approach where
both direct therapy and staff training were given.
Chatterton (1998) investigated the impact of a
2-day communication workshop, which the author
states was designed to increase staff participants’
‘‘knowledge about communication and its break-
down,’’ and to apply this to a resident with whom
they were familiar. This study involved evaluations
at a range of levels including use of a questionnaire
(Level 1 and 2), interview (Level 1), analysis of
interactions (coded as social or functional), and
duration of interactions (Level 3) before and after
training (a total of 11 weeks). The training resulted
in no significant difference in the duration of staff-
initiated interactions. However, there was a sig-
nificant increase in the immediacy of staff members’
responses to clients and a significant increase in
staff members’ use of nonverbal communication to
support interactions. The analysis of the question-
naire pre- and post-training indicated a significant
increase in participants’ knowledge.
Purcell et al. (2000) reported on a training
program that was evaluated by analysing a 15-minute
videotaped segment of staff interacting with a client
(Level 3 measurement in Kirkpatrick’s model). The
analysis was conducted before and after the training
(at least 3 months apart). Results indicated that the
training had no significant effect on the communica-
tion of the client and no significant effect on the
communication of the staff. The training included
specific recommendations on some changes staff
might make in their interactions; namely, less use of
closed questions, more use of open questions, more
responses, more nonverbal acts, more use of com-
ments, and more use of facial expressions. Fre-
quency counts of these behaviours indicated that all
staff made at least one positive change and staff
reported that they felt that they had benefited from
the training. Analysis of the data, however, revealed
no significant change.
Dobson et al. (2002) described a multidisciplin-
ary focus approach used to train staff to alter their
communication with adults with profound intellec-
tual disability. The study examined a training pro-
gram for nine care staff in a day centre in the UK.
The training centred around video recordings of
staff interacting with one service-user for each staff
member trained. The researchers analysed videos
for the amount of language used and the type of
communication acts involved (Level 3). Data were
collected before and immediately after training and
then again 6 months after training. Results in-
dicated a significant increase in the number of
utterances used by staff, especially those utterances
that were praise, acknowledgements, and requests
for information. These changes were not sustained
when measured 6 months after training. There was
little difference in the language level or structure
used. The authors commented that the lack of
change observed was due to the staff being an
Table 1. Evaluation of communication-based training
Authors Aim
Golden & Reese (1996) To train 12 staff to rate the
interaction between themselves and
12 residents.
Money (1997) To train 20 staff members in one of 3
models of speech therapy service
delivery:
(1) direct therapy alone
(2) staff training
(3) combined direct therapy and
staff training.
Chatterton (1998) To train 12 staff in a one-day
workshop to improve their
communication with 7 residents.
Purcell, McConkey, & To train 24 self-selected staff during a
Morris (2000) 2-day training session to improve
their communication with one
selected resident for each staff
member.
Dobson, Upadhyaya, & To train 9 staff in a multidisciplinary
Stanley (2002) approach to improve
communication between staff and 9
selected clients with profound
intellectual disability.
Smidt, Balandin, Reed, & To train 18 staff members in a 4-week
Sigafoos (2007) program to identify
communication goals for
themselves with one resident.
Results
Slight increases in staff positive
interaction behaviours. Little or
inconsistent change in resident
behaviours.
All approaches resulted in limited
changes in staff initiations, resident
responses, and modality of
communication. Significant changes
were found for staff in the
combination approach.
Staff increased their use of nonverbal
communication, responded more
quickly to resident communication,
and engaged in longer interactions.
No significant changes in staff
behaviour. Some increased
responsiveness from staff. Clients
became more active communicators.
Staff improved their verbal
communication with clients as well as
gaze monitoring and position during
interactions. Changes were
maintained for at least 6 months.
Staff increased their use of AAC and
praise with residents. Staff decreased
their use of inappropriate language.
Some limited decrease in client
challenging behaviour.
Evaluation Kirkpatrick level
Consumer satisfaction questionnaire. Level 1 – reaction
Measurement of change in staff– Level 3 – behaviour
resident interaction levels using
counts of pre-determined
behaviours.
Measurement of pre- and post- Level 3 – behaviour
intervention levels of types of staffs’
initiations of communication,
staffs’ modality of communication,
and residents’ modality of
communication by coding of
videotapes.
A questionnaire. Level 1 – reaction
Level 2 – knowledge
An interview between each staff Level 1 – reaction
member and a researcher about
their perception of the
communication skills of the
individual with whom they worked.
Analysis of functional and social Level 3 – behaviour
interactions between staff and
resident.
Measurement of the duration of Level 3 – behaviour
interactions.
Analysis of videos of staff–resident Level 3 – behaviour
interaction.
Analysis of the amount of language Level 3 – behaviour
used and the type of
communication acts involved using
videos of staff–client interactions.
Staff reaction using questionnaires. Level 1 – reaction
Measured staff–resident interactions. Level 3 – behaviour
Measured resident challenging Level 4 – results
behaviour.
Evaluation of training
269
270 A. Smidt et al.
established team with appropriate language skills
prior to the training. They noted that the purpose of
the training with this staff group was to recognise
their own existing skills and to encourage them to
use these skills more often, rather than to teach
skills that the staff did not have. The researchers did
use a Level 3 analysis of staff behaviour in the
workplace (analysis of videos of staff and resident
communicative acts), but the study involved small
numbers of staff who were all trained
simultaneously.
Smidt, Balandin, Reed, and Sigafoos (2007)
reported on a training program that supported
residential care staff to analyse the communicative
interactions between themselves and clients. The
staff worked collaboratively to formulate consensus
communication guidelines. The researchers used
analyses at Levels 1, 3, and 4 to evaluate not only
changes in staff–resident interaction but also levels of
residents’ challenging behaviour. In a multiple base-
line study, participants demonstrated an increased
use of praise, decreased use of inappropriate
language, and increased use of augmentative and
alternative communication (AAC). There was some
decrease in challenging behaviour, but this was not
sustained in the long-term.
Most of these reported training programs had
limited success at changing staff–resident interac-
tions. In order to gauge the success of any commu-
nication training program it is important to know if
the training resulted in any changes in communica-
tive behaviours of both staff and residents. Only one
of the studies listed in Table 1 (Smidt et al., 2007)
carried out evaluations at Level 4 (results) in the
Kirkpatrick model and therefore measured any
changes in the quality of life of residents. Smidt
et al. (2007) measured levels of resident challenging
behaviour following staff training.
All six of the communication training studies
summarised in Table 1 reported on small numbers of
staff participants (ranging from 9 to 24 staff
participants) completing some form of staff satisfac-
tion measure (Level 1). None of the six studies
reported measurements at Level 2 (knowledge and/
or skills). This may indicate that in a training
program it is difficult to take useful measurements
of staff knowledge and/or skills that are indicative of
the communication skills needed to work with a
person with a disability. All six studies reported
measures at Level 3 (behaviour). This indicates that
the authors were all concerned to determine that the
training resulted in changes in staff behaviour in the
workplace. However, only one of the six studies
attempted to measure changes at Level 4 (results).
This is arguably the most difficult level to measure as
it involves showing that training had an impact on
more general outcomes such as the quality of life of
the client. This is partly due to the difficulty in
demonstrating that any changes in client quality of
life have resulted from the training rather than from
any number of other factors including a change in
staff or physical conditions.
Training to improve staff management of challenging
behaviour
Six studies reporting the effectiveness of challenging
behaviour training were identified from the Psy-
chINFO and ERIC databases using the terms ‘‘staff
training,’’ ‘‘disabilities,’’ and ‘‘behaviour.’’ The
methods used to evaluate training programs that
aimed to teach staff to manage challenging behaviour
are presented in Table 2.
Allen, McDonald, Dunn, and Doyle (1997)
conducted a training program that focused on teach-
ing staff to understand aggressive incidents, primary
prevention, secondary prevention, and reactive stra-
tegies based on Positive Behaviour Support (PBS) in
a residential behavioural treatment unit. Evaluation
included measurements at Kirkpatrick’s Level 4
(behaviour). The study indicated a significant decre-
ase in staff injury and some decreases in the use of
restraint, emergency medication, and client injury.
McDonnell (1997) conducted a 3-day training
course designed to teach care staff to manage
challenging behaviour. Twenty-one care staff parti-
cipated in the training. Evaluation was made at
Level 2 (knowledge and/or skills) of Kirkpatrick’s
model. Results indicated a significant change in self-
confidence of care staff and all staff ‘‘passed’’ a role-
play test.
In a study consisting of three, 3-hour training
sessions teaching functional assessment of problem
behaviours, behaviour intervention, and methods of
instruction, Baker (1998) measured whether staff
completed a functional analysis of behaviour and
wrote a behaviour support plan (Level 3) when they
returned to the workplace. Content of the behaviour
support plans was also reviewed (Level 2). Data were
also collected to measure episodes of challenging
behaviour incidents by auditing incident report
forms (Level 4). Results indicated that effective
functional analyses and behaviour support plans
were produced and a significant decrease in problem
behaviours was noted. This is a comprehensive study
that measured knowledge and/or skills (Level 2),
compliance with departmental policy (Level 3), as
well as levels of resident challenging behaviour
(Level 4). The number of functional analyses and
behaviour support plans increased significantly and
Table 2. Evaluation of challenging behaviour training
Study Aim
Allen et al. (1997) To train staff (number unspecified) to
understand aggressive incidents, how to
prevent challenging behaviour by
modifying the environment, how to
recognise the start of challenging behaviour
and respond safely, and how to react if
behaviours did occur.
McDonnell (1997) To train 21 care staff in a 3-day training
course to understand the causes of
challenging behaviour, to use verbal and
physical skills to diffuse incidents, to use
non-violent physical restraint, and to
increase the confidence of staff in
managing challenging behaviour.
Baker (1998) To train staff (number not specified) to carry
out a functional assessment of problem
behaviours, to write a behaviour
intervention plan, and to teach new skills to
a resident using appropriate teaching
strategies.
Allen & Tynan (2000) To train 109 staff to prevent and manage
challenging behaviour, including
preventative strategies and use of physical
interventions. Participants were then
taught physical intervention strategies in a
practical session.
Gentry et al. (2001) To train 101 staff using a program based on
‘‘Interactive Staff Training.’’ Staff were
taught appropriate management
techniques for challenging behaviour,
including breakaway techniques and
restraint techniques.
McKenzie et al. (2000) To train 59 care staff about the definitions of
disability, challenging behaviour, and staff
responsibilities. Staff were taught to define
and record challenging behaviour, reactive
strategies, and positive programming
approaches.
Results Evaluation Kirkpatrick level
Some decrease in behavioural incidents for Measurements were made of the behaviour Level 4 – results
most residents, decrease in use of major levels of clients following training using the
reactive strategy use, and decrease in level Adaptive Behavior Scale.
of staff and resident injury.
Knowledge scores and confidence scores (1) A multiple choice knowledge test. Level 2 – knowledge
increased. (2) Managing Challenging Behaviour Level 2 – knowledge
Confidence Scale.
(3) A Restraint Role-Play Test. Level 2 – knowledge
Functional assessments and behaviour Measurement of whether staff completed a Level 3 – behaviour
support plans were written for all residents. behaviour support plan.
Frequency of problem behaviour Evaluation of the content of the behaviour Level 2 – knowledge
decreased. plan (i.e., staff knowledge).
An audit of incident reports was made to Level 4 – results
measure levels of challenging behaviour.
Participants were more confident in Confidence in Coping with Patient Level 1 – reaction
managing challenging behaviour and more Aggression Instrument.
confident in applying reactive strategies. Reactive strategies questionnaire. Level 1 – reaction
Staff increased knowledge about challenging Completion of a questionnaire at the start Level 2 – knowledge
behaviour and were able to demonstrate and end of the 3-day course.
use of management techniques in a role- Role-play exercises. Level 2 – knowledge
play situation. Staff expressed improved Attitudes were measured using participant Level 1 – reaction
confidence in managing challenging evaluation forms.
behaviour.
Staff improved their knowledge about all Participants completed a questionnaire Level 2 – knowledge
factors except those important in managing before and after training.
challenging behaviour. Participants described how they would Level 2 – knowledge
respond to two scenarios.
Evaluation of training
271
272 A. Smidt et al.
the number of episodes of challenging behaviour
decreased significantly.
Allen and Tynan (2000) conducted a study to
examine the impact of a training program on staff
members’ confidence to work with clients who were
aggressive. One hundred and nine staff participated
in the study, and were divided into a trained group of
staff and an untrained control group. Measurement
was made using the Confidence in Coping with
Patient Aggression Instrument (Level 1). The
training was successful in increasing participants’
perceived confidence to work with aggressive in-
dividuals. This study measured participants’ percep-
tion of their confidence at Level 1 in Kirkpatrick’s
model. The trained group was significantly more
confident in managing challenging behaviour than
the untrained group. The researchers then trained
the untrained group and found significant improve-
ments in their confidence following training.
Gentry et al. (2001) developed a training program
based on ‘‘Interactive Staff Training’’ (Corrigan &
McCracken, 1997), a staff training package for staff
working in psychiatric rehabilitation settings. Gentry
et al. modified the training package and included
some additional features based on ‘‘best practice’’
principles of training. They conducted the training at
the group home venue, enlisted support from super-
visory staff, and included the staff as ‘‘partners in the
training venture’’ (p. 144). The modified ‘‘inter-
active staff training’’ was provided to a total of 101
staff in a program lasting 3 days. Evaluation was
made at Level 2 of the Kirkpatrick model. Results
indicated a highly significant improvement in parti-
cipants’ knowledge (Level 2) as measured by a
questionnaire administered before and after training.
Measurements were made of participants’ skills and
attitudes during a role-play; however, these results
were not subjected to statistical analysis. Question-
naires at the end of the training indicated that the
participants found the course to be enjoyable and
useful (Level 1).
McKenzie, Paxton, Patrick, Matheson, and
Murray (2000) conducted a one-day training course
with the aim of improving the knowledge of staff
about the characteristics of intellectual disability,
definition and management of challenging beha-
viour, client choice, and duty of care. A total of 59
staff were trained. Evaluation of the training was
made via completion of a questionnaire before and
after training, and by participants describing how
they would respond to two scenarios (Level 2 in
Kirkpatrick’s model). Participants had significantly
better understanding about the definition of intellec-
tual disability and their duty of care; however, they
did not show a significant change in their
understanding of the management of challenging
behaviour, such as the use of reactive strategies.
The six challenging behaviour studies reported
here trained larger groups of staff (range 21–109)
than the communication training studies (range
9–24). Four of the six challenging behaviour studies
measured staff knowledge (Level 2). This is in
contrast to the communication training programs
where none of the included studies measured
changes of staff knowledge and/or skills (Level 2).
This may indicate that it is easier to measure the
skills required in managing behaviour than commu-
nication skills. However, only one behaviour man-
agement study measured staff behaviour in the
workplace (Baker, 1998). This may be partly due to
the larger size of participant groups in the behaviour
management studies and the consequent time and
cost to monitor the behaviour of such a large group
of staff. Although two studies (Allen et al., 1997;
Baker, 1998) attempted to measure change in
resident behaviour following training, the study by
Allen et al. (1997) noted that there were methodo-
logical problems with pre-training data that limited
the usefulness of these measures. It is interesting to
note that only two of the six behaviour management
studies measured staff satisfaction (Level 1), which
may indicate that staff are expected to participate in
challenging behaviour training as a requirement of
their employment rather than communication train-
ing, which may be based on staff interest rather than
staff and resident safety.
In summary, of the six behaviour management
studies that attempted to teach care staff knowledge
and skills for working with individuals with intellec-
tual disability, the majority used no measures of
changes in staff behaviour in the workplace (Level 3).
Some researchers demonstrated a significant change
in knowledge or skills immediately after the training
(Level 2) (e.g., Baker, 1998; Gentry et al., 2001;
McDonnell, 1997; McKenzie et al., 2000), but there
was little evidence that this increased knowledge
impacted on staff behaviour in the workplace or if
there was any change in the levels of residents’
challenging behaviour.
Conclusion
Many staff training programs related to individuals
with disability are conducted each year. These pro-
grams consume organisational time and resources.
Under these circumstances, it is important that
managers selecting a particular training program
are able to determine with some degree of certainty
the likely impact of the training on staff and resident
outcomes. If the efficacy of training in terms of

practical outcomes is not established, staff may be
inappropriately blamed for not ‘‘following through,’’
the behaviour of residents may be considered
intractable, and resources may be wasted.
To date, the evidence in support of the benefits
of disability-related staff training is equivocal.
Few studies have provided strong evidence that
training significantly improves outcomes for people
with disability. Reports of training need to be
appraised according to an appropriate standard. The
Kirkpatrick model provides one technique for apprai-
sal of the evidence for any reported training program.
Moreover, the model can be used to determine
whether a favourable outcome is limited to self-
reported staff attitudes and practices, or whether there
are improvements to relevant knowledge acquisition
and application, and even positive impacts on operat-
ing costs. The ultimate goal is a training program that
will meet the needs of both the staff who will
participate and the clients with disability for whom it
is intended; but also one that is likely to be
implemented by the responsible organisation.
References
Ager, A., & O’May, F. (2001). Issues in the definition and
implementation of ‘‘best practice’’ for staff delivery of
interventions for challenging behaviour. Journal of Intellectual
& Developmental Disability, 26, 243–256.
Allen, D., McDonald, L., Dunn, C., & Doyle, T. (1997).
Changing care staff approaches to the prevention and manage-
ment of aggressive behaviour in a residential treatment unit for
persons with mental retardation and challenging behaviour.
Research in Developmental Disabilities, 18, 101–112.
Allen, D., & Tynan, H. (2000). Responding to aggressive
behavior: Impact of training on staff members’ knowledge
and confidence. Mental Retardation, 38, 97–104.
Baker, D. J. (1998). Outcomes of behavior support training to an
agency providing residential and vocational support to persons
with developmental disabilities. Journal of the Association for
Persons with Severe Handicaps, 23, 144–148.
Baker, D. J., & Feil, E. G. (2000). A self-evaluation by agencies
providing residential support regarding capacity to support
persons with disabilities and challenging behaviours. International
Journal of Disability, Development and Education, 47, 171–181.
Butterfield, N., & Arthur, M. (1995). Shifting the focus: Emerging
priorities in communication programming for students with a
severe intellectual disability. Education & Training in Mental
Retardation, 30, 41–50.
Chatterton, S. (1998). An investigation of speech and language
therapy to improve the communication environment of people
with severe learning disabilities who have communication
difficulties and behaviours that challenge services. Journal of
Learning Disabilities for Nursing, Health and Social Care, 2, 203–
211.
Chatterton, S. (1999). Learning disabilities. Communication skills
workshops in learning disability nursing. British Journal of
Nursing, 8, 94–96.
Corrigan, P. W., & McCracken, S. G. (1997). Interactive staff
training: Rehabilitation teams that work. New York: Plenum.
Evaluation of training 273
Dobson, S., Upadhyaya, S., & Stanley, B. (2002). Using an
interdisciplinary approach to training to develop the quality of
communication with adults with profound learning disabilities
by care staff. International Journal of Language & Communica-
tion Disorders, 37, 41–57.
Doerner, M., Miltenberger, R. G., & Bakken, J. (1989). The
effects of staff self-management on positive social interactions
in a group home setting. Behavioral Residential Treatment, 4,
313–330.
Embregts, P. J. C. M. (2000). Effectiveness of video feedback and
self-management on inappropriate social behavior of youth
with mild mental retardation. Research in Developmental
Disabilities, 21, 409–423.
Felce, D. (1994). Costs, quality and staffing in services for people
with severe learning disabilities. Journal of Mental Health, 3,
495–506.
Felce, D., Lowe, K., & Blackman, D. (1995). Resident behaviour
and staff interaction with people with intellectual disabilities
and seriously challenging behaviour in residential services.
Mental Handicap Research, 8, 272–295.
Felce, D., & Perry, J. (1995). The extent of support for ordinary
living provided in staffed housing: The relationship between
staffing levels, resident characteristics, staff:resident interac-
tions and resident activity patterns. Social Science & Medicine,
40, 799–810.
Gentry, M., Iceton, J., & Milne, D. (2001). Managing challenging
behaviour in the community: Methods and results of interactive
staff training. Health and Social Care in the Community, 9, 143–150.
Golden, J., & Reese, M. (1996). Focus on communication:
Improving interaction between staff and residents who have
severe or profound mental retardation. Research in Develop-
mental Disabilities, 17, 363–382.
Grey, I. M., & Hastings, R. P. (2005). Evidence-based practices in
intellectual disability and behaviour disorders. Current Opinion
in Psychiatry, 18, 469–475.
Hundert, J., Walton-Allen, N., Vasdev, S., Cope, K., & Summers,
J. (2003). A comparison of staff-resident interactions with
adults with developmental disabilities moving from institu-
tional to community living. Journal on Developmental Disabil-
ities, 10, 93–112.
Jahr, E. (1998). Current issues in staff training. Research in
Developmental Disabilities, 19, 73–87.
Jenkins, R., Rose, J., & Lovell, C. (1997). Psychological well-being
of staff working with people who have challenging behaviour.
Journal of Intellectual Disability Research, 41, 502–511.
Kirkpatrick, D. (1996). Great ideas revisited. Techniques for
evaluating training programs. Revisiting Kirkpatrick’s four-
level model. Training and Development, 50, 54–59.
Light, J. C., & Binger, C. (1998). Building communicative
competence with individuals who use augmentative and alternative
communication. Baltimore: Paul H. Brookes.
Mansell, J. (1995). Staffing and staff performance in services for
people with severe or profound learning disability and serious
challenging behaviour. Journal of Intellectual Disability Research,
39, 3–14.
Markova, I., Jahoda, A., Cattermole, M., & Woodward, D.
(1992). Living in hospital and hostel: The pattern of
interactions of people with learning difficulties. Journal of
Intellectual Disability Research, 36, 115–127.
McConkey, R., Morris, I., & Purcell, M. (1999). Communications
between staff and adults with intellectual disabilities in naturally
occurring settings. Journal of Intellectual Disability Research, 43,
194–205.
McDonnell, A. (1997). Training care staff to manage challenging
behaviour: An evaluation of a three day training course. British
Journal of Developmental Disabilities, 43, 156–162.
274 A. Smidt et al.
McKenzie, K., Paxton, D., Patrick, S., Matheson, E., & Murray,
G. C. (2000). An evaluation of the impact of a one-day
challenging behaviour course on the knowledge of health and
social care staff working in learning disability services. Journal
of Intellectual Disabilities, 4, 153–165.
McLeod, H., Houston, M., & Seyfort, B. (1996). Communicative
interactive skills training for caregivers of nonspeaking adults
with severe disabilities. International Journal of Practical
Approaches to Disability, 19, 5–11.
McNaughton, D., & Light, J. (1989). Teaching facilitators to
support the communication skills of an adult with severe
cognitive disabilities: A case study. Augmentative and Alter-
native Communication, 5, 35–41.
Money, D. (1997). A comparison of three approaches to delivering
a speech and language therapy service to people with learning
disabilities. European Journal of Disorders of Communication, 32,
449–466.
Morris, J., & Ellis, J. (1997). The effect of verbal and graphic
feedback on the data-recording behavior of direct care trainers.
Behavioral Interventions, 12, 77–104.
Parsons, M. B., & Reid, D. H. (1993). Evaluating and improving
residential treatment during group leisure situations: A
program replication and refinement. Research in Developmental
Disabilities, 14, 67–85.
Purcell, M., McConkey, R., & Morris, I. (2000). Staff commu-
nication with people with intellectual disabilities: The impact
of a work-based training programme. International Journal of
Language & Communication Disorders, 35, 147–158.
Purcell, M., Morris, I., & McConkey, R. (1999). Staff perceptions
of the communicative competence of adult persons with
intellectual disabilities. British Journal of Developmental Dis-
abilities, 45, 16–25.
Reid, D. H., & Parsons, M. B. (1996). A comparison of staff
acceptability of immediate versus delayed verbal feedback in
staff training. Journal of Organizational Behavior Management,
16, 35–47.
Reid, D. H., Rotholz, D. A., Parsons, M. B., Morris, L., Braswell,
B. A., Green, C. W., et al. (2003). Training human service
supervisors in aspects of PBS: Evaluation of a statewide,
performance-based program. Journal of Positive Behavior
Interventions, 5, 35–46.
Reilly, S. (2004). The challenges in making speech pathology
practice evidence based. Advances in Speech–Language Patho-
logy, 6, 113–124.
Richman, G. S., Riordan, M. R., Reiss, M. L., Pyles, D. A. M.,
& Bailey, J. S. (1988). The effects of self-monitoring
and supervisor feedback on staff performance in a residential
setting. Journal of Applied Behavior Analysis, 21, 401–409.
Schepis, M. M., Reid, D. H., Ownbey, J., & Parsons, M. B.
(2001). Training support staff to embed teaching within
natural routines of young children with disabilities in an
inclusive preschool. Journal of Applied Behavior Analysis, 34,
313–327.
Seys, D. M., & Duker, P. C. (1993). Staff management
procedures and changes in the distribution of nontargeted
activities by residential staff members: A secondary analysis.
Behavioral Residential Treatment, 8, 21–28.
Seys, D., Kersten, H., & Duker, P. (1990). Evaluating a ward staff
program for increasing spontaneous and varied communicative
gesturing with individuals who are mentally retarded. Beha-
vioral Residential Treatment, 5, 247–257.
Sigafoos, J., Drasgow, E., Reichle, J., O’Reilly, M., Green, V. A.,
& Tait, K. (2004). Tutorial: Teaching communicative rejecting
to children with severe disabilities. American Journal of Speech-
Language Pathology, 13, 31.
Smidt, A., Balandin, S., Reed, V., & Sigafoos, J. (2007). A
communication training programme for residential staff work-
ing with adults with challenging behaviour: Pilot data on
intervention effects. Journal of Applied Research in Intellectual
Disabilities, 20, 16–29.
Suda, K. T., & Miltenberger, R. G. (1993). Evaluation of
staff management strategies to increase positive interactions
in a vocational setting. Behavioral Residential Treatment, 8,
69–88.
Vollmer, T. R., Iwata, B. A., Zarcone, J. R., Smith, R. G., &
Mazaleski, J. L. (1993). The role of attention in the
treatment of attention-maintained self-injurious behavior:
Noncontingent reinforcement and differential reinforce-
ment of other behavior. Journal of Applied Behavior Analysis,
26, 9–21.
Ziarnik, J. P., & Bernstein, G. S. (1984). Effecting change in
community-based facilities: Putting staff training in perspec-
tive. Behavior Therapist, 7, 39–41.