Counselling and Psychotherapy Research, 2013

Vol. 13, No. 3, 210
219, http://dx.doi.org/10.1080/14733145.2012.739633

RESEARCH ARTICLE

Repurposing process measures to train psychotherapists:

Training outcomes using a new approach

ANTONIO PASCUAL-LEONE* & CRISTINA ANDREESCU

Department of Psychology, University of Windsor, Windsor, Ontario, Canada

Abstract
Aims: First, this paper presents the rationale for a novel approach to training counsellors in which measures for
psychotherapy process research are taught to students before moving on to teaching basic empathic reflections and
interventions. The rationale for this is that client process measures can be re-purposed to help orient and sensitise trainees
to key in-session moments. Second, we present a training outcome study that assesses the effectiveness of this approach.
Method: Using an experiential-integrative therapy approach, a 13-week training program was used to teach psychotherapy
skills and process research measures to22 clinical graduate students taken from two cohorts. As part of the course, trainees
conducted several single sessions with volunteer clients on four separate occasions. Training outcomes were measured using
both trainee and client reports. Results: Compared to baseline, therapists reported significant and steady gains (all ps B.05)
in session management, reducing their anxious self-awareness, and in improved sense of self-efficacy, with the latter having
the largest effect (partial Eta Sq. .381). Discussion: While the findings provide some support for a new training strategy, a
dismantling design is needed next to more closely examine the process-measure approach to training.

Keywords: graduate student; measures; outcome; process; psychotherapy integration; training

Introduction This paper offers two main contributions. First,

Psychotherapy research has largely focused on
developing and testing treatment programmes,
while surprisingly little attention has been dedi-
cated to teaching psychotherapy and examining
how to best train professionals. Even so, an
increasing demand for psychotherapy points to
the need for more therapists who can provide
effective services for a range of difficulties (van
Deurzen-Smith, 1996). Furthermore, in a broader
context, the true effectiveness of psychological
treatments is always constrained in some way
by our ability to successfully train clinicians. It
follows that there is also an important need to
develop and test psychotherapy training pro-
grammes and, indeed, this has started to become
an area of increased interest (Hilsenroth, Defife,
Blagys, & Ackerman, 2006).
the introduction of this paper describes new ideas in
a research-informed practice for training; one which
draws on what we now know about psychotherapy
process to inform the strategy we use in training
novice therapists. This notion is one that has recently
informed our teaching of basic counselling skills, and
is also the approach we examine in this paper.
Second, we report on a pilot study examining the
impact of our introductory course in psychotherapy
skills training, by measuring training effects over
time.
Findings on psychotherapy training
In a recent meta-analysis, Hill and Lent (2006)
examined 14 studies on specific methods used in
counselling and psychotherapy skillstraining, which
were completed from 1967 to 2006 (surprisingly

*Corresponding author. Email: apl@uwindsor.ca

# 2013 British Association for Counselling and Psychotherapy


few). The authors found that the aggregated effect
size of training as compared to a no-training control
was quite large (i.e. d .89), although there was
substantial variability among studies. They also
concluded that modeling is more effective than
instruction and feedback, and multi-method ap-
proaches to training are more impactful than single
methods. Since the Hill and Lent (2006) review,
only a handful of new studies have been conducted
on training. There have been inconsistent findings
about whether training in specific approaches is
actually related to specific intervention use (Boswell,
Castonguay, & Wasserman, 2010; Hilsenroth et al.,
2006). Yet, trainees do report reliable increases in
their sense of self-efficacy (Hill et al., 2008).
Despite the generally promising findings, Hill and
Lent (2006) argue that much of the existing research
would benefit from more rigorous design and out-
come measurement. Boswell et al. (2010) have also
made the case that more proximal changes, such as
session outcomes, might be more suitable when
examining the impact of training. Finally, it seems
that psychotherapy training would also benefit from
developing newer formulations and more practical
strategies to training.
Training psychotherapists with client process measures:
A new approach
Although there have been important advances in the
research on training, pedagogy of psychotherapy has
not yet fully exploited the dramatic advances that
have also been made in process research (see
Pachankis & Goldfried, 2007). For example, there
are a number of well known process measures
(detailed below) that could be used concretely as
teaching tools. While this sometimes happens in-
formally, particularly when research and clinical
training are conducted under the same roof, it is
uncommon that researchers suggest re-purposing
process measures for psychotherapy training.
Hilsenroth (2007) is one exception, having proposed
that the Comparative Psychotherapy Process Scale
(CPPS) be used as a training tool to familiarise
trainees with discriminating features of cognitive-
behavioral vs. interpersonal-psychodynamic inter-
ventions. But this idea can be further developed:
While measures of therapist process or intervention
(i.e. the CPPS) could help trainees to identify and
grasp, what to do as a therapist,
client process
measures, in contrast, would help trainees to,
perceive what is happening in-session with the
Training psychotherapists 211
client.Current approaches to psychotherapy training
emphasise the former (i.e. intervention skills), while
the question of developing a trainees ability to
recognise and orient to key client processes (what
we will call perceptual acuity) receives much less
attention (see Hill & Lent, 2006; Ronnestad &
Ladany, 2006). We believe this imbalance may be
due to a scarcity of formal teaching strategies, which
might be used to heighten the perceptiveness and
sensitivity of trainees, helping them to recognise and
interpret phenomena that are emerging in-session.
Client process measures, however, may turn out to be
particularly useful in this regard.
For example, novice therapists often do not
immediately notice when clients deflect away from
evocative (hot) content; but narrative process
measures (i.e. Angus, Levitt, & Hartke, 1999) could
be used to help sensitise trainees in attending to their
clients spontaneous shifts in narrative style. Thus, a
research measure such as this can help trainees to
firstly: simply perceive, recognise, and track a clients
ephemeral shifts from external narratives (i.e. plot-
and-characters) to more internal narratives (i.e.
meaning-and-experiences). Secondly, this appercep-
tive framework provides a lens to help orient trainees
as they use interventions to actively facilitate a
clients elaboration of more promising processes
(i.e. internal narratives, hot cognitions).
In a similar example, certain client vocal qualities,
such as a highly external, rapid, lecturing voice vs. a
softer, searching, stop-and-go and internally focused
voice have also been operationalised and then related
to unproductive and productive processes, respec-
tively (Rice & Kerr, 1986). Descriptions of these
vocal qualities can provide very intuitive and easy
cues to trainees about a clients internal process and
his or her need for moment-by-moment interven-
tions. In the same vein, while beginner therapists are
often anxious about silences during therapy sessions,
familiarising them with research on the nature of
productive vs. obstructive client pauses (i.e. Levitt,
2001) also provides trainees with practical and
immediately usable information on what is happen-
ing with their clients and how to respond in-session.
To date, a few client process research instruments
have already been incipiently and informally inte-
grated into methods for teaching psychotherapy. The
client experiencing scale (Klein, Mathieu-Coughlan,
& Kiesler, 1986) has long been considered a gold
standard of therapeutic process because it has
been shown to be predictive of good treatment
outcomes in experiential, psychodynamic, and
212 A. Pascual-Leone & C. Andreescu
cognitive treatment approaches (Greenberg &
Pascual-Leone, 2006). Being competent in the use
of this observer rating scale is known (particularly in
the experiential tradition) to be helpful in orienting
therapists to a clients immediate process and then in
deliberately deepening a clients level of meaning
exploration in-session, whether that be affective or
cognitive in nature. However, it has become com-
mon that trainees studying emotion-focused therapy
are encouraged to practice using the experiencing
scale whenever such research training is available.
Furthermore, the fact that the research scale is
reprinted in a number of treatment manuals (e.g.
Paivio & Pascual-Leone, 2010) is particularly reveal-
ing as to its perceived usefulness in learning the
therapy itself.
This re-integration of a research measure, into the
training for a therapy from which the measure itself
originated, is even more dramatic in the psychody-
namic tradition. The core conflictual relationship
theme (CCRT; Luborsky, Popp, Luborsky, & Mark,
1994) method was first introduced as a new research
tool for testing and verifying Freuds hypothesis of
transference. Since then it has been increasingly used
as a way of palpably introducing students to the
construct of transference and teaching them to make
case formulations. This seamless shift in use of the
CCRT process measure (from research to training
purposes) is not often acknowledged in treatment
manuals or training literature. Nevertheless, becom-
ing proficient in using the CCRT by applying it to
transcripts can, again, sensitise trainees to the
perceptual framework that is important to a psycho-
dynamic therapy. A perceptual framework of this sort
is central to the training strategy we are proposing.
When it comes to observing key changes in a
therapeutic alliance, research criteria used to identify
and study relationship ruptures (Safran, Samstag,
Muran, & Stevens, 2001), have now been largely
assimilated into best practice of psychotherapy and
training. Thus, the original research criteria for
recognising and managing ruptures have influenced
a range of dynamic, experiential, and integrative-
cognitive approaches (e.g. see Castonguay et al.,
2004). Familiarising trainees with the original re-
search criteria that have been used for detecting and
classifying different types of relationship ruptures
between clients and their therapists, as well as the
steps in repairing a therapeutic relationship, offers
them concrete instruction for identifying key rela-
tional features. However, as with all the process tools
we have discussed, rather than simply teaching
students the evidence related to a given phenomen-
on, trainees could be asked to engage in a research-
like exercise of coding or rating phenomenon from
personal narratives or audio/video segments to pro-
vide the kind of perceptual training that is prerequi-
site to the incisive and timely use of interventions.
Starting a programme of research on perceptual acuity
training
We have proposed a new process research approach
to training but empirically exploring this notion
requires at least three steps. (1) One needs to
develop a training programme or course that ex-
plicitly focuses on perceptual acuity; particularly,
given that there is no coherent precedent for this
approach and given that it contrasts with an estab-
lished pedagogy which gives preference to interven-
tion as such. (2) Next, one should conduct a pilot
study,verifying that the new course has initial pro-
mise and can be delivered in a coherent and clinically
relevant fashion.(3) Finally, a fully controlled dis-
mantling study would be required to demonstrate
the benefit of the new training component.Thus, in
this paper, we do not yet hope to directly test the
training strategy against a control but rather aim only
to employ the new approach and then report within-
group training effects.
Our current training programme
In the context of training for professional practice we
have adopted the perceptual-acuity training as a key
strategy for training novice therapists. In what
follows we describe the course to illustrate how
process measures are integrated into the larger
context of skills training.
Course structure. The course is given using an
integrative framework with a primary emphasis on
experiential psychotherapy. The class follows a
13-week curriculum and meets for one three-hour
class per week. Students are first given weekly
lectures (60 minutes per class), and participate in a
discussion of readings for the week, followed by
video therapy demonstrations with commentary by
the course director (40 minutes). Finally, they
participate, on a weekly basis, in supervised informal
practice sessions with their peers (45 minutes).
Supervised practice with peers. During the weekly
practice sessions, trainees work in groups of three:

one offering his or her own personal material as client
(i.e. real and unscripted), another serving in the role
of counsellor, and a third acting as observer or on-call
counsellor, who makes process observations and
provides support when requested by the counsellor.
While these informal practice sessions are conducted
in semi-private settings, the course supervisor peri-
odically drops in and discretely attends to the
moment-by-moment process, offering live supervi-
sion and modeling as needed.
Assignments. There are eight assignments distrib-
uted throughout the semester. The purpose of these
assignments is to increase students awareness of
productive therapeutic processing. To that end, in
addition to learning intervention techniques, stu-
dents receive instruction and rate therapy-like ma-
terial using a number of established psychotherapy
process measures in the hopes of sensitising them to
relevant client markers of therapeutic process. As-
signments are as follows:
(1) Keep an emotion diary and use the levels of
emotional awareness scale (Lane, 1991).
(2) Write a personal narrative about a traumatic
experience (c.f. Pennebaker, 1997).
(3) With the help of a peer, create an audio-taped
session describing a personal relationship epi-
sode, and then transcribe it.
(4) Apply the narrative process coding system (An-
gus et al., 1999) to both the personal narrative
and to the transcribed relationship account
(i.e. assignments 2 and 3).
(5) Apply the vocal quality scale (Rice & Kerr,
1986) to the audio file one helped ones peer
make for assignment 3.
(6) Apply the client experiencing scale (Klein et al.,
1986) to both narrative and relationship
accounts (i.e. assignments 2 and 3).
(7) Describe a past relationship rupture from ones
personal life and retrospectively identify the
markers of that rupture. Then invent a ficti-
tious new ending to the story, one which
deliberately follows the steps in Safran et al.s
(2001) model for rupture repairs.
(8) Apply the core conflictual relationship themes
method (Luborsky et al., 1994) to narrative
and relationship accounts (i.e. assignments 2,
3, and 7).
Formal practice with volunteer clients. Finally, as an
extension of the practice with peers, trainees formally
Training psychotherapists 213
practice skills by providing eight single-sessions each
with a different volunteer client distributed across
several occasions during the course. Unlike the
supervised informal practice done with peers, volun-
teer clients are unknown to the trainee and they
present with real and unscripted mild-to-moderate
personal difficulties. These sessions provide trainees
with the uninterrupted opportunity to independently
practice their skills in a realistic context.
Client-level feedback to trainees. Following the
45-minute sessions, clients and trainees each com-
plete a set of session outcome questionnaires (see
measure in Method section, below). Similar to the
work of Lambert et al. (2005), this feedback is later
summarised and made available to students as part
of their training (these measures were also the source
of research data that is presented later in this paper).
Current study and hypotheses
Following the articulation of a new strategy for skills
training, the current research aim was to pilot the
training approach and provide initial support. It was
hypothesised that (1) client ratings of the sessions
they participated in, as well as (2) trainee/therapist
rating of their own sense of competence, would each
reflect the improvement of trainees skills over time.
Methods
Participants
Graduate trainees. Graduate counsellors in training
were 22 first year Masters-level students in a clinical
psychology programme and enrolled in an introduc-
tory psychotherapy skills course. The course was run
for two consecutive years so that participants volun-
teering for this study came from one of two cohorts
(each with 11 students). Thus, 22 graduate trainees
(16 female, four male) participated in the study, with
ages ranging from 22
48, (M 25.4).Two trainees
were dropped from the study on account of their
each having both missed classes and extreme out-
lying data, making the final sample N 18.
Volunteer clients. The clients that volunteered to
participate in single sessions were undergraduates
with personal problems and were solicited through a
research participant pool. These volunteer clients
responded to an internal online advertisement seeking
individuals to receive single session interventions
conducted by counsellors-in-training as part of a
214 A. Pascual-Leone & C. Andreescu
training evaluation. In particular, the notice called
for volunteer clients to, discuss personal and real
difficulties that are currently troublesome and of
mild to moderate concern. Volunteer clients re-
ceived a small course credit through the participant
pool for their one-time involvement in this study. A
total of 71 participants were recruited (61 females
and 10 males) with a mean age of 21.8 (ranging
from18
46).
Measures
At the end of each counselling session, trainees and
clients each completed a separate set of self-reports,
multiple measures were used to capture both trainee
and client evaluation.
Client self-report measures
Working Alliance Inventory-Short Revised Version
(WAI-SR). WAI-SR (Hatcher & Gillaspy, 2006)
consists of 12 Likert-scale items, ranging from 1
(strongly disagree) to 5 (strongly agree), designed
to gauge the depth of the therapeutic relationship.
The original Working Alliance Inventory (WAI) was
designed by Horvath and Greenberg (1989) to assess
the dyads ability to build a bond, and to collabora-
tively set up goals, and tasks for the session.
Correlations between the WAI-SR and the original
WAI suggest the new version is comparable (Hatcher
& Gillaspy, 2006). Reliability coefficients reached
satisfactory levels (between .88 and .92). The WAI-
SR also correlates strongly with the Confident
Collaboration Scale (Hatcher & Barends, 1996)
lending further validity to the scale.
Revised Session Reaction Scale (RSRS). The
RSRS (Elliott, 1993), a revision of the Session
Impact Scale (Elliott &Wexler, 1994), is a 24-item
questionnaire assessing a clients session experience.
Items are on a Likert scale ranging from 1 (not at all)
to 5 (very much) and comprise four sub-scales: the
Helpful Reactions (e.g. progress towards knowing
what to do about problems), Task Reactions (work
toward certain goals, e.g. gaining insight into others
behaviour), Relationship Reactions (i.e. feeling un-
derstood by the counsellor), and finally, Hindering
Reactions, which assesses negative feelings about the
therapy and/or therapist (e.g. feeling stuck or a lack
of progress.).The scale has demonstrated excellent
reliability for all scales (.83 to .92; Reeker, Elliott, &
Ensing, 1996), while convergent validity was also
satisfactory, when compared to the Session Evalua-
tion Questionnaire (Stiles, 1980) and other indices
of client reaction (Reeker et al., 1996). In short, the
RSRS is a measure for clients perception of the
usefulness of the therapy.
Trainee self-report measures
Counsellor Activity Self-Efficacy Scales (CASES).
The CASES (Lent, Hill, & Hoffman, 2003) is a
44-item self-report measure aimed at assessing skills
in overall sense of confidence on helping and
responding, session management, and counselling
challenges efficacy. At the end of individual counsel-
ling sessions items are rated by counsellors on a
Likert scale from 0 (no confidence) to 9 (complete
confidence).The measure has been shown to have
good convergent and discriminant validity when
compared with other counsellor self-reports such as
Counsellor Self-Estimate Inventory and Social De-
sirability scales. Test-retest reliability after two weeks
was also adequate, ranging from .42 to .91 (Lent,
Hill, & Hoffman, 2003).
Self-Awareness and Management Strategies Scales
for Therapists (SAMS). The SAMS (Williams,
Hurley, OBrien, & De Gregorio, 2003) is a self-
report measure designed to assess the prepon-
derance of disruptive internal experiences or obs-
tacles that intrude while conducting a therapy
session. The measure is composed of 33 Likert-scale
items ranging from 1 (never) to 5 (always), divided
into two subscales: hindering self-awareness (i.e.
anxious awareness, distracting self-awareness) and
management of these interferences (i.e. self-care,
relaxation, re-focusing on the client, suppression
of intrusive thoughts). The self-awareness and
management subscales are internally consistent
(.76 and .73, respectively). The measure has good
internal consistency (.73 to .76) and demonstrated
good convergent/discriminant validity when com-
pared with the Self-Monitoring Scale (Snyder,
1974) as well as a satisfactory reliability coefficient
(Williams et al., 2003). In short, SAMS was
designed to assess the degree to which counsellors
in training feel compromised by the distracting self-
awareness of their own thoughts, posture, anxiety
and any other elements that might interfere with
conducting a session.

Procedures
The psychotherapy skills training course was run as
described. Formal sessions, when clients were treated
(as contrasted with the weekly practice sessions con-
ducted among peers), were distributed over the
course to allow the observation of change over
time. Thus, sessions conducted with clients simul-
taneously served as practice and testing times in
this study. During the progression of the 13-week
course, trainees met with clients at four points in the
course: week 2 (time1), week 5 (time2), week 9
(time3), and week 12 (time4).
Sessions at each of these time points were used to
address, as thoroughly as possible, the personal
difficulties presented by clients, while working within
the limits of a single session. Sessions were 45 minutes
long and after each testing time/session both clients
and trainees each completed a set of measures. To
protect against a social desirability effect (on the part
of clients) or self-serving basis (on the part of
trainees), completed measures were placed in sealed
envelopes and both trainees and clients were in-
formed that ratings would remain anonymous.
Measurement at the four points in time provided
change indices of trainees skills acquisition in the
form of a learning curve. However, as explained in
Hill and Lents (2006) recommendations for the
evaluation of training, when single clients are used
one risks that the same trainee might receive, for
example, a very psychologically-minded client at the
beginning of the training course and a very reluctant
client at the end of the course. Such discrepancies in
client-based evaluations would not accurately reflect
the effects of training. Thus, in the aim of giving
further stability to the single session measurements
and to protect against any atypical/extreme ratings,
counsellors-in-training conducted two (separate)
single session interventions at each testing time.
Thus, at each of the four points in time, trainees
conducted two 45 minutes sessions back-to-back,
each with a different client, for a total of eight single
session cases. (Trainees never worked with the same
client twice). Finally, scores of every measure were
then averaged across the two clients treated by a
given trainee at each time point; this was done for
both client and trainee measures.
Ethical considerations
The design was approved by a university research
ethics board, and was evaluated as having taken
Training psychotherapists 215
suitable precautions in care, respect, and informed
consent for both trainees and clients. The trainer/
researcher was kept blind to the identity of trainees
who voluntarily participated and this was done with
the help of a teaching assistant who acted as the
keeper of data. Volunteer clients, presenting with
personal difficulties, attended an orientation just
prior to study participation where they were re-
minded of the aim and context of their counselling
sessions with trainees. Moreover, a licensed clinical
psychologist was on-site for all sessions, and a
resource sheet was given to clients following their
participation.
Results
Preliminary analyses
Missing therapist data was rare and did not occur
systematically. At the first time point, however, client
data was only available for one of the two cohorts.
All repeated analyses were conducted on both the
complete data set (N11 clients over all four time
points) and the larger, truncated data set (N 18
clients over three times points; times 2, 3, 4) and
both methods produced similar and consistently
non-significant findings. Thus, for simplicity, ana-
lyses of client data are presented for the complete
data set; while Figure 1 is based on all available data.
Additional analyses confirmed that there was no
indication of any known differences between the two
cohorts of graduate students.
Main analyses
Testing hypothesis 1: Do clients reports demonstrate
change? Clients did not rate the counselling sessions
as increasing in quality, which did not support the
first hypothesis. To that end, repeated-measures
ANOVAs revealed the following levels of non-sig-
nificance: WAI total (F[3, 30] .205, p .892) and
on RSRS subscales (all Fs[3, 30]B.167, ps .835).
Testing hypothesis 2: Do trainees self-reports de-
monstrate change? Trainee self-reports indicated some
skills improvement on each of the two measures,
supporting the second hypothesis (see Figure 1;
Black markers denote trainee self-report measures).
Counsellor Activity Self-Efficacy Scales. The total
CASES score indicated a significant improvement
of trainees over time: F(3, 51) 10.470, pB.001,
216 A. Pascual-Leone & C. Andreescu
Note: Graduate (N20) training effects: A learning curve over
13 weeks.
Figure 1. WAI, RSRS, SAMS measurements range from 0 to 5;
CASES ranges from 0 to 9; White markers client report
measures; Black markers trainee self-report measures; * p B
0.05, and ** p B0.005 significant difference from baseline
(time 1); There were no significant pairwise comparisons between
contiguous testing time. All therapist data (CASES, SAMS) is
based on N 18; Client data (WAI, RSRS) at time 1 is based on
N 11, and at all other times (2, 3, 4) based on N 18.
with an effect size (Partial Eta Sq. .381). The
improvement appeared to be linear, and post-hoc
analyses indicated the only significant difference of
improvement was between baseline (time 1) and
final session (time 4; p B.005). Thus, although the
pairwise changes between contiguous testing times
were not statistically significant, this increase ap-
peared to be consistent, suggesting that over the
13-week course trainees gradually felt more confi-
dent in their skills.
Self-awareness and Management Strategies Scales
for Therapists. Trainees reports indicated a signifi-
cant improvement (i.e. a decrease) in the hindering
self-awareness subscale of the SAMS, F(3, 51) alliance as slightly over 4 on a 5-point scale, leaving
6.464, p .001, indicating that they changed in their
ability to manage in-session anxiety and intrusive
self-focused thoughts. Pairwise comparisons indicate
a significant improvement from baseline to the
subsequent testing point (p B.046) suggesting im-
provement in this skill.
In addition, trainees showed a change on the
session management subscale, which refers to the
range of tools a therapist uses in order to minimise
self-awareness, anxiety and any other personal fac-
tors that might interfere with the flow of therapy. In
particular, results from this scale indicated a sig-
nificant improvement during the course: F(3, 51)
5.675, p B.002. The pattern of improvement ap-
pears to be linear, and analyses indicated a trend of
improvement between baseline (time 1) and final
session (time 4; p .065). In other words, trainees
reportedly use more self-care/management over the
course of the training programme.
Discussion
In the introduction of this paper, we have conveyed
the rationale for exploring a new strategy in psy-
chotherapy skills training and illustrated how this
might be feasibly integrated into a training pro-
gramme. Another aim of this paper has been to pilot
this new approach and demonstrate initial support
for a course that re-purposes process measures as
psychotherapy training tools for perceptual-acuity.
Thus, the study examines the feasibility of this
approach; it is neither a process-outcome study nor
a dismantling study and, as such, our finding cannot
support claims regarding separate components of the
programme. Nevertheless, trainees indicated a stea-
dy improvement in their confidence and sense of
self-efficacy, which was consistent with previous
research (Hill et al., 2008). They also reported
consistent gains in their ability to manage sessions
and in reducing anxiousness self-awareness (e.g. by
way of relaxation, re-focusing on the client, and
suppression of intrusive thoughts). Furthermore, the
learning curve of graduate training showed a steady
pattern ofchange.
Despite the positive changes in graduate trainees
self-perception, post-session client reports did not
echo these effects; a non-effect that has also been
reported in other research on graduate therapists
(Boswell et al., 2010). Even so, the lack of recorded
change in this study may be due (at least in part) to a
ceiling effect. Indeed, clients rated the therapeutic
little room for measurable improvement. This inter-
pretation is supported by Goodyear and Guzzardo
(2000), who describe a ceiling effect as possibly
masking graduate training improvements as, say,
compared to those of undergraduate trainees. Hill
and Lent (2006) also support this position, observing

that students who are accepted into graduate training
programmes tend to already have good communica-
tion skills.
Alternatively, from a developmental view, the
inconsistency between measured perspectives (trai-
nees reported self-perceptions vs. their clients
reported benefits), might actually reflect different
stages of the training process. In other words,
perhaps when novice therapists feel better oriented,
less anxious, and have a clearer grasp of the
therapeutic process it may first lead to feeling more
confident, but not until later does it translate into
observable intervention changes; and not during a
single session in an introductory course.
Strengths, limitations, and future directions
Given the quasi-experimental design used in this
paper we were unable to directly test the use of
process measures as a strategy for sensitising trainees
to key in-session phenomena. However, the study
does offer some preliminary findings regarding the
delivery of such training in a coherent and clinically
relevant fashion. It is also interesting to note that,
despite a quasi-experimental design, many of these
findings on professional development have received
some corroboration through other research methods
using the same training sample. In a qualitative study
(Pascual-Leone, Wolfe, & OConnor, 2012; Pascual-
Leone, Rubio-Rodriguez, & Metler, under review),
participants were asked to write open-ended narra-
tives about what they perceived as the personal
impact of this course. Results of that inquiry were
strikingly consistent with our current findings and
trainees attributed their changes to the training
course (rather than historical or maturational fac-
tors). Nevertheless, the current study does not offer
a controlled comparison group.
While it should be encouraging that our training
and study was conducted in the framework of a
classroom setting, suggesting that it is easily trans-
portable, the primary purpose of the setting was
educational rather than research. This is the
principal reason why it was not possible to have
the same sorts of controls one might expect from a
laboratory setting. The reasons for not having a
randomised no-training control group are similarly
documented by Hill et al. (2008) in their own
research. Namely, it was not possible to find
comparable classes in which students would agree
to complete measures and participate in formal
counselling sessions; not to mention the ethical
Training psychotherapists 217
concern of randomly assigning volunteer clients to
working with a nave, not-in-training group.
Furthermore, it was logistically impossible at our
university to find a course that could serve as a
control or suitable comparison group, given that
these specialised courses are only available to
students at determined times in their degrees.
Even so, we are encouraged by the triangulation
of our findings through other means (i.e. Pascual-
Leone et al.s qualitative study). Finally, the impact
of therapist training on client session reports may
have been better appraised following multi-session
interventions. Yet, while this could be seen as a
limitation of the current study, it is interesting to
note that Boswell et al. (2010) also found no effects
in client session reports, even when evaluations
were taken around session 10 of an ongoing
treatment.
As a quasi-experimental study of training effects,
our design also had a number of strengths. First, this
study set up several formal sessions with volunteer
clients seeking help for real life difficulties of a mild-
to-moderate nature. This provided a realistic clinical
experience for trainees and allowed for a more
ecologically valid test of training effects as compared
to the many studies that use classmates-as-clients in
mock counselling sessions or teach directly to the
test (for a review of these problems see Hill & Lent,
2006). Second, data was collected from multiple
clients (two per time point), from a multiple
perspective (i.e. both counsellors and clients eva-
luations), and at multiple (four) time points. This
multiplicity is relatively unique among studies on
training (Hill et al., 2008, is a notable exception).
Plotting a learning curve using multiple perspectives
is also important because, as we have seen, trainees
confidence and self-efficacy do not always reflect
better skills performance from the clients perspec-
tive. Moving forward, future research will require:
first a design that links process (of perceptual acuity)
to outcome (of training), and second a dismantling
study, to evaluate the benefit of adding this as a
training component.
Implications for practice
The most central implication of this work is that it
introduces a new and feasible strategy for training
psychotherapists, one which capitalises on existing
research tools by repurposing them for a broader
aim. Until recently, being well versed in psychother-
apy process measures has been somewhat of a
218 A. Pascual-Leone & C. Andreescu
narrow specialisation. However, applying process
measures to transcript, audio, or video records of
therapy is coming to be increasingly viewed as a
potentially fruitful strategy for training psychothera-
pists (i.e. Hilsenroth, 2007). While most of the
measures we have recommended were designed
with the original intention of exploring specific
research questions, they also represent a relatively
jargon-free operationalisation of key constructs,
often offering intuitive descriptions that are highly
accessible to novice therapists. Even more impor-
tantly, this is an efficient way of overcoming a
trainees limitations in perceptual acuity with respect
to productive therapeutic process, which is a largely
unaddressed obstacle for fledgling therapists learn-
ing to use interventions. Thus, rather than relying
on the gradual accumulation of clinical exposure
over many years, trainees can directly harvest from
the clinical clarity developed by psychotherapy
researchers through process research tools that are
readily available. Finally, as we have demonstrated,
this approach could be easily integrated into
common training courses of various therapeutic
approaches.
References
Angus, L., Levitt, H., & Hartke, K. (1999). The narrative
processes coding system: Research applications and implica-
tions for psychotherapy practice. Journal of Clinical Psychology,
55 (10), 1255
1270.
Boswell, J.F., Castonguay, G.L., & Wasserman, R.H. (2010).
Effects of psychotherapy training and intervention use on
session outcome. Journal of Consulting and Clinical Psychology,
78 (5), 717
723.
Castonguay, G.L., Schut, J.A., Aikins, E.D., Constantino, J.M.,
Laurenceau, P.J., Bologh, L., & Burns, D.D. (2004). Integra-
tive cognitive therapy for depression: A preliminary investiga-
tion. Journal of Psychotherapy Integration, 14 (1), 4
20.
Elliott, R. (1993). Revised Session Reaction Scale. Retrieved
February 5, 2010, from http://www.experiential-researchers.
org/instruments/elliott/rsrs.html
Elliott, R., & Wexler, M.M. (1994). Measuring the impact of
sessions in process-experiential therapy of depression: The
session impacts scale. Journal of Counseling Psychology, 41 (2),
166
174.
Goodyear, R.K., & Guzzardo, C.R. (2000). Psychotherapy super-
vision and training. In S. Brown & R.W. Lent (Eds.), Handbook
of counseling psychology (3rd ed., pp. 83
108). New York: Wiley.
Greenberg, L.S., & Pascual-Leone, A. (2006). Emotion in
psychotherapy: A practice-friendly research review. Journal of
Clinical Psychology: In Session, 62, 611
630.
Hatcher, R.L., & Barends, A.W. (1996). Patients view of the
alliance in psychotherapy: Exploratory factor analysis of three
alliance measures. Journal of Consulting and Clinical Psychology,
64, 1326
1336.
Hatcher, R.L., & Gillaspy, J.A. (2006). Development and valida-
tion of a revised short version of the Working Alliance
Inventory. Psychotherapy Research, 16 (1), 12
25.
Hill, C.E., Roffman, M., Stahl, J., Friedman, S., Hummel, A., &
Wallace, C. (2008). Helping skills training for undergraduates:
Outcomes and prediction of outcomes. Journal of Counseling
Psychology, 55 (3), 359
370.
Hill, C.E., & Lent, R.W. (2006). A narrative and meta-analytic
review of helping skills training: Time to revive a dormant area
of inquiry. Psychotherapy: Theory, Research, Practice, Training,
43 (2), 154
172.
Hilsenroth, M.J., Defife, J.A., Blagys, M.D., & Ackerman, S.J.
(2006). Effects of training in short-term psychodynamic
psychotherapy: Changes in graduate clinical technique.
Psychotherapy Research, 16 (3), 293
305.
Hilsenroth, M.J. (2007). A programmatic study of short-term
psychodynamic psychotherapy: Assessment, process, outcome,
and training. Psychotherapy Research, 17 (1), 31
45.
Horvath, A.O., & Greenberg, L.S. (1989). Development and
validation of the Working Alliance Inventory. Journal of
Counseling Psychology, 36, 223
233.
Klein, M.H., Mathieu-Coughlan, P., & Kiesler, D.J. (1986). The
experiencing scales. In L.S. Greenberg & W.M. Pinsof (Eds.),
The psychotherapeutic process: A research handbook (pp. 21
71).
New York: Guilford Press.
Lambert, M.J., Harmon, C., Slade, K., Whipple, J.L., &
Hawkins, E.J. (2005). Providing feedback to psychotherapists
on their patients progress: Clinical results and practice
suggestions. Journal of Clinical Psychology, 61 (2), 165
174.
Lane, R.D. (1991). LEAS scoring manual & glossary. Tucson, AZ:
University of Arizona Health Sciences Center.
Lent, R.W., Hill, C.E., & Hoffman, M.A. (2003). Development
and validation of the counselor activity self-efficacy scales.
Journal of Counseling Psychology, 50 (1), 97
108.
Levitt, H.M. (2001). Sounds of silence in psychotherapy: The
categorization of clients pauses. Psychotherapy Research, 11,
295
309.
Luborsky, L., Popp, C., Luborsky, E., & Mark, D. (1994). The
core conflictual relationship theme. Psychotherapy Research, 4
(3&4), 172
183.
Paivio, S.C., & Pascual-Leone, A. (2010). Emotion focused therapy
for complex trauma: An integrative approach. Washington, DC:
American Psychological Association.
Pachankis, J.E., & Goldfried, M.R. (2007). On the next genera-
tion of process research. Clinical Psychology Review, 27, 760

768.
Pascual-Leone, A., Wolfe, B., & OConnor, D. (2012). The
reported impact of psychotherapy training: Undergraduate
disclosures after a course in experiential psychotherapy.
Journal of Person Centered and Experiential Psychotherapy, 11,
152
168.
Pascual-Leone, A., Rubio-Rodriguez, B., & Metler, S. (under
review). Graduate students experiences in psychotherapy training.
Unpublished manuscript. University of Windsor.
Pennebaker, J.W. (1997). Writing about emotional experiences as
a therapeutic process. Psychological Science, 8, 162
166.
Reeker, J., Elliott, R., & Ensing, D. (1996). Measuring session
effects in process-experiential and cognitive-behavioral therapies of
PTSD: The Revised Session Reaction Scale. Paper presented at
the Society for Psychotherapy Research, Amelia Island, FL,
USA.

Rice, L.N., & Kerr, G.P. (1986). Measures of client and therapist
vocal quality. In L.S. Greenberg & W. M. Pinsoff (Eds.), The
psychotherapeutic process: A research handbook (pp. 73
105).
New York: Guilford Press.
Ronnestad, M.H., & Ladany, N. (2006). The impact of psy-
chotherapy training: Introduction to the special section.
Psychotherapy Research, 16 (3), 261
267.
Safran, D.J., Samstag, W.L., Muran, C.J., & Stevens, C. (2001).
Repairing alliance ruptures. Psychotherapy Research, 28 (4),
406
412.
Snyder, M. (1974). Self-monitoring of expressive behavior.
Journal of Personality and Social Psychology, 30, 526
537.
Stiles, W.B. (1980). Measurement of the impact of psychotherapy
sessions. Journal of Consulting and Clinical Psychology, 48, 176
185.
van Deurzen-Smith, E. (1996). The future of psychotherapy in
Europe. International Journal of Psychotherapy, 1 (1), 15
21.
Williams, E.N., Hurley, K., OBrien, K., & De Gregorio, A.
(2003). Development and validation of the self-awareness and
management strategies (SAMS) scales for therapists.
Training psychotherapists 219
Psychotherapy: Theory, Research, Practice, Training, 40 (4),
278
288.
Biographies
Antonio Pascual-Leone, PhD, is a psychologist
and professor. He has published on psychotherapy
process and outcomes, with a special focus on
emotional processing. He recently co-authored a
book on Emotion Focused Therapy for Complex Trauma
(APA). Dr Pascual-Leone runs a private practice in
Windsor seeing individuals and couples.
Cristina Andreescu, MA, is a doctoral candidate
in clinical psychology. Her areas of specialisation
include clinical skills training and the treatment of
eating disorders. She has also done work in occupa-
tional rehabilitation.
Copyright of Counselling & Psychotherapy Research is the property of Routledge and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.