Documente Academic
Documente Profesional
Documente Cultură
To cite this article: Thilo Deckersbach , Cary R. Savage , Aude Henin , David Mataix-Cols , Michael W. Otto , Sabine
Wilhelm , Scott L. Rauch , Lee Baer & Michael A. Jenike (2000) Reliability and Validity of a Scoring System for Measuring
Organizational Approach in the Complex Figure Test, Journal of Clinical and Experimental Neuropsychology, 22:5, 640-648
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of
the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied
upon and should be independently verified with primary sources of information. Taylor and Francis shall
not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other
liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
Journal of Clinical and Experimental Neuropsychology 1380-3395/00/2205-640$15.00
2000, Vol. 22, No. 5, pp. 640-648 © Swets & Zeitlinger
ABSTRACT
The Rey-Osterrieth Complex Figure Test (RCFT) is a widely-used measure of visuospatial construction
and nonverbal memory. One of the critical aspects of this test is that organizing the figure into meaningful
perceptual units during copy enhances its subsequent free recall from memory. This study examined the
psychometric properties of a new system for quantifying the organizational approach to the RCFT figure
and compared it to another compatible scoring system. We investigated interrater reliability of both sys-
tems and explored the influences of copy organization and copy accuracy on immediate recall. Seventy-one
participants meeting DSM-IV criteria for obsessive-compulsive disorder and 55 healthy control partici-
pants completed the copy and immediate free recall condition of the RCFT. Interrater reliability was evalu-
ated by Kappa coefficients and Pearson correlations. The effects of copy organization and copy accuracy
on immediate recall were evaluated using multiple regression analyses. Results indicated that the organiza-
tional approach could be assessed with high reliability using both scoring systems. Organization during
copy was a strong predictor for subsequent free recall from memory using both approaches. Multiple
regression analysis indicated that all organizational elements were not equally predictive of memory per-
formance. This new system represents a very simple and reliable approach to scoring organization on the
RCFT, since it requires the identification of only 5 figure components. These characteristics should con-
tribute to its clinical utility.
The Rey-Osterrieth Complex Figure Test (Os- example, verbal free recall benefits from group-
terrieth, 1944) is a popular measure of visuo- ing items into semantic categories, or imposing
spatial construction and nonverbal memory. De- a subjective structure on a set of words (Tul-
veloped by Rey (1941), it consists of a complex ving, 1962). Likewise, organizing the RCFT
geometric figure that is copied and then redrawn figure into meaningful perceptual units during
from memory. As a measure of memory, the encoding has been shown to enhance subsequent
RCFT taps the domain of episodic memory, free recall from memory (Savage et al., 1999,
which refers to the explicit recollection of infor- 2000; Shorr, Delis, & Massman, 1992; Waber &
mation encountered in a specific study episode Holmes, 1986).
(Graf & Schacter, 1985; Schacter, 1987). Orga- The prefrontal cortex has been implicated in
nization of information during encoding is the ability to utilize organizational strategies to
known to enhance recollection accuracy. For enhance encoding and retrieval of episodic
*
This research was supported by the David Judah Research Fund, by a Deutscher Akademischer Austauschdienst
(DAAD) Graduate Research Fellowship to Thilo Deckersbach and by the Spanish Ministerio de Education y
Cultura (David Mataix-Cols).
Address correspondence to: Cary Savage, Cognitive Neuroscience Group, Department of Psychiatry, 149-9102,
Massachusetts General Hospital, Bldg. 149, 13th St., Charlestown, MA 02129. Tel.: ++1 617 726-4076. Fax: ++1
617 726-4078. Email: savage@psych.mgh.harvard.edu
Accepted For Publication: March 28, 2000.
ORGANIZATION SCORING FOR COMPLEX FIGURE TEST 641
memories. Individuals with lesions of the frontal highly predictive of subsequent free recall from
lobes or with disorders affecting frontal-striatal memory in the neurologic patients.
system function (e.g., Parkinson’s disease, Hun- Recently, Savage and colleagues (1999) de-
tington’s disease) have difficulties with organi- vised a simple system for quantifying the orga-
zational processes crucial for efficient encoding nizational approach to the RCFT figure. Based
and retrieval of information (e.g., Bondi et al., on the work of Binder (1982), it consists of five
1993; Buytenhuijs et al., 1994; Gershberg & organizational units (the units of Binder plus the
Shimamura, 1995; Grossman et al., 1993; Pillon large base rectangle), each of which must be
et al., 1993; Savage, 1997; Shimamura et al., drawn as an unfragmented piece to receive
1991). This pattern of memory impairment can credit for organization. Thus, unlike other avail-
be differentiated from problems associated with able scoring systems (e.g., Shorr et al., 1992) it
Downloaded by [The University of Manchester Library] at 10:43 10 October 2014
medial temporal system dysfunction, in which scores organizational elements in a simple all-or
individuals have difficulty storing and consoli- none-fashion rather than considering multiple
dating new memories (Squire, 1992). Thus, it is junctures for each element. In a series of three
clinically important to differentiate strategic studies, our group (Deckersbach, Otto, Savage,
memory impairment from other types of mem- Baer, & Jenike, 2000; Savage et al., 1999, 2000)
ory dysfunction. The RCFT is ideally suited to evaluated the organizational approach in indi-
address this distinction because of the opportu- viduals with obsessive-compulsive disorder
nity to observe planning and organizational (OCD), a disorder for which neuroimaging re-
strategies used by patients as they copy the fig- search implicates fronto-striatal dysfunction
ure. (e.g., Baxter et al., 1987, 1988; Nordahl et al.,
Several systems have been designed to char- 1989, Rauch et al., 1994; Swedo et al., 1989).
acterize the organizational approach to the Despite drawing accurate copies of the geomet-
RCFT figure. These systems quantify the extent ric figure, individuals with OCD failed to con-
to which the figure is grouped into meaningful struct the geometric figure in an organized fash-
perceptual units as it is copied. For example, ion. Subsequently, patients with OCD showed
Binder (1982) identified four elementary units in impairments in immediate free recall. Multiple
the complex design (the two diagonals, the hori- regression modeling indicated that impairments
zontal midline, the vertical midline, and the ver- in immediate free recall were statistically ex-
tex of the triangle) which tended to be drawn as plained by deficits in organization during copy.
single unfragmented units by healthy controls. This approach has also been used to characterize
Waber and Holmes (1985, 1986) defined five memory functioning in body dysmorphic disor-
different levels of organization in the geometric der (Deckersbach, Savage, et al., 2000).
design and found that higher organization was The present study was designed to investigate
associated with better recall from memory in a the psychometric properties of the Savage et al.
large sample of children. Similarly, Akshoomoff (1999) organizational scoring system and com-
and Stiles (1995), using the Boston Qualitative pare it to a previously described scoring system
Scoring System (Stern et al., 1994), found that for organization (Shorr et al., 1992). The Shorr
organization in children became more integrated et al. (1992) approach was chosen because it
with age. As shown by Shorr et al. (1992), the adopts a compatible approach for quantifying
relationship between organization and memory organization. It has been shown to predict im-
is also found in adults. They identified critical mediate recall performance, and both systems
junctures in the RCFT figure where breaks in overlap substantially in their organizational ele-
continuous drawing may occur, and constructed ments. The Shorr et al. system characterizes ad-
a scale that assessed how well individuals orga- ditional features of the figure; thus, one goal of
nized the RCFT figure. Using this approach in a this study was to evaluate whether these features
mixed sample of neurologic and psychiatric pa- added to the predictive power of the organiza-
tients, Shorr et al. (1992) documented that orga- tion score.
nizational scores obtained during copy were
642 T. DECKERSBACH ET AL.
the Massachusetts General Hospital Outpatient nificantly from each other (t(124) = .69, p = .49).
Psychiatry Service. Control participants were re- All participants completed the Rey-Osterrieth
cruited through bulletin board notices at the Mas- Complex Figure Test (RCFT; Osterrieth, 1944;
sachusetts General Hospital. Diagnoses of partici- Rey, 1941).
pants with OCD and the status of healthy controls
were determined by a Structured Clinical Interview Rey-Osterrieth Complex Figure Test (RCFT)
(SCID; First, Spitzer, Gibbon, & Williams, 1995). The RCFT figure was presented on a sheet of pa-
Normal controls had no history of any Axis I psy- per in landscape (horizontal) orientation. Partici-
chiatric disorder. Participants with OCD had no pants copied the figure (see Fig. 1), and immedi-
history of psychosis, bipolar disorder, or substance ately afterwards, they were instructed to redraw
dependence and did not meet criteria for current the design from memory. For copy and recall, par-
major depression. All participants had no history ticipants were provided colored pencils that were
of significant head injury, seizure, neurologic con- changed approximately every 15 seconds in order
dition or current medical condition. All partici- to allow subsequent scoring of organization based
pants were Caucasian. Members of the sample pro- on the order in which the component parts were
vided written consent prior to enrolling and were constructed. If participants were completing a ma-
paid for their participation. jor feature at the end of 15 seconds, they were al-
Participants with OCD and healthy controls had lowed to complete that feature before the pencil
mean ages of 36.1 (SD = 10.8, range = 19-65) and was switched.
35.13 (SD = 12.6, range = 19-64; t(124) = .47, p =
.64), respectively. Mean years of education were Copy accuracy
15.6 (SD = 2.3, range = 11-20) for the group with Copy and recall drawings were scored for the ac-
OCD and 16.7 for the control group (SD = 2.3, curacy of construction (correctly copied or recalled
range = 12-20; t(124) = 2.58, p = .01). Scores on elements) using the system described by Meyers
the Yale-Brown Obsessive-Compulsive Scale and Meyers (1995). In this system, the figure is
(YBOCS; Goodman et al., 1989) indicated moder- subdivided into 18 components. Each piece is eval-
ate to severe OCD symptoms (M = 21.2, SD = 5.5, uated with respect to its drawing accuracy (1 point)
range = 10-31). The degree of depression as as- and its correct placement (1 point) resulting in a
sessed by the Beck Depression Inventory (BDI; range of scores per element from 0 to 2 and a total
Beck & Steer, 1987) was subclinical in the group range of scores from 0 to 36. As demonstrated by
with OCD (M = 14.1, SD = 8.4, range = 0-41), and Meyers and Meyers (1995), interrater reliability is
well-within the nondepressed range for the healthy high (r > .90). To ensure sufficient reliability of
control sample (M = 2.3, SD = 3.2, range = 0-15; copy and recall accuracy scores, a subsample of 30
t(121) = 9.97, p < .001). Scores on the Edinburgh copy and recall drawings (15 randomly selected
Handedness Inventory (Oldfield, 1971) indicated from each group) were scored by two independent
that 68 of the 71 individuals in the group with raters blind to group membership.
OCD, and all healthy control participants were
right-handed. Verbal intelligence was estimated Organization Scoring
with the Vocabulary subtest of the Wechsler Adult The organizational approach used during the copy
Intelligence Scale – Revised (WAIS-R; Wechsler, condition was assessed using the Shorr et al.
1981) or the Vocabulary subtest of the Shipley In- (1992) and Savage et al. (1999) scoring ap-
stitute of Living Scale (Zachary, 1991), which re- proaches.
quires indviduals to find the synonyms of given A detailed description of the Shorr et al. organi-
words. Twenty control (36%) and 36 (51%) partic- zational scoring system can be found in Shorr et al.
ORGANIZATION SCORING FOR COMPLEX FIGURE TEST 643
(1992). Briefly, in this system, 20 junctures in 8 sion analysis. The predictive value of individual
organizational units are identified (the large rect- organizational units for immediate recall was also
angle, the diagonals, the vertical and horizontal evaluated using simple and multiple regression
midline, the vertex of the triangle, the small rect- analyses.
angle within the large rectangle, the diagonals
within the small rectangle and the small square
beneath the large rectangle). Individuals receive a RESULTS
point for organization if a line on each side of a
juncture is drawn continuously or contiguously.
Scores for organization are summed together re- Interrater Reliability
sulting in a scale with a range of scores from 0 to Cohen’s Kappa coefficients indicated moderate
20. to high interrater agreement for the organiza-
Downloaded by [The University of Manchester Library] at 10:43 10 October 2014
In the Savage et al. (1999) scoring system, the tional elements included in Savage et al.’s
geometric figure is subdivided into five configural (1999) scoring system. The highest agreement
elements: the large rectangle, the diagonal cross,
was achieved for the vertical midline (Cohen’s
the vertical midline, the horizontal midline and the
vertex of the triangle on the right (see Fig. 1). Kappa = .92, p < .001), followed by the large
Participants receive points for constructing each rectangle (Cohen’s Kappa = .90, p < .001), the
element as an unfragmented unit. The large rectan- diagonals (Cohen’s Kappa = .89, p < .001), the
gle is assigned two points to reflect its importance horizontal midline (Cohen’s Kappa = .84, p <
to the fundamental organization of the figure. All .001) and the vertex of the triangle (Cohen’s
other elements are assigned one point, resulting in Kappa = .69, p < .001). The number of disagree-
a range of scores from 0 to 6. The accuracy of
drawing is not considered in this score. An organi-
ments for the elements ranged from as few as 2
zational element is considered to be drawn as an for the vertical midline to as many as 16 for the
unfragmented unit whenever (1) each side com- vertex of the triangle. Interrater agreements for
prising a unit (e.g., each of the four sides of the copy organization (whole scale) in both scoring
rectangle) is drawn as a continuous line without systems were high as indicated by Pearson cor-
interruption; and (2) all sides are drawn one after relations of r = .94 (p < .001) for both ap-
another. The order in which the sides of a unit are proaches. Accuracy scores for copy and recall
drawn is not taken into account as long as they are
drawn one after another. Participants did not re- also showed high interrater agreement, as indi-
ceive credit for organization whenever a part of an cated by Pearson correlations of r = .93 (p <
organizational unit was missing. .001) and r = .97 (p < .001), respectively. Copy
Two raters whom were blind to group member- organization scores for both organization sys-
ship independently evaluated copy organization. tems were highly intercorrelated as indicated by
The raters had no experience with either scoring a Pearson correlation of r = .80 (p < .001).
system prior to training. Final scores for the pur-
pose of multiple regression analyses were deter-
mined by consensus discussion; agreement was Group Comparisons
reached for all drawings. Table 1 shows means and standard deviations
for copy accuracy scores, organization scale
Statistical Analysis scores (both Savage et al., 1999 and Shorr et al.,
Interrater reliability for the organizational scoring 1992), and free recall scores (all agreement
of each element in the Savage et al. (1999) scoring scores).
system was evaluated by Cohen’s kappa coeffi-
Group comparisons for a subsample of partic-
cient. Interrater reliability for both the Shorr et al.
(1992) and Savage et al. (1999) organization ipants with OCD and controls included in this
scales, as well as interrater reliability of copy ac- study have been previously reported in Savage
curacy, was evaluated by Pearson correlations. et al. (1999, 2000). For the pooled data in this
Group differences (OCD vs. Control) in copy orga- study, independent samples t tests indicated that
nization (both Savage et al., 1999 and Shorr et al., participants with OCD copied the figure less
1992), and copy and recall accuracy, were evalu- accurately than control participants (t(124) =
ated with independent samples t-tests. The impact
of copy organization and copy accuracy on imme- 2.02, p = .05), and organized the RCFT figure to
diate free recall was evaluated by multiple regres- a lower extent than control participants as as-
644 T. DECKERSBACH ET AL.
Downloaded by [The University of Manchester Library] at 10:43 10 October 2014
Savage et al.,
Organizational 1999
Elements
Points
(0-2)
(0-1)
(0-1)
(0-1)
(0-1)
Fig. 1. The original Savage et al. (1999) system for scoring the organizational approach on the Rey-Osterrieth
Complex Figure. Five core configurational elements are identified: The large rectangle (2 points), the
two diagonals (1 point), the vertical midline (1 point), the horizontal midline (1 point), and the vertex
of the triangle to the right (1 point). The scores for each element are summed, resulting in a scale with
a range of scores from 0 to 6.
ORGANIZATION SCORING FOR COMPLEX FIGURE TEST 645
Table 1. Scores for the Two Organizational Scoring Approaches, Copy Accuracy and Immediate Free Recall.
sessed by both the Savage et al. (1999) scoring groups in order to increase statistical power.
system (t(124) = 4.06, p < .001) and the Shorr et Table 2 shows the correlations between copy
al. (1992) scoring system (t(124) = 3.99, p < organization, copy accuracy, and recall accuracy
.001). Participants with OCD also recalled fewer for the whole sample.
elements of the RCFT figure than control partic- As shown in Table 2, both copy organization
ipants (t(124) = 4.44, p < .001). and copy accuracy scores were significantly cor-
related with each other and correlated with im-
Prediction of Free Recall Scores mediate free recall (p < .01).
Both accuracy and organization of the copy con- Copy organization and copy accuracy scores
dition might affect the quality of encoding in the were entered stepwise in a multiple regression
RCFT. The impact of copy organization and equation as predictors for immediate free recall.
copy accuracy on immediate free recall was, Copy organization as assessed by Savage et al.
therefore, evaluated separately for both scoring (1999) explained 36% of the free recall variance
systems. The correlations between copy organi- (R2 = .36, F(1,124) = 70.65, p < .001). Inclusion
zation (agreement scores) as assessed by both of copy accuracy did not significantly increase
Savage et al. (1999) and Shorr et al. (1992), and explanatory power (R2 = .42, F(2,123) = 45.26,
copy accuracy (agreement scores) and immedi- )R2 = .06, p > .05). Likewise, copy organization
ate free recall (agreement scores) were almost as assessed by Shorr et al. (1992) explained 28%
identical in the OCD and control groups. A se- of the variance (R2 = .28, F(1,124) = 48.65, p <
ries of independent t tests confirmed that these .001). Inclusion of copy accuracy did not in-
correlations did not differ significantly from crease the amount of free recall variance ex-
each other across the two groups (all p > .16). plained (R2 = .35, F(2,123) = 33.66, )R2 = .07,
Therefore, the data were collapsed over the two p > .05). The simple correlation of organization
Note. Copy accuracy and immediate recall were scored based on Meyers & Meyers (1995).
* p < .05; ** p < .01.
646 T. DECKERSBACH ET AL.
scale scores with immediate free recall scores .001). All elements but one predicted significant
did not differ significantly across the two sys- portions of the free recall variance, including the
tems (p > .05). large rectangle ($ = .22, p = .006), the diagonals
We also investigated whether adding the three ($ = .31, p < .001), the horizontal line ($ = .21,
additional elements included in the Shorr et al. p = .009), and the triangle ($ = .16, p = .04). The
(1992) organization scoring system (the small vertical line within the large rectangle did not
rectangle in the left half of the large rectangle, significantly predict free recall from memory ($
the small diagonals within the small rectangle, = .04, p = .58) in the multiple regression equa-
and the small square below the large rectangle) tion.
to the Savage et al. (1999) scale increased ex-
planatory power. As with the other elements,
Downloaded by [The University of Manchester Library] at 10:43 10 October 2014
Goodman, W.K., Price, L.H., Rasmussen, S.A., Savage, C.R., Deckersbach, T., Wilhelm, S., Rauch,
Mazure, C., Delgado, P., Heninger, G.R., & S.L., Baer, L., Reid, T., & Jenike, M.A. (2000).
Charney, D.S. (1989): The Yale-Brown Obsessive- Strategic processing and episodic memory impair-
Compulsive Scale (YBOCS). Part II: Validity. Ar- ment in obsessive-compulsive disorder. Neuropsy-
chives of General Psychiatry, 46, 1012-1018. chology, 14, 141-151.
Graf, P. & Schacter, D.L. (1985). Implicit and explicit Schacter, D.L. (1987). Implicit memory: History and
memory for new associations in normal subjects current status. Journal of Experimental Psychol-
and amnestic patients. Journal of Experimental ogy: Learning, Memory and Cognition, 13, 501-
Psychology: Learning, Memory and Cognition, 11, 518.
501-518. Shimamura, A.P., Janowsky, J.S., & Squire, L.R.
Grossman, M., Carvell, S., Peltzer, L., Stern, M.B., (1991). What is the role of frontal lobe damage in
Gollomp, S., & Hurtig, H.I. (1993). Visual con- memory disorders? In H.S. Levin, H.M. Eisenberg,
struction impairment in Parkinson’s disease. Neu- & A.L. Benton (Eds.). Frontal lobe function and
Downloaded by [The University of Manchester Library] at 10:43 10 October 2014