This article was downloaded by: [JAMES COOK UNIVERSITY]

On: 17 March 2015, At: 12:02

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Clinical Neuropsychologist

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/ntcn20

The boston qualitative scoring system

for the rey-osterrieth complex figure: A
study of children with attention deficit
hyperactivity disorder
a a b a c d
Deborah A. Cahn , Ann C. Marcotte , Robert A. Sten , James
a d e d
E. Arruda , Natacha A. Akshoomoff & Isabell C. Leshko
a
Department of Psychiatry and Human Behavior , Brown University
School of Medicine , Providence, Rhode Island
b
Department of Pediatrics , Memorial Hospital of Rhode Island ,
Pawtucket, Rhode Island
c
Department of Clinical Neurosciences , Brown University School of
Medicine , Providence, Rhode Island
d
Rhode Island Hospital , Providence, Rhode Island
e
Georgia State University , Atlanta, Georgia
Published online: 08 Nov 2007.

To cite this article: Deborah A. Cahn , Ann C. Marcotte , Robert A. Sten , James E. Arruda , Natacha
A. Akshoomoff & Isabell C. Leshko (1996) The boston qualitative scoring system for the rey-osterrieth
complex figure: A study of children with attention deficit hyperactivity disorder, The Clinical
Neuropsychologist, 10:4, 397-406, DOI: 10.1080/13854049608406700

To link to this article: http://dx.doi.org/10.1080/13854049608406700

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
 Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
 The Clinical Neuropsychologist 1385-4046/96/1004-397$12.00
1996, Vol. 10, NO.4, pp. 397-406 0 Swets & Zeitlinger

The Boston Qualitative Scoring System for the

Rey-Osterrieth Complex Figure: A Study of Children
with Attention Deficit Hyperactivity Disorder*
Deborah A. Cahn', Ann C. Marcottelr2,Robert A. S t e n ~ ' , ~James, ~ , E. A r r ~ d a ' . ~Natacha
, A.
Akshoomoff5, and Isabel1 C. Leshko4
'Department of Psychiatry and Human Behavior, Brown University School of Medicine, Providence, Rhode
Island, *Department of Pediatrics, Memorial Hospital of Rhode Island, Pawtucket, Rhode Island, 3Department
of Clinical Neurosciences, Brown University School of Medicine, Providence, Rhode Island, 4RhodeIsland
Hospital, Providence, Rhode Island, and 'Georgia State University, Atlanta, Georgia
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015

ABSTRACT

The Boston Qualitative Scoring System (BQSS) for the Rey-Osterrieth Complex Figure (ROCF) was
utilized to examine the qualitative features of ROCF performance of children with Attention Deficit Hyper-
activity Disorder (ADHD). Thirty-nine children with ADHD were compared to age-matched controls (n =
39) on their reproduction of the ROCF. ADHD children performed more poorly than did control children
on measures of attention to detail, expansion, accuracy, and neatness. Sensitivity and specificity of individ-
ual BQSS measures for discriminating ADHD from control subjects were determined, and a logistic regres-
sion model was derived, yielding an overall sensitivity of 64% and specificity of 97% for the classification
of ADHD. Eighty-one percent of all children were correctly classified. Cross-validation of this model on
an independent sample of ADHD and control subjects revealed good predictive accuracy. These findings
suggest that the BQSS may be a useful measure in the neuropsychological evaluation of children with
suspected ADHD.

Attention Deficit H y p e r a c t i v i t y D i s o r d e r sin Card Sorting Task (Chelune, Ferguson,

(ADHD) is a constellation of developmentally
inappropriate behaviors, including hyperactivity,
inattention, and impulsivity, that arises in child-
hood and impairs social, academic, and/or occu-
pational functioning (American Psychiatric As-
sociation, I 994). Several neuropsychological
studies of children with ADHD have demon-
strated impairment on some tasks that are be-
lieved to reflect problems with executive func-
tioning, such as integration, planning, organiza-
tion, and inhibition. For example, children with
ADHD have been shown to perform more
poorly than d o matched controls on the Wiscon-
Koon, & Dickey, 1986; Gorenstein, Mammato,
& Sandy, 1989; Reader, Harris, Schuerholz, &
Denckla, 1994), verbal fluency tasks (Felton,
Wood, Brown, Cambell, & Harter, 1987; Grod-
zinsky & Diamond, 1992), and the Porteus
Mazes (Grodzinsky & Diamond, 1992; Homa-
tidis & Konstantareas, 1981; Kuehne, Kehle, &
McMahon, 1987). However, not all studies have
demonstrated executive dysfunction in subjects
with ADHD (Barkley & Grodzinsky, 1994;
Reader et al., 1994). A possible explanation for
these disparate findings is that studies have var-
ied tremendously in subject selection (e.g., oper-

* Dr. Cahn is now at the Department of Psychiatry and Behavioral Sciences, Stanford University School of
Medicine. Portions of this paper were presented at the 15th Annual Conference of the National Academy of
Neuropsychology, San Francisco, CA, November, 1995.
Address correspondence to: Ann C. Marcotte, Ph.D., Department of Pediatrics, Memorial Hospital of Rhode
Island, 111 Brewster St., Pawtucket, RI, 02860, USA.
Accepted for publication: January 26, 1996.
 398 DEBORAH A. CAHN ET AL.
ational definition of ADHD, co-morbid learning
disabilities) and test-related performance vari-
ables (e.g., differences in test versions, adminis-
tration, and scoring).
One test that has been used to assess the exec-
utive aspects of visuomotor integration is the
Rey-Osterrieth Complex Figure (ROCF) (Cor-
win & Blysma, 1993; Lezak, 1995; Osterrieth,
1944; Rey, 1941). This test requires the subject
to copy a complex figure, engaging visuomotor
integration and visuoperception abilities in con-
cert with planning and organization. Grodzinsky
and Diamond (1992) reported that boys with
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
ADHD performed more poorly than did normal
control subjects on their copy of the ROCF. In
contrast, several other studies have reported
negative findings (Barkley & Grodzinsky, 1994;
Frost, Moffitt, & McGee, 1989; Reader et al.,
1994). One explanation for these discrepant
findings may be that traditional quantitative
scoring systems were used and, as suggested by
Reader et al. (1994), these scoring systems may
not be sensitive enough to capture organiza-
tional aspects of the ROCF.
Recently, the Boston Qualitative Scoring Sys-
tem (BQSS) (Stern et al., 1994) was developed
in order to capture specific impairments in the
process by which the ROCF is reproduced. The
system includes 17 dimensions upon which the
drawing is judged, and uses an extensive set of
criteria, templates, and example productions in
deriving each score. The dimensions of the
BQSS were chosen based on clinical observa-
tions of a wide range of patients with focal and
diffuse neurobehavioral disorders, as well as on
the literature pertaining to visuoconstructional
functioning and visuospatial learning and mem-
ory. The BQSS has been shown to have very
good interrater reliability (Stern et al., 1994) and
is particularly useful because it provides a vari-
ety of scores, each assessing specific qualitative
features of a production and the processes em-
ployed. Akshoomoff and Stiles (1995) have re-
cently demonstrated that the BQSS is a useful
tool for assessing visuospatial constructional
and organizational skills in normal children.
Because planning and organizational skills
are purported to be affected in ADHD, and be-
cause the BQSS has been developed specifically
to assess process aspects of ROCF reproduction,
we undertook this exploratory study to examine
the ability of the BQSS to differentiate children
with ADHD from children without ADHD. A
number of hypotheses were generated. These
were based upon the suggested executive dys-
function thought to characterize the cognitive
impairments in ADHD, and upon clinical expe-
rience using the ROCF with the traditional 36-
point scoring system (Lezak, 1995) applied to
children.
We anticipated qualitatively different perfor-
mances on selected BQSS measures in children
with ADHD as compared to normal control chil-
dren despite average constructional and visuo-
spatial functioning in the ADHD children. As-
sumed deficits in inhibitory and attentional pro-
cesses led us to expect poorer scores on the mea-
sures of Accuracy and Neatness in copying the
figure. Difficulty in generating efficient and or-
ganized strategies were expected to result in a
lower score on the Planning score, which
assesses the overall organization of the produc-
tion, including such aspects as the order in
which particular elements are drawn and the
placement on the page of the entire production.
Two measures included in the BQSS, Vertical
and Horizontal Expansion, were hypothesized to
be impaired in the ADHD group, possibly re-
flecting difficulty inhibiting graphomotor output
and self-monitoring.
METHOD
Subjects
ADHD Subjects
Potential subjects were identified retrospectively
from clinic files and were included in the study if
they met DSM-IV (APA, 1994) criteria for ADHD
and the inclusionary criteria described below. All
clinic subjects were seen for evaluation in a Neuro-
developmental Evaluation Clinic at a metropolitan
university-affiliated medical center in Rhode Is-
land. Each child was evaluated individually by a
Child Neuropsychologist (AM) and a pediatrician.
Prior to their evaluation, each subject’s school re-
cords, including all past evaluations, were ob-
tained. Each child received comprehensive neuro-
psychological testing (including measures of ex-
 BOSTON QUALITATIVE SCORING SYSTEM IN ADHD
pressive and receptive language, visual perception,
visual motor integration, memory, executive func-
tioning, and attention), several parent and teacher
behavioral rating forms were completed, and a full
neurological examination was performed on each
child. All subjects had been administered the
ROCF as part of the neuropsychological evalua-
tion.
Selection for study inclusion was made without
knowledge of the subject’s performance on the
ROCF. Furthermore, BQSS scores were not avail-
able or used in the clinical diagnosis of ADHD. All
subjects had obtained a Full Scale IQ score of
greater than 80 (on either the WISC-I11 or Stanford
Binet) within the last 3 years. If no IQ score was
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
available, a child was included only if she or he
had never been referred for special educational
testing, had demonstrated average abilities on stan-
dardized group achievement tests, and had no out-
standing deficits on recent report cards. All chil-
dren had English as their primary language.
Potential subjects were excluded if they were
receiving any special educational services in
school at the time of evaluation, or if they had a
documented history of head injury, seizure disor-
der, neuromuscular disorder, psychiatric problems,
learning disability, motor disorder (e.g., tics, Tou-
rette’s Syndrome), and sensory problems. No sub-
ject was on medication for the treatment of ADHD
at the time of evaluation. Vision and hearing were
screened at the time of evaluation, and no subject
exhibited deficits on either test. Subjects were not
excluded on the basis of gender or handedness.
The resulting ADHD group was comprised of
39 (32 male, 7 female) children between the ages
of 6 and 12 years (mean age = 8.3; SD = 1.9). This
ratio (4: 1) of male to female is consistent with gen-
der ratios in the community reported for this disor-
der (APA, 1994). Furthermore, we did not antici-
pate gender differences on the ROCF in this age
group, based on the findings of Akshoomoff and
Stiles (1995). The ADHD subjects performed
within normal limits on the Developmental Test of
Visual Motor Integration (Beery, 1989) (mean
standard score = 95.7; SD = 9.8) and on the Hooper
Visual Organization Test (Hooper, 1958) (mean z
score = 0.0; SD = .81).
Normal Control ( N C ) Subjects
Control subjects were selected from a larger sam-
ple of children who had received the ROCF as part
of another study investigating the normal develop-
ment of visuospatial abilities in children (Akshoo-
moff & Stiles, 1995). Each control subject was
matched in age to a child in the ADHD group, re-
sulting in 39 control subjects with a mean age of
399
8.5 (SD= 1.7) years, and a gender distribution of
20 boys and 19 girls. The NC children were all in
regular school placements and did not have a his-
tory of major medical illness, psychiatric illness,
developmental disorder, or significant visual or
auditory impairments. All children were tested in
San Diego, California, in either a laboratory, local
community center, or local middle school, and all
came from primarily middle-class families and
were representative of the local community in
terms of race and gender.
Instrumentation
All subjects were individually administered the
ROCF using a standardized procedure. Control
subjects received copy and immediate recall trials
as part of a brief assessment protocol (Akshoomoff
& Stiles, 1995). The ADHD group received copy,
immediate recall, and 30-min delayed recall trials
as part of a larger neuropsychological evaluation.
For the present study, only copy conditions were
included. All subjects were instructed to copy the
ROCF as carefully as possible. Children were told
that at specific intervals they would be given a dif-
ferent colored pen to continue their drawing. Pens
were switched approximately every minute, or
when the subject began to draw a new part of the
figure ( e g , to detect fragmentation of a specific
element of the figure). Children were not allowed
to rotate the model or the blank sheet of paper, and
were told to indicate to the examiner when they
were finished with the copy. All productions were
scored by trained research assistants who were
blind to group membership and to the hypotheses
being explored in the study.
Each production was scored using the BQSS
(Stern et al., 1994). Table 1 outlines and describes
the 17 BQSS dimensions. The 17 scores were de-
rived to assess pertinent qualitative features of
ROCF production, including the following: the
presence, accuracy, and placement of major com-
ponents of the figure (i.e., configural elements,
clusters, details); fragmentation of key elements;
overall planning; reduction and expansion of the
entire figure; rotation; perseveration; confabula-
tion; neatness; and asymmetry. Many of the scores
are rated by comparison to specific criteria (and
example productions) included in the manual.
Some scores (e.g., Horizontal and Vertical Expan-
sion) are rated by use of transparency template
overlays. The version of the scoring system used in
the present study (September 1994 Version) dif-
fered from the version described by Stern et al.
(1994) in that each score could receive a possible
score ranging from 0 to 4, with 0 representing poor
performance and 4 representing good performance;
 400 DEBORAH A. CAHN ET AL.
the earlier version utilized a 1 to 5 scale. AS
described above, the present study utilized selected
BQSS variables based on hypotheses regarding the
impact of ADHD on ROCF performance. The vari-
ables chosen for the current investigation included:
Configural Accuracy, Cluster Accuracy, Detail
Presence, Fragmentation, Planning, Vertical Ex-
pansion, Horizontal Expansion, Perseveration, and
Neatness.
Statistical Analyses
Group comparisons were performed on the BQSS
variables of interest using nonparametric Mann-
Whitney U tests, where children with ADHD were
compared to NC children. Measures of sensitivity
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
and specificity were calculated to determine the
overall accuracy of each BQSS measure in identi-
fying a child with ADHD. For the present analy-
ses, each possible BQSS score for an item (i.e., 0,
1 , 2, 3, 4) was treated as a possible cut-off score
and the corresponding sensitivity (i.e., percentage
of children with ADHD scoring at or below this
score) and specificity (i.e., percentage of NC chil-
dren scoring above this score) for the diagnosis of
ADHD were calculated. A receiver operating char-
acteristic (ROC) curve was then generated for each
of the BQSS variables by plotting sensitivity
against specificity for every possible cut-off score.
Since it was decided a priori that sensitivity and
specificity were to be of equal importance, the op-
timal cut-off score was the point at which the sum
of sensitivity and specificity reached a maximum
value. In the present study, the criterion “gold
standard” for correct classification of ADHD was
the diagnosis made at the time of the comprehen-
sive multidisciplinary clinic evaluation.
In addition to determining the optimal cut-off
score and sensitivity and specificity for each indi-
vidual BQSS score, a stepwise logistic regression
analysis with forward variable selection was per-
formed to determine the combination of BQSS
measures that would provide the most accurate
differentiation between NC and ADHD subjects.
Prior to conducting the stepwise logistic regres-
sion, all BQSS scores were dichotomized accord-
ing to the previously derived cut-off scores. Con-
sequently, subjects scoring at or below the cut-off
were given a value of 0 (i.e., were considered
ADHD), while those subjects scoring above the
cut-off were given a value of 1 (i.e., were consid-
ered normal). All BQSS variables investigated in
the ROC analyses were entered into the logistic
model.
The stepwise approach to variable selection in-
volves selecting variables for inclusion or exclu-
sion in a sequential fashion based solely on statisti-
cal criteria. In forward selection, variables are al-
lowed to enter into the logistic regression model if
they contribute significantly to the differentiation
of normal and clinical subjects. In the present
study, a single BQSS variable was allowed to enter
the equation at each step if the probability associ-
ated with that variable’s “F to enter” was less
than .05. Once the logistic regression equation was
derived, an “estimated probability” of having
ADHD was subsequently derived for each subject.
These probabilities range from 0.0 to 1.0, with a
value of 0.0 representing “perfect classification”
of a NC subject, and 1.0 representing “perfect
classification” of an ADHD subject. The most
commonly used cut-off value for the estimated
probability is 0.5 (Hosmer & Lemeshow, 1989).
and was thus chosen for the current study. There-
fore, an estimated probability value less than or
equal to 0.5 classified a subject as NC, while an
estimated probability value greater than 0.5 classi-
fied a subject as ADHD.
To assess the stability of the logistic model, it
was then cross-validated with an independent sam-
ple of ADHD (n = 15) and NC (n = 15) subjects
who were similar in demographic characteristics to
the original sample but who were not included in
the derivation of the original logistic model.
ADHD subjects were drawn from the same clinic
sample and met the same inclusionary criteria as
the original sample. All ADHD subjects included
in the cross-validation were male. The mean age of
these ADHD subjects was 9.2 years (SD= 1.7).
NC subjects were drawn from the original sample
(Akshoomoff & Stiles, 1995) with an effort to
match the ADHD cross-validation sample on age.
The mean age of the NC subjects was 9.1 (SD=
1.4) years and there were 6 girls and 9 boys. The
NC and ADHD cross-validation groups did not
differ significantly in age ( t = -.14, p > .05).
RESULTS
The medians of the BQSS items of interest at-
tained by both groups and corresponding Mann-
Whitney-U tests are presented in Table 2. NC
children performed significantly better than chil-
dren with ADHD on Configural Accuracy, Clus-
ter Accuracy, Detail Presence , Vertical Expan-
sion, Horizontal Expansion, and Neatness. The
difference between ADHD and NC children on
Perseveration approached significance (p < .07).
There were no group differences on Fragmenta-
tion or Planning. Figure l displays examples of
 BOSTON QUALITATIVE SCORING SYSTEM IN ADHD 40 1

Table 1. Description of Scores for the Boston Qualitative Scoring System (BQSS) for the Rey-Osterrieth Com-
plex Figure (ROCF). *

Score Description

1. Configural Presence The degree to which some representation of each of the six configural elements (i.e.,
rectangle, two diagonals, two bisectors, triangle on right) exists in the production,
regardless of the quality of those representations.
2. Configural Accuracy The quality of each of those configural elements present, based on the completeness
of element, accuracy of placement as related to other elements, size, proportion, etc.
3. Cluster Presence The degree to which some representation of each of the nine clusters (i.e., subunits
of the figure, such as the rectangle with diagonals in the left side of the figure [Clus-
ter I], the circle with three dots) exists in the production, regardless of the quality of
those representations.
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015

4. Cluster Accuracy The quality of each of those clusters present, based on the completeness of the cluster,
accuracy of placement and orientation of each cluster’s parts, size, proportion, etc.
5. Cluster Placement The degree to which each of the clusters present is positioned in the proper region of
the figure.
6. Detail Presence The degree to which some representation of each of the six detail elements (i.e., addi-
tional single lines in the figure, not included as either configural elements or clusters)
exists in the production, regardless of the quality of those representations.
7. Detail Placement The degree to which each of the detail elements present is positioned in the proper
region of the figure, especially in relation to configural elements.
8. Fragmentation The degree to which the configural elements and Cluster 1 are drawn as whole units
(e.g., completed in a single pen stroke or before another part of the figure is begun).
9. Planning The overall sense of importance given to the configural rectangle; the degree to
which the configural elements are completed prior to beginning the clusters and de-
tails; and the awareness of external structure, boundaries of the page, and centering
of the figure on the page.
10. Size Reduction The degree to which the figure has been reduced from the size of the original stimu-
lus (using a transparency template for scoring).
1 1. Vertical Expansion The degree to which the figure has been expanded along the vertical axis from the
size of the original stimulus (using a transparency template for scoring).
12. Horizontal Expansion The degree to which the figure has been expanded along the horizontal axis from the
size of the original stimulus (using a transparency template for scoring).
13. Rotation The degree to which the entire figure is rotated on the page (using a transparency
template for scoring).
14. Perseveration The amount of recognizable and inappropriate replication in the figure.
15. Confabulation The amount of additions to the figure that are foreign to the original stimulus and not
recognizable as perseverations.
16. Neatness The degree of care taken in producing a neat production, including the extent of dou-
ble drawn lines or overwriting, self-corrected errors, curved or tremulous lines, etc.
17. Right/Left Asymmetry An assessment of whether the figure is disproportionately distorted or lacking details
on one (right or left) side.

Note. All items are scored on a 0 (poor) to 4 (good) scale, with the exception of RighULeft Asymmetry which is
scored as either None, Right, or Left.
j: Scores are made using specific criteria, exemplars, and templates included in the Manual for the BQSS (avail-

able from Dr. Robert Stern, Neurobehavioral Research, Rhode Island Hospital, 1 10 Lockwood Street, Providence,
Rhode Island, 02903).
 402 DEBORAH A. CAHN ET AL

reproductions of the ROCF made by an ADHD expansion were generally quite effective for dis-
subject and by an age-matched control subject. tinguishing between ADHD and NC subjects.

Sensitiviv and Specijicig Logistic Regression Analysis

Table 3 displays the optimal cut-off score and
corresponding sensitivity and specificity for
classification obtained for each BQSS measure.
As illustrated in Table 3, the two measures of
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
The following variables were entered into the
logistic regression in a step-wise manner with
forward selection: Configural Accuracy, Cluster
Accuracy, Detail Presence, Fragmentation, Plan-

Fig. 1. Top: Rey-Osterrieth Complex Figure (ROCF) stimulus. Middle: Copy of the ROCF produced by a 12-
year-old healthy female without Attention Deficit Hyperactivity Disorder (ADHD). Bottom: Copy of
the ROCF produced by a 12-year-old male with ADHD diagnosed using DSM-IV criteria by a neuro-
psychologist and a pediatrician. Note the extreme horizontal and vertical expansion in the production
by the child with ADHD.
 BOSTON OUALITATIVE SCORING SYSTEM IN ADHD 403

Table 2. Median Scores (Rank Sums) of the Selected BQSS Measures Obtained by the Attention Deficit Hyper-
activity Disorder (ADHD) Group and Normal Control (NC) Group.

NC ADHD Mann-Whitney U Statistic

Configural Accuracy 1.0 (1827.0) 0.0 (1254.0) 1047.0**
Cluster Accuracy 1.0 (1782.5) 1.0 (1298.5) 1002.5**
Detail Presence 4.0 (1829.0) 3.0 (1252.0) 1049.0**
Fragmentation 1.0 (1415.5) 2.0 (1665.5) 635.5
Planning 2.0 (1623.5) 1.0 (1457.5) 843.5
Vertical Expansion 4.0 (2046.5) 2.0 (1034.5) 1266.5* *
Horizontal Ex pans i on 4.0 (2025.0) 2.0 (1056.0) 1245.0**
Perseveration 2.0 (1712.5) 1.0 (1368.5) 932.5
Neatness 2.0 (1765.5) 1.0 (1315.5) 985.5*
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015

* p < .05, **p < .01.

Table 3. Sensitivity and Specificity for the BQSS Measures.

Cut-off score' Sensitivity Specificity

Configural Accuracy 0 69% 64%
Cluster Accuracy 1 97% 26%
Detail Presence 3 77% 59%
Fragmentation 2 74% 26%
Planning 0 21% 92%
Vertical Expansion 2 64% 97%
Horizontal Expansion 3 82% 67%
Perseveration 2 77% 46%
Neatness 1 62% 67%

'Those subjects scoring at or below the cut-off score were considered ADHD; those scoring above the cut-off
score were considered Normal.

ning, Vertical Expansion, Horizontal Expansion,
Perseveration, and Neatness. Using an a priori
probability value of .05 for inclusion of vari-
ables in the model, the following logistic equa-
tion for computing the probability of being clas-
sified with ADHD was produced:
Estimated Probability for the diagnosis of
e_
x for Vertical Expansion and Detail Presence
ADHD= _
l+e
where x = 4.46 + (- 4.72 Vertical Expansion) +
(- 2.26 Detail Presence). Using a cut-off proba-
bility estimate of 0.50, 64% of ADHD and 97%
of NC subjects were correctly classified by the
logistic model. Eighty-one percent of all sub-
jects were correctly classified using this model.
The classification results, based upon the logis-
tic regression model are presented in Table 4.
To assess the stability of the logistic model,
the classification procedure was cross-validated
with an independent sample of 30 subjects as
described above. When the dichotomized scores
(based on the optimal cut-off scores) of these
subjects were entered into the logistic model,
67% of ADHD and 80% of NC subjects were
correctly classified. The final results of the clas-
sification procedure are presented in Table 4.
 404 DEBORAH A. CAHN ET AL.
Table 4. Classification Table of ADHD and NC Subjects Based on the Logistic Regression Model.
Classification by the Logistic Model
Clinical Diagnosis
Original Sample
ADHD 25/39
NC 1/39
Cross-validation Sample
ADHD 10115
NC 3/15
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
DISCUSSION
The present study sought to examine differences
between children with ADHD and age-matched
control children on the reproduction of the
ROCF. To this end, the BQSS, a scoring system
providing quantitative analysis of specific quali-
tative features of visuoconstruction, was utilized
to capture potential impairments in executive
functioning. The results of this study revealed
that children with ADHD but otherwise average
visuomotor and visuoperceptual skills did, in-
deed, display impairments on their reproduc-
tions of the ROCF that reflected underlying dif-
ficulties with accuracy, attention to detail, and
inhibition, relative to normal controls. Specifi-
cally, children with ADHD earned lower scores
on BQSS measures of Configural and Cluster
Accuracy, Detail Presence, Vertical and Hori-
zontal Expansion, and Neatness. Contrary to a
priori hypotheses, the ADHD children were not
impaired relative to normal controls on the
BQSS Planning measure.
There are several possible explanations for
this unexpected finding. First, because the
BQSS was initally developed for adult popula-
tions, it is possible that the Planning measure is
not appropriate for use with child populations as
currently defined. It has been reported (Akshoo-
moff & Stiles, 1995) that the BQSS Planning
measure was not sufficiently sensitive to devel-
opmental trends. Second, as Table 2 indicates,
both subject groups performed poorly on this
item, suggesting that a floor effect may have
ADHD NC
(64%) 14/39 (36%)
(3%) 38/39 (97%)
(67%) 5/15 (33%)
(20%) 12/15 (80%)
obscured any possible group differences. That
is, children in this age group (6 to 12 years of
age) may exhibit poor planning ability when the
adult-based BQSS criteria are employed.
Akshoomoff and Stiles (1995) have developed a
modified planning score that is more sensitive to
age-related changes in planning performance
and that takes into account strategies that chil-
dren use in starting their drawing that may ex-
plain differences in the final organization of the
reproduction.
The results of the sensitivity and specificity
analyses revealed that specific BQSS measures
discriminated between ADHD and NC children.
The BQSS items found to have the best
discriminative ability were those of Horizontal
and Vertical Expansion. It is possible that these
measures may reflect impairment in planning
ability and impulsivity inasmuch as expansion
of the figure suggests difficulty anticipating the
future requirements of the task, impairments in
self-monitoring, and behavioral dyscontrol. The
logistic model derived in this study demon-
strated that the measures most effective for the
differentiation of ADHD from NC children were
Vertical Expansion and Detail Presence. In ag-
gregate, these variables resulted in 64% sensitiv-
ity an 97% specificity. In the cross-validation,
the classification accuracy was moderate, with a
correct classification of 67% of the children
with ADHD and 80% correct classification of
NC children. The variables comprising the lo-
gistic model appear to tap the organization and
attention to detail problems that are thought to
 BOSTON QUALITATIVE SCORING SYSTEM IN ADHD
be the cardinal features of the disorder. It should
be noted that Horizontal Expansion alone pro-
vided 84% sensitivity and 67% specificity, but
did not load into the logistic model, most likely
because of shared variance with the other mea-
sures (i.e., Vertical Expansion).
Despite average visuomotor and visual inte-
gration abilities, the children with ADHD in the
current study demonstrated impairments on their
reproductions of the ROCF. Analysis of their
productions using the BQSS suggests that un-
derlying impairments in accuracy, impulse con-
trol, and organization, reflecting the executive
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
dysfunction associated with this disorder, re-
sulted in the children with ADHD obtaining
scores that were significantly lower than the
scores obtained by normal, age-matched chil-
dren. These findings are consistent with those of
other researchers who have found that ADHD
children perform more poorly than normal chil-
dren when task demands are complex (Barkley,
1988; Chelune et al., 1986; Douglas, 1983; Fel-
ton et a]., 1987; Gorenstein et al., 1989; Grod-
zinsky & Diamond, 1992; Homatidis & Kon-
stantareas, 1981; Kuehne et al., 1987; Reader et
al., 1994).
The ability to detect these executive impair-
ments was enhanced by the use of a scoring sys-
tem sensitive to these process variables, and
may have been overlooked if the traditional sin-
gle score (i.e., 36 point) system (Lezak, 1995)
had been employed. Given the high frequency
with which the ROCF is used in pediatric neuro-
psychology evaluations, the current study lends
support for the introduction of the use of the
BQSS in scoring the ROCF with this population
of children.
As previously discussed, however, modifica-
tions to the present system may need to be made
to accommodate issues in visuospatial skill de-
velopment (Akshoomoff & Stiles, 1995). Never-
theless, the current study provides initial support
for the use of the BQSS in assessing visuo-
constructional and executive functioning diffi-
culties in ADHD. Indeed, two of the most sensi-
tive measures, Horizontal and Vertical Expan-
sion, can be readily assessed (i.e., in less than 1
min) with BQSS templates (Stern et al., 1994).
405
One limitation of the current study is that the
model employed only provides information with
regard to the differentiation of ADHD from nor-
mal children. The differential diagnosis of
ADHD often involves ruling out other childhood
disorders, such as learning disabilities and con-
duct disorder. Barkley (1990) has suggested that
behavioral rating scales such as the Revised
Conners Behavioral Rating Scales (Goyette,
Conners, & Ulrich, 1978) and Child Behavior
Checklist (Achenbach & Edelbrock, 1983) play
an important role in the diagnosis of ADHD.
The role of neuropsychological testing in the
diagnosis of ADHD has been disputed. This
may, in part, reflect the fact that the criteria for
the diagnosis of ADHD are behaviorally rather
than cognitively delineated (APA, 1994). The
generally equivocal findings of neuropsycholog-
ical test performance in children with ADHD
may be a function of the heterogeneity and
comorbidity of other disorders seen in this syn-
drome. Although the logistic model derived in
this study provided good diagnostic accuracy, it
may perhaps be further enhanced by adding
other neuropsychological measures of executive
dysfunction as well as behavioral rating scales.
Future studies examining the predictive ability
of a spectrum of psychometric and behavioral
measures may result in a statistical model with
very high sensitivity and specificity for the clas-
sification of ADHD.
The current exploratory study lends prelimi-
nary support for the inclusion of neuropsycho-
logical measures such as the ROCF in the com-
prehensive evaluation of ADHD. Future studies
should be undertaken to better understand the
predictive properties of qualitative neuropsycho-
logical instruments such as the BQSS in discri-
minating ADHD from other common childhood
disorders.
In summary, several BQSS measures appear
to have some utility in detecting underlying
qualitative differences in executive functioning
between ADHD and NC children. Improved pre-
dictive accuracy might result from the inclusion
of developmentally driven measures of planning
that take into consideration factors that affect
children’s performance on this task (Akshoo-
moff & Stiles, 1995) as well as behavioral rating
 406 DEBORAH A. CAHN ET AL.
scales (Barkley, 1990).Continued exploration of
the clinical and research utility of this instru-
ment with different child populations appears
warranted.
REFERENCES
Achenbach, T. M., & Edelbrock, C. (1983). Manual
for the Child Behavior Checklist and Revised Child
Behavior Profile. Burlington: University of Ver-
mont, Department of Psychiatry.
Akshoomoff, N. A., & Stiles, J. (1995).Developmen-
Downloaded by [JAMES COOK UNIVERSITY] at 12:02 17 March 2015
tal trends in visuospatial analysis and planning:
Copying a complex figure. Neuropsychology, 9,
364-377.
American Psychiatric Association (1994). Diagnostic
and statistical manual of mental disorders (4th
ed.). Washington, DC: Author.
Barkley, R. A. (1988). Attention. In M. G. Tramon-
tana & S. R. Hooper (Eds.), Assessment issues in
child neuropsychology (pp. 145-176). New York:
Plenum.
Barkley, R. A. (1990). Attention deficit hyperactivity
disorder: A handbook for diagnosis and treatment.
New York, NY: Guilford Press.
Barkley, R. A., & Grodzinsky, G. M. (1994). Are tests
of frontal lobe functions useful in the diagnosis of
Attention Deficit Disorders? The Clinical Neuro-
psychologist, 8, 121-139.
Beery, K. E. (1989). The Developmental Test of Vi-
sual-Motor Zntegration: Administration, scoring
and teaching manual (3rd ed.). Cleveland, OH:
Modem Curriculum Press.
Chelune, G. J., Ferguson, W., Koon, R., & Dickey, T.
0. (1986). Frontal lobe disinhibition in attention
deficit disorder. Child Psychiatry and Human De-
velopment, 16,221-234.
Corwin, J., & Blysma, F. W. (1993). Translation: Psy-
chological examination of traumatic encaphalo-
pathy and the Complex Figure Copy Test. The
Clinical Neuropsychotogist, 7 , 3-21.
Douglas, V. I. (1983). Attentional and cognitive prob-
lems. In M. Rutter (Ed.), Developmental neuropsy-
chiafry (pp. 280-329). New York: Guilford Press.
Felton, R. H., Wood, F. B., Brown, I. S., Cambell, S .
K., & Harter, M. R. (1987). Separate verbal mern-
ory and naming deficits in attention deficit disorder
and reading disability. Brain and Language, 31,
171-184.
Frost, L. A., Moffitt, T. E., & McGee, R. (1989). Neu-
ropsychological correlates of psychopathology in
an unselected cohort of young adolescents. Journal
of Abnormal Psychology, 98, 307-313.
Gorenstein, E. E., Mammato, C. A., & Sandy, I. M.
(1989). Performance on inattentive-overactive chil-
dren on selected measures of prefrontal-type func-
tion. Journal of Clinical Psychology, 45,619-632.
Goyette, C . H., Conners, C. K., & Ulrich, R. F.
(1 978). Normative data on Revised Conners Parent
and Teacher Rating Scales. Journal of Abnormal
Child Psychology, 6,221-236.
Grodzinsky, G. M., & Diamond, R. (1992). Frontal
lobe functioning in boys with attention-deficit hy-
peractivity disorder. Developmental Neuropsychol-
ogy, 8,427-445.
Homatidis, S . , & Konstantareas, M. M. (1981). As-
sessment of hyperactivity: Isolating measures of
high discriminant ability. Journal of Consulting
and Clinical Psychology, 47,533-541.
Hooper, H. E. (1958). The Hooper Visual Organiza-
tion Test Manual. Los Angeles, CA: Western Psy-
chological Services.
Hosmer, D. W., & Lemeshow, S. (1989). Applied lo-
gistic regression. New York: John Wiley & Sons.
Kuehne, C., Kehle, T. J., & McMahon, W. (1987).
Differences between children with attention deficit
disorder, children with specific learning disabili-
ties, and normal children. Journal of School Psy-
chology, 25,161-166.
Lezak, M. (1995). Neuropsychological assessment
(3rd ed.). New York, N Y Oxford University Press.
Ostenieth, P. A. (1944). La test d'une figure com-
plexe. Archives de Psychologie, 30,206-353.
Reader, M. J., Harris, E. L., Schuerholz, L. J., &
Denckla, M. B. (1994). Attention deficit disorder
and executive dysfunction. Developmental Neuro-
psychology, 10,493-512.
Rey, A. (1941). L'examen psychologique dans les cas
d'encephalopathie traumatique. Archives de
psychologie, 28, 286-340.
Stern, R. A., Singer, E. A,, Duke, L. M., Singer, N.
G., Morey, C. E., Daughtrey, E. W., & Kaplan, E.
(1 994). The Boston Qualitative Scoring System for
the Rey-Osterrieth Complex Figure: Description
and interrater reliability. The Clinical Neuropsy-
chologist, 8,309-322.