This article was downloaded by: [Universidad Nacional Colombia]

On: 30 September 2012, At: 19:33

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Personality Assessment

Publication details, including instructions for authors and subscription
information:
http://www.tandfonline.com/loi/hjpa20

Drawing Conclusions--Or Not-From

Drawings
Thomas E. Joiner Jr. & Kristen L. Schmidt
Version of record first published: 10 Jun 2010.

To cite this article: Thomas E. Joiner Jr. & Kristen L. Schmidt (1997): Drawing Conclusions--Or Not-From
Drawings, Journal of Personality Assessment, 69:3, 476-481

To link to this article: http://dx.doi.org/10.1207/s15327752jpa6903_2

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Any substantial
or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply,
or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand,
or costs or damages whatsoever or howsoever caused arising directly or indirectly in
connection with or arising out of the use of this material.
 JOURNAL OF PERSONALITY ASSESSMENT, 1997,69(3),476-481
Copyright O 1997, Lawrence Erlbaum Associates, Inc.

Drawing Conclusions-Or Not-

From Drawings
Thomas E. Joiner, Jr.
Division of Child and Adolescent Psychiatry
Department of Psychiatry and Behavioral Sciences
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012

University of Texas Medical Branch at Galveston

Kristen L. S c h m i d t
Prevention Research Center
Department of Mental Hygiene
Johns Hopkins University

Riethmiller and Handler (this issue) critiqued our article on drawing indices and
emotional distress (Joiner, Schmidt, & Barnett, 1996). On several counts, we find
their points unconvincing. Specifically,we reject the possibilities that our null results
are explained by (a) the distinction between self-attributed and implicit motives, (b)
measurement error, and (c) curvilinear relations between variables. Furthermore, we
document, using Riethmiller and Handler's own data, that previous research on
drawings, construed by Riethmiller and Handler as supportive, actually undermines
the validity of drawings as indices of emotional distress.

In their response to our article on drawings as measures of emotional distress

(Joiner, Schmidt, & Barnett, 1996), Riethmiller and Handler (this issue) make some
agreeable points, as follows:

1. It is a well-known psychometric truism that single-item measures may

suffer from reliability problems.
2. Drawing indices may display a curvilinearrelation to measures of emotional
distress.
3. Our data apply to three specifically defined indices and do not necessarily
generalize to other drawing indices.
4. There may be a difference between self-attributed and implicit motives, and
this difference may have implications for comparisons of self-report to
projective assessments.
 DRAWING CONCLUSIONS 477

Others of Riethmiller's and Handler's (this issue) points, as well as some of the
conclusions they draw from otherwise legitimate points, are less persuasive. We
discuss these in turn.
First, Riethmiller and Handler (this issue) state, "Recent research repeatedly
demonstrates that self-report measures miss essential information that is captured
by projective measures (Bornstein, Bowers, & Robinson, 1995; McClelland,
Koestner, & Weinberger, 1989; Meyer, 1996, 1997; Shedler, Mayman, & Manis,
1993)."
This statement is problematic in many ways. Not only does it beg the question
of what constitutes "essential information," the statement is exaggerated and not
relevant to our article on drawings. Consider, for emmple, the interesting study by
Shedler et al. (1993; commented on, incidentally, by Joiner, 1994). These authors
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012

demonstrated that clinician-ratings-a far cry fro@ projective drawings-may add

important information to, and may even gainsay, self-reporlt assessment. We
entirely agree.
Or, take the review by McClelland et al. (1989) and the stiicly by Bornstein et
al. (1995), both of which emphasize the interesting distinction between self-&rib-
uted and implicit motives. Neither of these articles addresses projective drawings,
and neither demonstrates that self-report measures miss essential information,
although they do show that they may miss some information. Similarly, the articles
by Meyer (1996,1997) do not involve drawings and do not show that the objective
personality measures (in this case, the MinnesotaMtllltiphasicPersonality Inventory
[MMPT])miss essential information.
It is important to note that the distinction betwpen self-attributed and implicit
motives cannot explain the fact that the drawing indices used in our study,
supposedly measures of implicit motives, did not correlate with thematic stories,
another measure of implicit motives. Indeed, thay did not even correlate with
themselves.
We suggest that clinicians interested in essential information about a child's
emotional and behavioral status do the following (in this order): (a) ask the child;
(b) observe the child; (c) ask the child's caretakers and teachers; and (d) administer
validated assessment instrumentsto children, teachers, and parents, such as any one
or combination of the following,as appropriate: the Bersonality Inventory for Youth
(Lachar & Gruber, 1995), the Fmsonality Inventory for Children (Wirt, Lachar,
Klinedinst, & Seat, 1984), the MMPI-A (for adoles(;ents;Butcher et al., 1992), the
Child Behavior Checklist (Achanbach & Edelbrack, 1983), the Clonners Behavior
Rating Scales (Canners, 1969, 1970), the Clhildren's Depression Inventory (CDI;
Kovacs, 1992), and the Revised Children's Manifest Anxiety Scale (RCMAS;
Reynolds & Richmond, 1985). To imagine that p~ojectivedravvings would add
diagnostic, prognostic, or (other) treatment-relevant information to this approach
is, at best, difficult.
 478 JOINER AND SCHMIDT

Second, Riethmiller and Handler (this issue) question whether the drawing
indices used in our study possessed low reliability.
Insofar as a main point of our study was to empirically document that these
drawing indices possessed very high reliability,Riethmiller and Handler's question
about reliability is puzzling. Reliability estimates for the indices averaged .93 (each
was > .90). Riethmiller and Handler are right to note that single-item measures may
suffer low reliability, but they overlook the fact that our drawing indices do not.
Furthermore, it is arguable whether our drawing indices, which were scored based
on global estimates af independent raters, are analogous to a single-item measure.
What is not arguable is that our drawing indices are about as error-free as it gets in
behavioral science research.
In a related manner, Riethmiller and Handler (this issue) rhetorically wondered
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012

whether one would expect a single item from a test to correlate with the results of
distinct but conceptually related tests. Actually, we would, as long as the single
item, although of (probably) low reliability, is a valid indicator of the underlying
construct in question.
Consider, for example, data from a separate study of youth psychiatric patients
(Joiner, 1997), 60 of whom completed the CDI and the RCMAS-both measures
of general emotionaldistress and symptoms. Two months later, 34 of these children
completed the same scales. These data allowed us to estimate the reliability of
individual CDI and RCMAS items (using test-retest), and to deterrhine whether a
single CDI item correlated with the RCMAS Total score and whether a single
RCMAS item correlated with the CDI Total score. To move clearly drpmonstrate
our point, we chose a CDI item and an RCMAS item with diffWhg contents (CDI
Item 24: "I can never be as good as other kids;" RCMAS I&m 5: " O h n I have
trouble getting my breath.").
As might be eapected, the test-retest reliability of these items was low (approxi-
mately .30).Nonetheless, there were significant correlations between the CDI item
and the RCMAS Total score, r = .50, p <: .O5, and between the RCMAS Item and
the CDl Total score;,r = .26,p < .05. It is noteworthy that a virtually identical pattern
of findings emergad when we similarly examined sevaral othm CDI and RCMAS
items.
?Jle pattern of these results, considered together with data from our original
article on drawings, bears emphasis. Even despite low rdliability, sihglle-item
measures of emotional distress correlated in expected ways with sepmate scalds of
emotional distress. This was the case even though the s ~ d t moft two items was
quib different. By contrast, even though the drawi~gindims evinoed veky high
reliability, they do not correlate with much of anythiag, not evvieb emh other. For
the data on the CDI md RCMAS itkms and scales, theitemsbdidity shone:through
their reliability problems; for the drawing indices, th@r@ were exaelIeat conditions
for validity to shine through (i.e., high reliability), but it did not, hcause, we argue,
it does not exist.
 DRAWING CONCLUSIONS 479

Third, Riethmiller and Handler (this issue) comment that "both small size and
large size extremes indicate anxiety or stress" (i.e., that the relation between
drawing size and emotional distress may be nonlinear). We therefore empirically
addressed the possibility that the drawing indices used in our study possessed a
curvilinear relation to emotional distress.
There was no evidence whatsoever for this possibility. Specifically,using power
polynomials in the context of hierarchical multiple regressionl~orrelation(Cohen
& Cohen, 1983, pp. 224-234), we tested whether my of the three drawing indices
displayed either a quadratic or cubic relation to either CDI scores, RCMAS scores,
or scores from the Depression and Anxiety scales of the Roberts Apperception Test
for Children. It should be noted that the quadratic analysis specifically tests
Riethmiller and Handler's (this issue) hypothesisthat both small size and large size
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012

extremes indicate anxiety or stress. In no case did the F value fior the quadratic or
cubic term achieve significance, or even approach it. Based on th~esedata, we reject
Riethmiller and Handler's hypothesis.
Fourth, Riethmiller and Handler (this issue) question our interpretatnon of
Handler and Reyher's (1965) review, especially the data in their Table 1. In our
original article, we construed these data as unsupportive of the validity of drawing
indices.
Overall, the kappa coefficient for agreement between clinicians and drawing-
based interpretations, as presented in Riethrniller and Handler's (this issue) Table
1, is .17. It is on this basis that we claim that these data are unsupportive of the
validity of projective drawings.
Of course, one may find what one seeks, and Riemiller and Handler (this issue)
are impressed that a few of the drawing indices in Table 1 perform relatively well.
This is dso the case regarding data from our Table 1 of this article. The structure
and format of the table is similar to Riethmiller and Handler's Table 1. As can be
seen in our Table 1, visual inspection of rows 7,8, 13, and 20 indicates that these
indices perform well. The problem, however, is that our data in Table 1 were
generated by SPSS's random number generator. Riethmiller and Handler's Table
1 is not very different from a table of random numbers; the kappa coefficient was
similarly close to zero, and several rows looked pretty good. The question, then,
about the "good-1ooking"rows in Riethmiller and Handler's table is not whether
they look good-they do, as do some of our random numbers. Rather, the question
is whether these indices have been subsequently supported empirically. It is
interesting to note that two of these indices (detail and line heaviness) have been
subsequently unsupported by our original article.
Finally, Riethmiller and Handler's (this issue) report, especially the concluding
paragraph, allows us credit as skeptics, but then implies that we are defeatist,
conceptually uninformed, and reactionary (see the title of their article). They have
made a drawing of us, our experience-near interpretation of which we do not like.
Happily, this drawing, like the drawing indices from our original article, is invalid.
 480 JOINER AND SCHMIDT

TABLE 1
Random Numbers

Index Agreement Opposite Findings Nonsignificant Findings

Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012

Note. Numbers in table generated by SPSS's random number generator.

REFERENCES

Achenbach, T. M., & Edelbrock, C. S. (1983). Manual for the Child Behavior Checklist and Revised
Child Behavior Profile. Burlington, VT: University Associates in Psychiatry.
Bornstein, R. F., Bowers, S. K., &Robinson, K. J. (1995). Differential relationships of objective and
projective dependency scores to self reports of interpersonal life events in college student subjects.
Journal of Personality Assessment, 65, 255-269.
Butcher, J. N., Williams, C. L., Graham, J. R., Archer, R. P., Tellegen, A,, Ben-Porath, Y. S., &
Kaemmer, B. (1992). MMPI-A (Minnesota Mulripharic Personality Inventory-Adolescent): Man-
ual for administration, scoring, and interpretation.Minneapolis: University of Minnesota Press.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral
sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Conners, C. K. (1969). A teacher rating scale for use in drug studies with children. American J o u m l
o f Psychiatry, 126, 884-888.
Conners, C. K. (1970). Symptom patterns in hyperkinetic, neurotic, and normal children. Child
Development, 41, 667-682.
Handler, L., & Reyher, J. (1965). Figure drawing anxiety indices.Journal of Projective Techniques, 29,
305-313.
Joiner, T. E., Jr. (1994). On denial and defensiveness in experimental psychopathology. American
Psychologisf, 49, 973-974.
 DRAWING CONCLUSIONS 481

Joiner, T. E., Jr. (1997). A test of the causal aspects of the hopelessness theory of depression among
youth psychiatric inpatients. Manuscript submitted for publication.
Joiner, T. E., Jr., Schmidt, K. L., & Barnett, J. (1996). Size, dletail, and line heaviness in children's
drawings as correlates of emotional distress: (More) negative evidence. Jountal of Personality
Assessment, 67, 127-141.
Kovacs, M. (1992). Children's Depression Inventory Manual. Los Angeles: Western Psychological
Services.
Lachar, D., & Gruber, C. P. (1995). Personality Inv,entoryfor Youth Manual. Los Angeles: Western
Psychological Services.
McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How do self attributed and implicit motives
differ? Psychological Review, 96, 690-702.
Meyer, G. J. (1996). The Rorschach and MMPI: Toward a more scientifically differentiated under-
standing of cross-method assessment. Journal of Persona1it)i Assessment, 67, 558-578.
Meyer, G. J. (1997).On the integration of personality assessment methods: The Rorschach and MMPI.
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012

Journal of Personality Assessment, 68,297-330.

Reynolds, C. R., & Richmond, B. 0. (1985). Revised Children's Manifest Anxiety Scale (RCMAS)
Manual. Los Angeles: Western Psychological Services.
Shedler, J.. Mayman, M., & Manis, K. (1993). The illusion of mental health. American Psychologist,
48, 1117-1131.
W k t R. D., Lachar, D., Klinedinst, J. K., &Seat, P. D. (1984). Multid~mensionaldescription of child
personality: A manual for the Personality Inventoryfor Children. Los Angeles: Western Psycho-
logical Services.

Thomas E. Joiner, Jr.

Department of Psychiatry
University of Texas Medical Branch at Galveston
Galveston, TX 775554425

Received January 2 1,1997