Documente Academic
Documente Profesional
Documente Cultură
To cite this article: Thomas E. Joiner Jr. & Kristen L. Schmidt (1997): Drawing Conclusions--Or Not-From
Drawings, Journal of Personality Assessment, 69:3, 476-481
This article may be used for research, teaching, and private study purposes. Any substantial
or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply,
or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand,
or costs or damages whatsoever or howsoever caused arising directly or indirectly in
connection with or arising out of the use of this material.
JOURNAL OF PERSONALITY ASSESSMENT, 1997,69(3),476-481
Copyright O 1997, Lawrence Erlbaum Associates, Inc.
Kristen L. S c h m i d t
Prevention Research Center
Department of Mental Hygiene
Johns Hopkins University
Riethmiller and Handler (this issue) critiqued our article on drawing indices and
emotional distress (Joiner, Schmidt, & Barnett, 1996). On several counts, we find
their points unconvincing. Specifically,we reject the possibilities that our null results
are explained by (a) the distinction between self-attributed and implicit motives, (b)
measurement error, and (c) curvilinear relations between variables. Furthermore, we
document, using Riethmiller and Handler's own data, that previous research on
drawings, construed by Riethmiller and Handler as supportive, actually undermines
the validity of drawings as indices of emotional distress.
Others of Riethmiller's and Handler's (this issue) points, as well as some of the
conclusions they draw from otherwise legitimate points, are less persuasive. We
discuss these in turn.
First, Riethmiller and Handler (this issue) state, "Recent research repeatedly
demonstrates that self-report measures miss essential information that is captured
by projective measures (Bornstein, Bowers, & Robinson, 1995; McClelland,
Koestner, & Weinberger, 1989; Meyer, 1996, 1997; Shedler, Mayman, & Manis,
1993)."
This statement is problematic in many ways. Not only does it beg the question
of what constitutes "essential information," the statement is exaggerated and not
relevant to our article on drawings. Consider, for emmple, the interesting study by
Shedler et al. (1993; commented on, incidentally, by Joiner, 1994). These authors
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012
Second, Riethmiller and Handler (this issue) question whether the drawing
indices used in our study possessed low reliability.
Insofar as a main point of our study was to empirically document that these
drawing indices possessed very high reliability,Riethmiller and Handler's question
about reliability is puzzling. Reliability estimates for the indices averaged .93 (each
was > .90). Riethmiller and Handler are right to note that single-item measures may
suffer low reliability, but they overlook the fact that our drawing indices do not.
Furthermore, it is arguable whether our drawing indices, which were scored based
on global estimates af independent raters, are analogous to a single-item measure.
What is not arguable is that our drawing indices are about as error-free as it gets in
behavioral science research.
In a related manner, Riethmiller and Handler (this issue) rhetorically wondered
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012
whether one would expect a single item from a test to correlate with the results of
distinct but conceptually related tests. Actually, we would, as long as the single
item, although of (probably) low reliability, is a valid indicator of the underlying
construct in question.
Consider, for example, data from a separate study of youth psychiatric patients
(Joiner, 1997), 60 of whom completed the CDI and the RCMAS-both measures
of general emotionaldistress and symptoms. Two months later, 34 of these children
completed the same scales. These data allowed us to estimate the reliability of
individual CDI and RCMAS items (using test-retest), and to deterrhine whether a
single CDI item correlated with the RCMAS Total score and whether a single
RCMAS item correlated with the CDI Total score. To move clearly drpmonstrate
our point, we chose a CDI item and an RCMAS item with diffWhg contents (CDI
Item 24: "I can never be as good as other kids;" RCMAS I&m 5: " O h n I have
trouble getting my breath.").
As might be eapected, the test-retest reliability of these items was low (approxi-
mately .30).Nonetheless, there were significant correlations between the CDI item
and the RCMAS Total score, r = .50, p <: .O5, and between the RCMAS Item and
the CDl Total score;,r = .26,p < .05. It is noteworthy that a virtually identical pattern
of findings emergad when we similarly examined sevaral othm CDI and RCMAS
items.
?Jle pattern of these results, considered together with data from our original
article on drawings, bears emphasis. Even despite low rdliability, sihglle-item
measures of emotional distress correlated in expected ways with sepmate scalds of
emotional distress. This was the case even though the s ~ d t moft two items was
quib different. By contrast, even though the drawi~gindims evinoed veky high
reliability, they do not correlate with much of anythiag, not evvieb emh other. For
the data on the CDI md RCMAS itkms and scales, theitemsbdidity shone:through
their reliability problems; for the drawing indices, th@r@ were exaelIeat conditions
for validity to shine through (i.e., high reliability), but it did not, hcause, we argue,
it does not exist.
DRAWING CONCLUSIONS 479
Third, Riethmiller and Handler (this issue) comment that "both small size and
large size extremes indicate anxiety or stress" (i.e., that the relation between
drawing size and emotional distress may be nonlinear). We therefore empirically
addressed the possibility that the drawing indices used in our study possessed a
curvilinear relation to emotional distress.
There was no evidence whatsoever for this possibility. Specifically,using power
polynomials in the context of hierarchical multiple regressionl~orrelation(Cohen
& Cohen, 1983, pp. 224-234), we tested whether my of the three drawing indices
displayed either a quadratic or cubic relation to either CDI scores, RCMAS scores,
or scores from the Depression and Anxiety scales of the Roberts Apperception Test
for Children. It should be noted that the quadratic analysis specifically tests
Riethmiller and Handler's (this issue) hypothesisthat both small size and large size
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012
extremes indicate anxiety or stress. In no case did the F value fior the quadratic or
cubic term achieve significance, or even approach it. Based on th~esedata, we reject
Riethmiller and Handler's hypothesis.
Fourth, Riethmiller and Handler (this issue) question our interpretatnon of
Handler and Reyher's (1965) review, especially the data in their Table 1. In our
original article, we construed these data as unsupportive of the validity of drawing
indices.
Overall, the kappa coefficient for agreement between clinicians and drawing-
based interpretations, as presented in Riethrniller and Handler's (this issue) Table
1, is .17. It is on this basis that we claim that these data are unsupportive of the
validity of projective drawings.
Of course, one may find what one seeks, and Riemiller and Handler (this issue)
are impressed that a few of the drawing indices in Table 1 perform relatively well.
This is dso the case regarding data from our Table 1 of this article. The structure
and format of the table is similar to Riethmiller and Handler's Table 1. As can be
seen in our Table 1, visual inspection of rows 7,8, 13, and 20 indicates that these
indices perform well. The problem, however, is that our data in Table 1 were
generated by SPSS's random number generator. Riethmiller and Handler's Table
1 is not very different from a table of random numbers; the kappa coefficient was
similarly close to zero, and several rows looked pretty good. The question, then,
about the "good-1ooking"rows in Riethmiller and Handler's table is not whether
they look good-they do, as do some of our random numbers. Rather, the question
is whether these indices have been subsequently supported empirically. It is
interesting to note that two of these indices (detail and line heaviness) have been
subsequently unsupported by our original article.
Finally, Riethmiller and Handler's (this issue) report, especially the concluding
paragraph, allows us credit as skeptics, but then implies that we are defeatist,
conceptually uninformed, and reactionary (see the title of their article). They have
made a drawing of us, our experience-near interpretation of which we do not like.
Happily, this drawing, like the drawing indices from our original article, is invalid.
480 JOINER AND SCHMIDT
TABLE 1
Random Numbers
REFERENCES
Achenbach, T. M., & Edelbrock, C. S. (1983). Manual for the Child Behavior Checklist and Revised
Child Behavior Profile. Burlington, VT: University Associates in Psychiatry.
Bornstein, R. F., Bowers, S. K., &Robinson, K. J. (1995). Differential relationships of objective and
projective dependency scores to self reports of interpersonal life events in college student subjects.
Journal of Personality Assessment, 65, 255-269.
Butcher, J. N., Williams, C. L., Graham, J. R., Archer, R. P., Tellegen, A,, Ben-Porath, Y. S., &
Kaemmer, B. (1992). MMPI-A (Minnesota Mulripharic Personality Inventory-Adolescent): Man-
ual for administration, scoring, and interpretation.Minneapolis: University of Minnesota Press.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral
sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Conners, C. K. (1969). A teacher rating scale for use in drug studies with children. American J o u m l
o f Psychiatry, 126, 884-888.
Conners, C. K. (1970). Symptom patterns in hyperkinetic, neurotic, and normal children. Child
Development, 41, 667-682.
Handler, L., & Reyher, J. (1965). Figure drawing anxiety indices.Journal of Projective Techniques, 29,
305-313.
Joiner, T. E., Jr. (1994). On denial and defensiveness in experimental psychopathology. American
Psychologisf, 49, 973-974.
DRAWING CONCLUSIONS 481
Joiner, T. E., Jr. (1997). A test of the causal aspects of the hopelessness theory of depression among
youth psychiatric inpatients. Manuscript submitted for publication.
Joiner, T. E., Jr., Schmidt, K. L., & Barnett, J. (1996). Size, dletail, and line heaviness in children's
drawings as correlates of emotional distress: (More) negative evidence. Jountal of Personality
Assessment, 67, 127-141.
Kovacs, M. (1992). Children's Depression Inventory Manual. Los Angeles: Western Psychological
Services.
Lachar, D., & Gruber, C. P. (1995). Personality Inv,entoryfor Youth Manual. Los Angeles: Western
Psychological Services.
McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How do self attributed and implicit motives
differ? Psychological Review, 96, 690-702.
Meyer, G. J. (1996). The Rorschach and MMPI: Toward a more scientifically differentiated under-
standing of cross-method assessment. Journal of Persona1it)i Assessment, 67, 558-578.
Meyer, G. J. (1997).On the integration of personality assessment methods: The Rorschach and MMPI.
Downloaded by [Universidad Nacional Colombia] at 19:33 30 September 2012