Beutler, L. E., Harwood, T. M., Kimpara, S., Verdirame, D., & Blau, K. (in press). Coping Style.

In Norcross, J. C. (Ed). Relationships that work: Therapist Contributions and Responsiveness to Patient Needs (2nd ed). New York: Oxford University Press.

Coping Style Larry E. Beutler Satoko Kimpara David Verdirame T. Mark Harwood Kathy Blau

University of Palo Alto Pacific Graduate School of Psychology

Correspondence should be addressed to: Larry E. Beutler, PhD Palo Alto Univesity 1791 Arastadero Rd Palo Alto, CA 94063

Voice: (530) 642-1353 (home) e-mail: <larrybeutler@yahoo.com>

Coping Style It is important that children, early on, acquire both the ability to engage in selfreflection and to appraise the behavior of others in the objective world. As one begins to look both internally and externally, they must then integrate and compare the information obtained from each without becoming overwhelmed with either. The integration between internal sensitivity and external judgmentthe subjective and the objectiverequires that one maintain a complex but modulated response to both sources of information and to rely on a flexible system of values by which to appraise both the impact of others on self and of self on others. A perfect balance is unlikely and, not infrequently, an individual will develop a preference for, or sensitivity to, either internal experiences or external events. This preference results in one coming to rely on the preferred or least arousing source of information as a means by which they filter their view of the world. Kagan (1998) acknowledged the emergence of a lack of balance between internal and external focus as he uncovered the nature of governing temperaments, and suggested that this led to distinctive traits and temperaments. He observed that some infants were, by nature, behaviorally highly reactivei.e., very responsiveto internal events, resulting in a degree of emotionality that contributed to behavioral instability. In contrast, others were observed to be less reactive to these events and, instead, preferred attending to external happenings while ignoring internal experiences. He concluded that hyper-reactive children were easly overwhelmed and distressed by sudden or novel stimuli in their environments. Their responses were characterized by high arousal, distress, and fear. They viewed both the occurrence and anticipation of external events as intrusions and

threats which upset their internal experiences and produced avoidance and seclusion. In later life, these children were observed frequently to develop substantial amounts of anxiety and to become overwhelmed by their fears and avoidant in their behaviors. They often became socially withdrawn, self-critical, phobic and intolerant of emotional experience or environmental change. They turned to internal experience, fantasy, and obsessive reconstruction of events to achieve stability. Alternatively, Kagan asserted that a second temperament existed among infants who were characterized by low reactivity. Those with this temperament of low sensitivity were thought to be relatively more tolerant of novelty and change; they were observed to seek, rather than avoid, stimulation from their environment, to take action to engage and change their environments, and tended to be gregarious and outgoing in their relationships with others. When they did develop problems, the problems frequently expressed themselves as intrusive behaviors, insensitivity to others feelings and needs, lack of empathy, and with overt signs of anger and rage. Patterns like those observed by Kagan have been noted to occur within all age groups. Introversion-Extroversion (Eysenck, 1960), internalization-externalization (Welsh, 1952), and a bimodal array of similarly descriptive terms have characterized these distinctions in the experiences that people prefer and the way they adapt to change. Many of these terms have become accepted and constitute valid and useful ways of identifying a continuum of ways that people adjust to and respond to novelty and change. At one end of this continuum are individuals who protect themselves from stimulation by being self-critical, avoiding change, and withdrawing in the face of anticipated change or discord. These individuals are sensitized and over-reactive to change and are prone to be

overwhelmed by fear. They seek stability and safety in a focus on internal experiences rather than on the instability and uncertainty of external events. At the other end of the continuum are individuals who prefer to embrace novelty and change with activity and assertion (e.g., Beutler, Moos, & Lane, 2003; Beutler, Clarkin, & Bongar, 2000). They seek contact with others, enjoy change, and are gregarious in their interactions with their world. These accumulative individualization-processes of coping with novelty and change are similar across cultures. The same two basic temperaments endure. In virtually all cultures, individuals with a highly reactive temperament are described as internalizing, avoidant, restrained, or introverted. Those with a low reactive temperament, in contrast, have been described as externalizing, gregarious, and extroverted. Across cultures, there are preferences for one or another of these temperamental styles; western cultures tend to foster the development of external, assertive, and individualistic styles of adjusting to change, while those living in Eastern cultures prefer more avoidant, self-inspection, and internalizing styles, even sharing attachments across the communal group (Kawai, 1993; 1996). In their search for factors that mediate the effects of psychotherapy, researchers have been drawn to reflect on these temperament styles and their derivatives as being implicated in how people may be affected by different therapeutic interventions (Beutler & Clarkin, 1990; Beutler, Clarkin, & Bongar, 2000; Beutler & Harwood, 2000; Castonguay & Beutler, 2006). While clinicians and researchers have always harbored the hope that some patient factors may temper the effects of psychotherapy and provide a means of tailoring treatments to specific patients, identifying the particular patient attributes that signal the specific qualities of the psychological treatments with which 4

patient attributes may be matched (i.e., aptitude by treatment interactions; ATIs) has proven to be a complex and arduous task. The first efforts to fit structure of the psychotherapy offered to the patient who receives it were embodied by the technical eclectic approaches. These approaches derived a list or menu of techniques that were to be applied to patients with different symptoms or diagnoses (Beutler, 1983; Lazarus, 1981). There followed a movement to identify more general models of treatments which were packaged around a common theory and to adapt these models to different diagnostic conditions. This Empirically Supported Treatment (EST) approach assumed that a discrete but integrated list of interventions for patients of every relevant diagnostic group existed (Chambless & Hollon, 1998; Chambless & Ollendick, 2001). Technical eclecticism and ESTs have, relied on patient and treatment dimensions that were, respectively, either too narrow to generalize across therapists or treatments or too broad to reflect the real core of interactive processes that were actually the most closely associated with therapeutic changes. In both cases, they were vulnerable to application in too rigid a fashion and they tended to ignore important therapist, treatment, or patient differences. The typical result of studying treatment fit at either the technical eclectic or EST level has been the statistical acceptance of the dodo bird verdict--all treatments have essentially equivalent and indistinguishable outcomes (Beutler, 2009; Butler & Hughes, 2009). The alternative view which has emerged over the past two decades has been a model that describes interventions and patients in terms of principles or strategies that are broader than techniques but more informative than entire theories (e.g., Goldfried, 1980). The principles by which these approaches are organized seek to specify the conditions under which various strategies of intervention are optimal; however, the potential permutations of patient, therapist, and treatment dimensions that may constitute a fit

that enhances outcome are staggering in their magnitude, numbering well over one million (Beutler, 1991). At least some of the more effective pairings of patient and treatment strategy have been elucidated in psychotherapy research (Beutler, Clarkin, & Bongar, 2000; Castonguay & Beutler, 2007). Even with a manageable list of principles on which one might build a treatment, the concept of fit requires that one be able to identify and measure both the patient and the treatment dimensions that constitute the strategy being used. The analyses and measures used must consider the possibility that 1) these matching dimensions are neither completely independent of one another nor related to outcome in an equivalent way; 2) that treatment outcome may reflect both the main effects of patient and treatment factors as well as the fit of these factors together; and 3) that misfit in some patient-treatment dimensions may cancel out the positive effects of a good fit on other dimensions. In order to avoid problems associated with either the excessive narrowness of technical eclecticism or the over-inclusiveness of the dimensions underlying the identification of an EST, Goldfried (1980) urged the field to begin a process of reducing the list of potential variables by cataloguing the principles or strategies of treatment rather than either the techniques or theories. Some investigators (e.g., Beutler & Harwood, 2000; Prochaska, 1984; Prochaska & DiClemente, 1992) developed models and measures that work at the level of principles rather than at the levels either of technique or treatment brand. Beutler and colleagues, for example, initiated an effort to identify and subsequently test the most robust patient and treatment matching dimensions that predict treatment outcomes. Their approach was dubbed, Systematic Treatment Selection (STS; Beutler & Clarkin, 1990; Beutler, Clarkin, & Bongar, 2000) and comprised a series of specific patient, treatment, and matching dimensions that were thought to provide optimal

treatment (Beutler & Berren, 1995; Beutler & Clarkin, 1990; Beutler, Goodrich, et al., 1999; Beutler & Harwood, 2000; Beutler & Consoli, 1992). The result was a series of hypotheses about the cascading influences of these dimensions on complex, multi-level treatment processes. Instruments were developed and then, using these instruments in a final research step, the model was independently tested to determine if it yielded better predictions of outcome than the previous, simple systems (Beutler, Clarkin, & Bongar, 2000; Beutler, Moleiro, Malik, et al, 2003). The authors concluded that the complex model improved predictions substantially over simple, single-dimensional systems. One of the specific dimensions of fit that caught the interest of these investigators was that between patient coping style and the degree to which change occurred directly or indirectly. Patient coping style was reminiscent of the temperament described by Kagan (1998) and matched with the degree to which effective change was moderated by insight. More specifically, a relationship was found in the early validation studies and literature reviews between patient coping style and the differential use of interventions that either sought to change skills and behaviors directly or which focused on the indirect processes of achieving insight and internal awareness (Beutler & Clarkin, 1990). The fact that Beutler et als concept of coping style was conceptually similar to Kagans temperaments was serendipitous but provided some degree of construct validation. Likewise the two hypotheses relating to the fit of this patient dimension of the nature of treatment influences also paralleled the historical work of Kagan and other developmental psychologists. Specifically, it was posed that: 1) Among patients whose characteristic coping styles were identified as Internalizing (Beutler, Clarkin, & Bongar, 2000)a dimension that seemed to parallel the construct of hyper-reactive behavior by Kagan (1998)---outcomes were positively associated with the use of insight-oriented

interventions; and conversely, 2) among patients whose characteristic coping styles were identified as externalizing, based on the same logic, positive outcomes were associated with the frequency of using procedures that relied largely on skill development and direct behavior change. These two hypotheses were confirmed in several subsequent randomized clinical trials as reported in the earlier edition of this volume (Beutler, Clarkin, & Bongar, 2000; Beutler, Harwood, Alimohamed, & Malik, 2002; Beutler, Moleiro, Malik, Harwood, et al, 2003). The earlier chapter (Betuler, et al, 2002) provided a box score in which 15 of 19 studies confirmed the expected pattern or relationship between the goodness of patient-treatment fit and outcome. The remainder of the current chapter will delve deeper into the rationale for these two hypotheses and will subject them to a meta-analytic assessment to determine both the statistical significance of the findings supporting them and the strength of this matching dimension as a contributor to outcome. Definitions and Measures In order to determine the strategy of matching a patients coping style to the focus of psychotherapeutic interventions, both the patients coping style and the nature of interventions ranging from insight- to symptom-focused must be defined in operational terms. Beutler and colleagues have developed instruments to assess these dimensions of patients and treatments. They began with a conceptualization based on extant research in psychotherapy process and personality, and conceptualized both the focus of therapy and the corresponding patient coping style as bi-directional in nature (Beutler & Clarkin, 1990; Beutler, Clarkin, & Bongar, 2000; Beutler & Harwood, 2000). That is, coping style and therapy focus are assumed to exist along a continuum, with the nature of the effective interaction assumed to vary as a function of which end of each continuum best describes both the patient and the treatment. In their measures of Coping style, for

example, ratings of externalization and internalization were ordered along a continuum and were based on the rated preponderance of actions that occur under conditions of environmental change (Beutler, Moos, & Lane, 2003). Likewise, measures of treatment focus consider this dimension as varying on a continuum from insightoriented to symptom/skill-oriented. This latter designation is based both on a rating of the objectives of the intervention and the degree to which the efforts to induce change are aimed directly at the symptom or indirectly through a mediating/moderating variable (e.g., insight). Coping Style. Coping Style is a concept that has been described by different personality and psychopathology theorists via a collection of often unrelated sounding but conceptually similar terms, many of which are associated with widely different means of measurement (Endler, Parker, & Butcher, 1993; Lazarus & Folkman, 1984). At least two conceptual aspects of coping are controversial. For example, some theorists define coping style from behavioral observations and relate these observations to how one copes with environmental novelty and change under normal conditions (Lazarus & Folkman, 1984; McKay, McLellan, Alterman, Cacciola, Rutherford, & OBrien, 1998). Others emphasize the adequacy with which one copes when faced with stressful situations or unusual environmental changes and give a more pathological twist to the interpretation (Eysenck, 1960; Latack & Havlovic, 1992). Still others emphasize the role of trait-like aspects of coping (Endler, Parker, & Butcher, 1993), in contrast to those who concentrate on state or situational qualities of adjustment (e.g., Ouimette, Ahrens, Moos, & Finney, 1997). Beutler and Clarkin (1990) resolved the conflict among these varying theoretical points with a broad, statistical definition. They included within the term, Acoping

style@, a variety of dimensionssome trait-like and others more state-dependent, but all of which are intercorrelated with one another when measured. Within this broad definition, ones Coping Style was defined as the pattern of behavior that is predominantly employed when one faces a new or unusual situation. This definition combined both state- and trait- aspects of ones response and removed the requirement that coping styles only be observed during and following stressful situations. Thereby, the definition effectively eliminated the need to judge the level of stress experienced or the generalizability of the situation in which it occurred. From this broadened perspective, Acoping styles@ are recurrent patterns of behavior that characterize the individual when confronting new or problematic situations. They identify ones vulnerability to change and ones predominant tendency to respond to novelty. Thus, coping styles are not discrete behaviors but are a cluster of related behaviors that are distinguished because they are repetitive, durable across similarly perceived events, and observable when problems or unexpected events are being addressed. Descriptively, the specific behaviors that form the clusters include both repetitive situational responses such as impulsivity, discrete acting out behaviors, and general temperaments. Unlike more narrow definitions of coping style, definitions based upon correlated clusters of behaviors are not explanatory concepts. Given the diversity of measurements used to study coping styles, we will adopt this broad definition as descriptive of behavioral clusters that reflect an enduring propensity and are repetitive. This view of coping style as a descriptive, heritable, relatively stable, trait-like cluster of behaviors is generally consistent with other views in both the human and animal literatures (e.g., Eysenck, 1990; Kagan, 1998; Koolhaas et al., 1999; McGue & Bouchard, 1998). For example, most factor analytic studies of behavior have found a consistent, bi-directional dimension that is characterized by introverted/introspective

10

behaviors at one extreme and extroverted/extratensive behaviors at the other (Eysenck, 1957). Gray (1981) suggested that Eysencks (1957) original two-factor model of introversion-extroversion and neuroticism-stability, could be rotated by 45 degrees to form new axes that he labeled impulsivity and anxiety, respectively, bringing them more into line with the extended concepts of externalization and internalization used by Beutler and colleagues (Beutler & Clarkin, 1990; Beutler, Clarkin, & Bongar, 2000) and as originally defined by Kagans (1998) temperaments. Following the description of Eysenck (1957) and Kagan (1998), the quality that distinguishes internalizing traits and dispositions from other coping styles is that they are governed by the forces of inhibition and excitation. While, Eysenck and Kagan differ in the proposed level of behavioral reactivity characterizing these two groups, they agree that internalizers/introverts are more easily overwhelmed by change and tend to become shy, withdrawn, and self-inspective while externalizers/extroverts are more likely to act out, to seek stimulation and change, and to be confrontational and gregarious in expressing problems. Animal behaviorists have extended these qualities to a dimension of active to passive, or proactive vs. reactive behaviors (Koolhaas et al., 1999), and others have incorporated similar concepts into the big five personality factors (Costa & McCrae, 1985). For research purposes in psychotherapy, patient coping styles are best measured objectively either through individualized observations and ratings (e.g., Beutler, Clarkin, & Bongar, 2000) or through standardized, self-report, omnibus personality and psychopathology measures such as the Minnesota Multiphasic Personality Inventory (MMPI-2; Butcher, 1990; Butcher & Beutler, 2003), supplemented by reviewing the patients past and present reactions to problems. These individualized methods serve the broad definition used in this literature somewhat better than using more indirect measures

11

that often are identified as measures of coping (e.g., Lazarus & Folkman, 1984), but whose definition is much narrower than that used in treatment planning. The internalization ratio (IR) formula, extracted from the MMPI-2, that has been used frequently by our own research group captures the interactive nature of coping style and treatment focus (e.g., Beutler, Engle, et al., 1991; Beutler & Mitchell, 1981; Beutler, Mohr, Grawe, et al., 1991; Beutler, Moleiro, Malik, Harwood, Romanelli, GallagherThompson, & Thompson, 2003). In our modification of a formula originally proposed by Welsh (1952), eight MMPI-2 subscale scores are entered as a standard T scores: IR = Hy + Pd + Pa + Ma Hs + D + Pt + Si

An IR that favors the numerator suggests that a patient is disposed to use externalizing coping behaviors. These individuals blame others for their feelings (Pa); they display active, dependent behaviors (Hy), high levels of unfocused energy (Ma), are impulsive, and frequently have social adjustment problems (Pd). Sometimes, when using archival data, it is not practical or possible to utilize direct, selfreport measures of coping style. When such objective measurement is not available, the nature of the patients dominant coping style must be inferred through indirect means, using what information is available. Usually such indirect measures are based on group similarities, such as a common diagnosis, rather than on individual patterns of response or behavior. Diagnostic problems that are characterized by intense distress, ruminations, and social withdrawal are usually indirectly identified as related to internalizing patterns of coping. Thus, Axis I diagnoses within the spectrum of anxiety disorders as well as Axis II avoidant personality disorders can usually be assumed to be internalizing conditions while anti-social personality, chemical abuse, and paranoid personality disorder can be seen as more highly dominated by externalizing patterns. Disorders associated with ambivalence, such as borderline personality disorders, are usually most

12

reliably classified as externalizing; however, many diagnostic conditions are too diverse and variable to reliably map onto coping style descriptions using these indirect methods. They are not as reliable as direct measures of each patient and cannot reliably be used to classify individual coping styles among group members. For this reason, the use of indirect reliance on diagnosis or personality descriptions based on group characteristics as a means for determining a patients dominant coping style must be undertaken with some caution. Focus of Intervention The interventions selected and used by a therapist are also measured in two ways---through direct observations of each individual therapists behaviors or by indirect measures based on the common model of intervention used. The most sensitive method of measuring the focus of the interventions used is to observe and calibrate in-therapy actions and intentions of the therapist. Using individual, direct measuring methods, rating the use of various techniques such as interpretation, transference analysis, dream analysis, interpersonal analysis, and the like can be made to identify procedures which are most frequently associated with the effort to evoke insight and awareness of previously cathected, unconscious, and symbolized material (e.g., Beutler, Malik, et al, 2003). Direct observation such as the foregoing can yield numerical data on the frequency with which any type of intervention is used. One can count the use of symptom reports, techniques based on reinforcement paradigms, therapist instruction in the use of problem solving strategies, and efforts to enhance patient self-monitoring of symptoms, in order to identify the procedures that are predominant in use to evoke changes in symptoms and overt problems as well as to stimulate the resolution of inferred problems or causes. Where possible, the use of direct measures is very advantageous in either case. The measures are reliable, easily tested for inter-rater validity, and can be 13

used to rate a wide array of discrete techniques that share a common set of objectives; however, there are many instances when direct observations of therapy interactions are not possible. This is especially true when using archival data or when working from published reports to understand the nature of the treatment used. Under these circumstances, the focus of the intervention must be inferred from the brand name of the therapy involved. Usually, it is most reliable simply to categorize the model of intervention in terms of purity as a prototypic insight-oriented procedure (i.e., psychodynamic) or a symptom-focused procedure (i.e., behavioral). In this bifurcation, interpersonal, experiential, and psychodynamic therapies are usually classed as insightfocused procedures and cognitive, cognitive-behavioral, and behavioral models are identified as symptom-focused interventions. While not ideally sensitive, such a classification can be assessed for reliability and evidence of treatment fidelity can be used in research practice to provide some cross validation of ones classifications. By either means of assessing treatment focus, one categorizes the interventions used as either specifically focused on the symptom or invoked as a mediator of change. Symptom-focused change procedures attempt to identify specific problems and to intervene with them directly. That is, the problems that are most easily observed are considered the focus of the intervention. In contrast, interventions that address mediators of change address the overt problems indirectly, through focusing on an intervening or mediating variable (e.g., unconscious processes, emotional experience, unfinished business) that becomes the focus of change efforts. Thus, one can think about the mechanism of change as direct or indirect, in a way that is parallel to our previous description of direct and indirect measurement of change; however, this similarity should not lead one to confuse the descriptive nature of the measurement with the inferential theoretical assumptions that guide the focus of treatment.

14

Using a direct change intervention, the problems or symptoms themselves are assumed to be paramount and are addressed directly as the relevant point of focus. The treatment applies direct actions to identify, monitor, and alter these problems or symptoms. In contrast, an indirect change intervention uses symptoms or problems as an index by which to identify some other, more general construct, which is then adopted as the point of focus for the intervention procedure. The assumption is made in these indirect change interventions that the obvious symptom or problem will reduce or dissipate if the underlying and hypothesized cause can be corrected. It is probably an oversimplification to think of the distinction between direct change and indirect change interventions as being discrete and finite. More accurately, interventions are ordered along a continuum that ranges from the degree to which they address mediating variables to the degree to which they focus on the symptoms themselves (Beutler, Malik, et al, 2003). For example, in the purest form of symptomfocused interventions, behavior therapy directly addresses changes in symptoms and skills while eschewing the presence of underlying problems. These therapies take each symptom that is disruptive to the patients adjustment or happiness at face value, working sequentially to eradicate it. The symptom is identified, monitored, and subjected to interventions that are designed to alter it directly. At the other end of the treatment-focus dimension, psychodynamic procedures emphasize the use of interventions that can make an Indirect or mediated change on expressed problems and symptoms. These interventions take little note of the symptom, itself, seeing it as merely a symbolized expression of some unseen and more important conflict. That which is not directly seen, but which can be inferred from the theoretical model used, is then assumed as the point of focus for the change effort. Treatment models that emphasize unconscious processes are the best examples of these indirect interventions. In psychodynamic models, for example, the unseen mediators are 15

unconscious, symbolized, traumatic, and reflective of primitive experiences that can no longer be observed directly. Between these two pointsbehavioral to psychodynamicare many models of treatment that are balanced between the extremes of variations in direct and indirect interventions. Close to the direct intervention end, cognitively based interventions propose that symptoms such as anxiety and depression are reflective of faulty problem solving strategies or pre-set maladaptive cognitions that determine perceptions and activate automatic responses. It is these mediators, rather than the symptom, that becomes the focus of treatment. These mediators of importance to cognitive therapy are very proximal to the symptoms, their changes are assumed to be quite directly linked to changing these symptoms, and they are within easy grasp of ones awareness or consciousness. Thus, they occupy a position close to direct behavioral interventions along the treatment-focus dimension. Occupying positions closer to the psychodynamic/indirect end of the treatmentfocus spectrum, are interpersonal models of intervention which posit the presence of a connection between symptoms and events that are within ones ability to recollect, but are still more distal events than those used by behavior and cognitive therapists. These focusing events include such things as interpersonal loss, interpersonal conflict, and dysfunctional family patterns. Clinical Examples There are many examples of how patient coping styles are manifest in treatment. Even if the therapist does not have pre-treatment self-report measures, he/she may observe the patient respond to the life crises that occur during treatment by withdrawal and self-blame (internalizing) or by becoming angry, blaming, and avoidant (externalizing).

16

L.C. was a 42 year-old, married (2 years) man who was referred for psychotherapy by his physician, who he also described as his best friend. The patients presenting problems were many, including chemical abuse, depression, impaired work performance, and deficits in interpersonal functioning. The patient recalled being very depressed since the age of 12 and described a family history of abuse, alcohol dependence, and finally, abandonment. He was on his own at age 16 and what had begun three years before as recreational marijuana use rapidly developed into extensive cocaine, methamphetamine, and heroine abuse. He held several jobs between the ages of 16 and 40, losing most because of behaviors related to chemical dependency. At age 29 he got married and was divorced by age 35. At age 40 he began his own internet business in an effort to escape the rigid rules that had frequently led to his termination from other jobs. His progress had been uneven and slow; he maintained a marginal existence on the income that he could produce. Direct and indirect measures of L.C.s coping style might lead to different conclusions. Using an indirect measure, based on diagnosis, the presenting difficulties with drug abuse would probably result in his being classified as an Externalizer. Indeed, in many if not most cases, a diagnosis associated with acting out is a reasonably reliable index of this form of preferred coping; however, a direct measure (the MMPI-2, IR) of this patients coping pattern revealed that he had a mixture of both internalizing and externalizing coping patterns, with an overall balance favoring the use of internalizing strategies. The patients Internalization Ratio suggested that while he had a mixed coping style, it was predominantly internalizing. Hypochondriasis, Depression, anxiety (Psychasthenia), and Social Introversion scales averaged 7 points higher than the corresponding externalizing scales. His internalizing style of coping was further illustrated and observed in how he conceptualized both the cause and the consequences of his drug use. He expressed the belief that his drug abuse began because he was weak and 17

defectivean introtensive injunction. He indicated that his problem had continued because he was not strong enough to follow his consciencea self-critical injunction. While not a religious man, he expressed strong guilt for having enticed his wife into a marriage in which he was unable to take care of her. A direct evaluation of his history would support this view of L.C.s internalizing coping style. His clinical history of long standing depression and his depressive presentation at the time of intake would confirm the conclusions based on the MMPI-2. In contrast, R.W. was a 43 year old woman with a history of social avoidance and shyness. In her 20s and 30s the problems had become so bad that she had to quit her job as a secondary school teacher because she could not face her class. At that time she was diagnosed with Social Phobia and with Avoidant Personality Disorder. The MMPI-2 provided a direct confirmation for an internalizing coping style. The Internalization ratio showed dramatic elevations on Psychasthenia (scale 7) and Social Introversion (scale 10), with a secondary elevation on Depression (scale 2), relative to the externalizing scales. In both of these examples, the treatment of choice is to begin with a focus on developing a working relationship and then to move rapidly to begin a program of symptom change. When the symptoms of greatest danger and significance began to change, the strategy would shift to a focus on self-evaluation and insight with attention given to understanding the social patterns that constitute each patients theme of functioning. L.C. and R. W. would differ with respect to the symptoms that would be of primary focus during this early phase and in the theme that guided the insight-oriented work. For L.C. the initial symptom focus would probably be on behaviors that indicated risk for drug abuse and self-harm, with a secondary focus on social functioning. In contrast, the initial focus for R. W. would probably be on social withdrawal and depression with secondary attention given to any issues of self-harm that emerge. 18

L.C.s thematic focus would probably incorporate both internalizing (selfevaluation) and externalizing (other-appraisal) strategies. The therapist would seek to understand the interplay of these coping behaviors and to identify evoking perceptions that moved the patient in a cycle from internalizing (depression) to externalizing (drug abuse). These might include a theme that included the cycle of abandonment, depression, and self-medication. It is possible that the sequence of problems could exist in another pattern, but given the family history of abandonment, the early development of depression, and the later drug abuse history, this same causal sequence is likely to be salient in the broad theme that describes his behavior. R.W.s theme is likely to be quite different than L.C.s. R. W. may represent the hyper-reactive temperament described by Kagan (1998) and thus, be a very early developmental phenomenon; therefore, hypervigilance, chronic fear, and a dread of appraisal from others would probably dominate the theme. Compared to L.C., the expressed coping style is likely to be much more consistently internalizing, with a lot of attention given to self-evaluation and criticism. This means that one would probably move quite quickly to a theme- or insight-focused intervention. There are equivalent examples of the differential treatment of individuals who prefer externalizing coping styles. Patterns of consistent acting out, conflict with authorities, and phobias, are examples of individuals who cope in externalized ways. The identification of a preference for these externalized patterns may be inferred from diagnoses like Antisocial Personality Disorder, Borderline Personality Disorder, Chemical Abuse or Dependency, and varieties of Impulse and Phobic Disorders. While these categorical, indirect measures of coping style are useful, they lack the sensitivity that a continuous measure might provide. Most people manifest both externalizing and internalizing styles of coping, depending on how they appraise different situations. The

19

degree to which they prefer and rely on externalizing behaviors over internalizing ones can only be assessed by individual measures that tap both types of behavior. R.G., for example, was a 21 year old woman who was referred for psychotherapy from her psychiatrist because of a longstanding pattern of explosive outbursts. In recent years, she had begun using alcohol and had been picked up for driving under the influence on two occasions. She had experienced problems in school because of her failure to control her temper and had been a chronic problem to her parents because of similar behavior. She had been in and out of treatment since age eight, but with little help. Except for her first experience with behavior therapy, her treatment had always focused on allowing expression of her feelings, trying to uncover the source of her rage, and developing some self-awareness and insight. Direct measure of coping style, using the Systematic Treatment Selection-Clinician Rating Form and the MMPI-2 confirmed the dominance of impulsivity and confrontational coping behaviors over rational selfcontrol. She had poor insight, high levels of poorly directed energy, and a strong sense of persecution. Accordingly, treatment focused, not on self-expression and unloading, but on control and tolerance for the discomfort associated with anger and environmental stimulation. Intervention began by identifying specific situation in which problematic behaviors and symptoms occurred. She was taught to self-monitor her arousal and to identify risk-provoking situations. She then was engaged in a process of learning stress tolerance, where negative emotions were selectively evoked by visual imagery and role playing. Instruction in pro-social behavior accompanied these activities and behavioral rehearsal was used to engage her in a process of social learning and to develop useful skills in impulse control, self-appraisal, and tolerance for novelty and change. Meta-Analytic Review Coping style 20

In undertaking the meta-analysis on which this chapter is based, we began with the review of research studies presented in the earlier edition of this volume by Beutler, et al (2002). Our focus was on identifying studies in which the interaction of coping style and treatment focus could be assessed and effect sizes could be calculated. These are called, moderating studies throughout this chapter. That is, they addressed the moderating role that patient coping style exerted on the effectiveness of a particular focus (direct or indirect) of treatment; however, like the chapter in the earlier volume, we also wanted to assess the independent effects of patient coping style, if any, on outcome. That is we also wanted to know the main effects of patient coping style. This latter, or main effect analysis, addresses a prognostic question while moderating studies address a treatment planning question: What treatment is best for what patient?. We began our meta-analysis by collecting studies that had addressed patient coping style as a mediator between treatment and outcome. The initial pool of possible studies was comprised of those reviewed in the earlier edition of this volume. Nineteen studies had been identified, 15 of which had been supportive of the hypothesized mediating effects of coping style in treatment. We then excluded studies that had major methodological weaknesses and those whose results did not allow the calculation of an effect size. Methodological weaknesses included the failure to use blind or masked outcomes, indefinite forms of treatment in which the focus could not be defined with relative certainty, and inaccurate interpretations or calculations. We then added studies that emerged from a review of more current literature on treatment outcomes if they fell within the methodological parameters of our review. Applying both indirect measures of coping style and direct ones, we initially identified over 25 studies that had addressed the roles of coping style either as a main effect or as a mediator of treatment outcome. Unfortunately, a number of these studies presented incomplete statistical results or glaring methodological weaknesses, precluding the construction of an effect size (ES) 21

measure. From this pool, we extracted 13 investigations that permitted an analysis of ESs. These studies and their results are presented in Table 1. Table 1 about here Effect sizes (ESs) associated with the fit of treatment focus and patient coping style were calculated as suggested by several sources. We used the calculation procedure and formula that best fit the characteristics of the data presented in an individual study. There are (usually) minor variations among the different ways that statistical experts have calculated ESs. Cohens d is the accepted statistic in all cases but ESs are often presented as correlation coefficients or even regression coefficients. In our case we transferred all estimates of ES to a d coefficient. Several sources were consulted in making this transformation. When there was a difference between formulae, and no single one was consensually accepted as the one of choice, which was often the case, we used the formula that was most consistent with other sources and that provided what appeared to be the most unbiased estimate of ES. For example, if one formula could be used without converting any data, this formula was used because it appeared that parsimony would result in less error in calculation. We frequently calculated and recalculated formulas 2 or 3 times to ensure accuracy. We did not change or alternate among formulas for data that had consistent characteristics in order to ensure consistency. For example, if means, sample sizes, and SDs were available, we always employed the same formula across studies; however, when data were incomplete or reports did not contain some important information, we relied on accepted alternative procedures. In reality, the calculations of d were usually relatively straightforward. The most challenging aspect of the calculation process was identifying the correct formula for calculating ESs or for converting data when a new form of data was encountered. As a

22

general guide, we employed Borenstein, Hedges, Higgins, and Rothstein (2009) as a source. For calculating simple means and standard deviations when these were not reported, we relied on Lipsey and Wilson (2001) and Hunter and Schmidt (2004) as supplementary resources. For calculating d from t-tests, ANOVA, ANCOVA, MANOVA, we used Cortina and Nouri (2000), Hunter and Schmidt (2004), Lipsey and Wilson (2001), Newton and Rudestam (1999), and Thalheimer and Cook (2002) as resources. For transforming the results of regression analyses into d, we used Hunter and Schmidt (2004), Newton and Redestam (1999), and Thalheimer and Cook (2002). Finally, for transforming correlational data, we found Borenstein, et al (2009) and Hunter and Schmidt (2004) to be very useful. In the final phase, we calculated 95% confidence intervals for all ES estimates. These figures are reported in the last column of Table 1. We also calculated an overall mean ES estimate across studies, weighting the individual study ds with the number of subjects used. Our source for the calculation of 95% confidence intervals was Smithson (2003). Mediators and Moderators Main Effects of Coping Style In the first edition of this volume, Beutler et al (2002) reviewed and tabulated in a box score fashion, studies that bore on the relationship between coping style and treatment outcome, especially as moderated by the focus of treatment. Only one study was identified from which the effects of coping style on outcome could be extracted, independently of the type of treatment employed. Beutler, Clarkin, and Bongar (2000) explored the prognostic value of coping style, finding that externalizing patients did more poorly than internalizing ones across a range of different treatment and problem types. Interestingly, this latter study was eliminated from the current data set because we could 23

not find an adequate way of translating the effects of structural equation modeling to effect sizes that were comparable to d. One advantage of a meta-analytic procedure over the box score approach used in the earlier review is that it allows a relatively precise estimate of the strength of effect by including both studies that use indirect and those that use direct measures of a continuous variable like coping style. Moreover, using studies with samples that are comprised mainly of patients with one type of coping style (external or internal), a separate estimate can be derived for each of these styles and the differences can be compared; however, one downside involves the necessary exclusion of some studies in a meta-analytic review. It will be useful to explore some of the methodological and data-based weaknesses that limited the inclusion of some previously used studies. The main reason that studies were dropped from the meta-analysis was that they did not report data from which effect sizes could be computed. In many cases this was simply a failure on the part of the investigator to conduct necessary analyses or report necessary statistics. In other cases, the problem was that their statistical procedures provided data that were appropriate but we were unable to reliably calculate effect sizes. Such is the case for the study by Beutler, Clarkin, and Bongar (2000). This latter study included a large number of patients (N= 284), representing both internal and external coping styles. Coping style was assessed directly using the Internalization ratio (IR) described earlier. This ratio reflected a ratio among eight scores from the Minnesota Multiphasic Personality Inventory (MMPI), a self-report omnibus personality measure. Unfortunately, the foregoing study utilized a continuous, directly assessed measure of coping style and a Structural Equation Modeling (SEM) analyses from which there is no reliable method that we could identify to extract an effect size estimate for the main effect.

24

It is also notable that, in some instances, a studys strength impeded the calculation of some effect sizes. For example, when a study concentrated on measuring coping style as a continuous measure, a separate effect size cannot be extracted for the two ends of the continuum wherein the prototypic coping styles exist. This was the case for several studies in the current set (e.g., Barber & Muenez, 1996; Beutler, Engle, et al, 1991; Calvert, Beutler, et al, 1998). Yet, the use of continuous measures is a methodological strength because of the increased sensitivity over categorical measures. In these instances, an estimate of the fit between patient and therapy is typically easy to derive; however, this ease of derivation comes at the expense of generating a separate effect size for each coping style, itself. Among the 13 studies (Table 1) on which comparable effect sizes could be extracted, five provided information from which an effect size estimate could be extracted to indicate the predictive value of coping style. Only one of these was on an internalizing group and three were on externalizing groups. Thus, these data were insufficient to calculate a difference between these coping styles in predicting treatment outcome. We are unable to conclude whether there was a substantial effect in favor of one or the other way of coping. Estimating the effect of the focus of the therapists intervention was an easier matter since all the treatments could be coded in the same direction relative to their insight or symptomatic focus. The results of these analyses indicated an effect size of 1.01 favoring symptom-focused over insight-focused interventions. This is a very strong effect and clearly, at least in the short term treatments comprising the majority of this data set, a direct symptomatic focus is superior to an indirect, insight focus of treatment. Moderated Effects of Patient Coping Style The studies in our meta-analysis all allowed an analysis of the proposition that coping style could also serve as a moderator of the effect of different therapeutic foci. 25

The statistic of interest in these analyses was the difference between a mean ES estimate among patients who were well-matched to the treatment they received and a similar estimate for those patients who were poorly matched to treatment. A composite mean was computed for each study, based on all dependent variables. The size of this mean, then, indicated the role of treatment matching, over and above the value of symptomatic focus itself. The final ES estimate for fit was comprised of a difference between the mean ES across all outcome variables used. A good match was taken as being composed of either: 1) externalizing patients and symptom focused therapy or 2) internalizing patients and insight-focused therapy. No distinction was possible between these two kinds of good matches or between the corresponding estimates of poor matches in most studies. Among studies that utilized an indirect method of assessing the symptom-insight dimension of psychotherapy, the conventional method of assessing ES (d) from standard scores computed on each treatment group was used; however, when the more sensitive direct method of measuring therapist focus was used (e.g., rating scales of individual therapy session processes and therapist actions), a continuous measure of the fit resulted. For many of these direct measures of process, the estimate of the relationship between the goodness of fit between coping style X treatment focus and outcome was expressed as a multiple regression or beta coefficient which then was converted into a d equivalent score. This transformed score served as the final expression of the magnitude of change associated with level of fit, expressed as a d. In the analysis reported in Table 1, ten of the 13 studies used a direct measure of patient coping style. Only three used a direct measure of therapy focus. All thirteen of the studies yielded a significant effect for good over poor matches, based on the fit of patient coping style and treatment focus. Each effect size represented a mean difference between good matches and poor matches. 26

A total of 5 ESs were computed on the total sample of 13 studies with individual studies having from 1 to 14 effect sizes comparing the level of fit to outcome. The mean of the effect sizes indicating the effects of level of fit was .55 (d), indicating a medium to large effect size associated with fitting patient coping style to treatment focus. In all cases the effect sizes favored good matches over poor ones. This finding indicates that the average well-matched treatment produced an 8% greater effect than the poorly matched treatmentthe average patient with a good fit was better off than 58% of those with a poor match. This finding supports the conclusions of the earlier review and adds important information about the strength of the effect. Moreover, given the correspondence among the two reviews, one an inclusive review and this, a truncated review of only those studies that had reported relevant statistics, the conclusion gains some veracity. Table 1 summarizes the research that has been included in this meta-analysis of patient coping style as a moderator of treatment outcome. The table reveals that the 13 studies analyzed on this dimension demonstrated differential effects of symptom and skill building procedures or treatments versus insight-oriented procedures or treatments, as a function of patient coping style. All of the studies support the conclusion that interpersonal and insight oriented therapies are most effective among internalizing patients and symptom and skill building therapies are most effective among externalizing patients. It is useful to look at the studies that were included in the earlier review but were not included in this meta-analysis. All but two of these excluded studies obtained results that were similar to the ones obtained in the current meta-analysis. For example, we have already discussed the exclusion of the structural equation modeling method used by Beutler, Clarkin, and Bongar (2000), which was consistent with the current findings but

27

excluded in the current analysis. Like this latter study, the others that were excluded utilized methods that did not permit an accurate extraction of effect size data. For example, in one of the first such studies published, Beutler and Mitchell (1981) treated 40 mixed-diagnosis (depressed and anxious) outpatients with either analytic or experiential treatment techniques. Patient coping style (internalizer or externalizer) was assessed using the MMPI. Patients whose MMPI profiles suggested a preponderance of impulsive (Pd), projective (Pa), and excitable (Ma) symptoms were classified as AExternalizers@. Those whose profiles suggested the presence of selfcriticism (D), aggitation (Pt), and social withdrawal (Si) were classified as AInternalizers@. The results revealed systematic patient aptitude (coping style) X treatment interaction effects that were independent of diagnoses. Externalizing patients were found to achieve greater benefit from experiential treatment than from analyticbased therapy; however, among internalizing patients, insight-oriented (analytic) treatment achieved its greatest effects and, correspondingly, the behavioral therapies had the least beneficial impact. Unfortunately, these results were based on a box score tabulation of studies that were and were indicative of a relationship between therapypatient fit and outcome. The lack of more precise statistics rendered this study inappropriate for inclusion in the meta-analysis. In 1984, Beutler, Frank et al., compared the effectiveness of three types of group psychotherapy (i.e., cognitive-behavioral, experiential-gestalt, interactive-supportive therapy) to a treatment-as-usual control condition. The patients comprised a group of acutely disturbed mixed-diagnosis psychiatric inpatients who presented for short-term care. Patients who received insight-oriented or abreactive treatment experienced a worsening of symptoms, while those who received interactive-supportive treatment benefitted. Again, the results were appropriate for a tabular analysis but were not conducive to meta-analysis. 28

One study reported in the earlier edition of this volume (Beutler, Moleiro, Malik, & Harwood, 2000) was not used in this analysis because it used a composite measure of treatment X patient fit and the specific effects of patient coping style X treatment focus could not be extracted. This latter study was a randomized controlled study of 40 comorbid depressed and stimulant abusing patients which found that the overall fit of treatment to this collection of patient variables confirmed that good matches between patient variables and type of treatment was a positive predictor of outcome. A more intensive analysis of this study by Beutler, Moleiro, et al, (2003) revealed that a good fit with a collection of matching dimensions accounted for from 80% to 93% of the variance in depression scores at a six month follow-up period and from 57% to 79% of the sixmonth variance in drug use. The two studies that were included in the earlier review but excluded in this one included the largest RCT in the exploration of matching dimension to date (Project Match Research Group, 1997). In Project MATCH, 952 outpatients and 774 inpatients diagnosed as alcohol dependent were assigned to one of three 12-week, manual-guided treatments (cognitive-behavioral coping skills therapy, motivational enhancement therapy, or 12 facilitation therapy). Coping style could not be extracted from this study in a numerical way and the matching algorithms that were used were not conducive to assigning direct numerical values. Patient Contributions Coping style is a relatively stable and enduring patient quality. Thus, it fits our definition of a personality trait. That is not to say that coping style is always uniform. It can both be changed and is responsive to ones appraisal of a situation. Thus, it may be described by its stability as well as by its position along a continuum. In the foregoing analysis, we could not extract a reliable estimate of the degree to which a given coping style helped or hindered therapeutic gain. In the earlier review (Beutler, et al, 2002) it 29

was suggested that those patients who tend toward the internalizing end of the coping style continuum would be better prognostic risks in psychotherapy than those who were more externalizing. We cannot refute this point and the data available point in this direction, but in the absence of comparable data on externalizing patients, we cannot reach a reliable conclusion for this earlier one. Notably, this also means that, neither did we find evidence for Kagans (1998) assumption that the fearful, hyper-sensitive internalizer would be more of a prognostic risk. Judging from the current findings, the coping style preferences for individuals are distributed broadly within the population at large and all along the coping style continuum. Individuals with preferences for both internal and external styles of coping are capable of benefitting from psychotherapy, assuming that the form and nature of that treatment is appropriate to their own preferred coping style. Interestingly, we did find a very strong effect for the use of symptom focused procedures over directive ones, across patient types. This finding generally suggests that, at least in early sessions or in short term treatments, such a direct, focused intervention may be preferred over a more indirect one. More research needs to be done on this finding, extracting the focus of the intervention from other aspects of the model studied.

Limitations of the Research Reviewed There are limitations to any research analysis, including meta-analyses. Two major threats need to be considered in meta-analytic studies such as this. First, many studies are excluded because they do not include data that allows effect sizes to be constructed in a way that is comparable across studies. That was certainly a problem here where 11 studies found in our review of the literature were not included because of missing statistical information. Fortunately, a tabulation of these studies confirmed that

30

the direction of their findings were strikingly consistent with the direction of the effect sizes that we were able to compute. While it is still possible that the magnitude of effects were smaller or larger than in those studies that we did include, there is no reason to reject the meta-analysis on this basis. The second threat to the validity of the findings is that very important individual findings may be masked by the grouping of all studies together. It may be, for example, that hidden within these studies, is one that has a unique methodological feature that reveals a relationship that is masked because the other studies do not have such a feature. Unlikely as it may be, this is sometimes the argument that is made by scholars who ignore the results of meta-analytic analyses which tend to find no significant differences between different forms of psychotherapy. Thus, in spite of this latter persistent and consistent finding of treatment equivalence that is obtained almost universally when studies are combined for meta-analytic comparisons, many very good scholars and authors continue to conclude that some treatments are consistently better than others (e.g., Baker, McFall, & Shoham, 2009; Chambless & Ollendick, 2001; Ollendick & King, 2007). These conclusions are based on individual studies that depart from the general finding of equivalence. While such conclusions may be correct, the justification for ignoring the meta-anlaytic findings of treatment equivalence has not been persuasive.

Therapeutic Practices Patient coping style is a promising moderator of the effects of treatment focus on outcomes. Coping style positively impacts outcome when appropriately matched with the focus of treatmentinsight-focused for internalizing and symptom-focused for externalizing patients. Clinicians should take advantage of the information provided by 31

studies that explore patient-level and treatment-level variables and adjust their treatments accordingly. Patients who have little support from other people and who manifest impairment in two or more areas of functioning (family, social, intimate, work, etc.) may benefit from treatment that includes a medication component and increases the intensity of interventions by lengthening treatment. Likewise, patients who manifest externalizing tendencies might be provided with treatments that are focused on skill building and on symptom change. In contrast, those who manifest patterns of self-criticism and emotional avoidance are likely to benefit from an interpersonally focused and insightoriented treatment. The second of the two major findings indicated that symptom-focused interventions may be superior to insight-oriented ones. This finding must be restricted to the relatively short period of time in which most of the current studies were cast. The general conclusion would suggest that there is value in beginning treatment with direct, symptom focused procedures and as the coping style of the patient becomes clear, switching to a more indirect, insight approach if that coping style is weighted toward internalizing patterns. The effective clinician will be one who is able to recognize a patients emerging coping style and modify interventions and treatment plans to fit the patient. While the evidence is reasonably clear that all patients do not respond equivalently to a given intervention and that patient factors moderate treatment response, exact cut-off points on measures and exact procedures for implementing treatment variations are not certain. At least the level of care, in the form of intensity and the use of adjunctive medications, and the differential use of behavioral versus interpersonal/insight procedures may facilitate treatment outcomes when appropriately applied to patients who differ in coping styles.

32

Assessment of these patient attributes need not be time consuming or tedious; cues for the identification of a variety of patient attributes are included in Beutler and Harwood (2000) to enable the clinician to make any necessary in-session treatment matching adjustments. These procedures combine self-report and clinician ratings to define characteristic ways that the patient responds to change and novelty. Investigators, as well as clinicians, who are interested in psychotherapy outcome research are encouraged to continue with, or begin, the exploration of therapy process relevant ATIs in the hopes of developing treatments that outperform extant conventional treatment packages. In support of this effort, Beutler, Clarkin, and Bongar (2000) have extracted two principles from extant research literature that they believe may be useful in helping practicing clinicians to first, recognize relevant patient characteristics and second, select and apply an effective treatment.

1.

Therapeutic change is most likely if the initial focus of change efforts is to alter disruptive symptoms.

2.

Therapeutic change is greatest when the internal or external focus of the selected interventions parallel the external or internal methods of avoidance that are characteristically used by the patient to cope with stressors. The evidence for the first of these principles is uncertain in the current review, but

the general finding is at least partially supported. The evidence for the second principle is strong, with most available studies providing support for the principle and with an average effect size (d) of .55 across studies. A promising number of studies have accumulated suggesting that task- and symptom-focused interventions are more effective than insight-oriented ones among patients who are impulsive, extroverted, and noninsightful. The converse also appears to be true. Insight-oriented and interpersonally 33

focused interventions are most effective among patients who are introspective, introverted, and self-critical. These patients seem to do less well with behaviorally and skill focused interventions.

34

References Abrams, R. C. (1996). Personality disorders in the elderly. International Journal of Geriatric Psychiatry, 11(9), 759-763. Ackerman, D. L., Greenland, S., & Bystritsky, A. (1994). Predictors of treatment response in obsessive-compulsive disorder: Multivariate analyses from a multicenter triall of comipramine. Journal of Clinical Psychopharmacology, 14, 247-253. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders. (4th ed.). Washington, DC: Author. Andrew, B., Hawton, K., Fagg, J., & Westbrook, D. (1993). Do psychosocial factors influence outcome in severely depressed female psychiatric in-patients? British Journal of Psychiatry, 163, 747-754. Badger, T. A., & Collins-Joyce, P. (2000). Depression, psychosocial resources, and functional ability in older adults. Clinical Nursing Research, 9(3), 238-255. Barber, J. P., & Meunz, L. R. (1996). The role of avoidance and obsessiveness in matching patients to cognitive and interpersonal psychotherapy: Empirical findings for the treatment for depression collaborative research program. Journal of Consulting and Clinical Psychology, 64, 951-958. Barkham, M., Rees, A., Stiles, W. B., Shapiro, D. A., Hardy, G. E., & Reynolds, S. (1996). Dose-effect relations in time-limited psychotherapy for depression. Journal of Consulting and Clinical Psychology, 64(5), 927-935. Barry, M. M., & Zissi, A. (1997). Quality of life as an outcome measure in evaluating mental health services: A review of the empirical evidence. Soc. Psychiatry Psychiatr. Epidemiol, 32, 37-47. Bartels, S. J., Mueser, K. T., & Miles, K. M. (1997). A comparative study of elderly patients with schizophrenia and bipolar disorder in nursing homes and the community. Schizophrenia Research, 27(2-3), 181-190.

Basoglu, M., Marks, I. M., & Swinson, R. P. (1994). Pre-treatment predictors of treatment outcome in panic disorder and agoraphobia treated with alprazolam and exposure. Journal of Affective Disorders, 30, 123-132. Beutler, L. E. (1983). Eclectic Psychotherapy: A systematic approach. New York: Pergamon Press. Beutler, L. E. (1991). Have all won and must all have prizes? Revisiting Luborsky et al.'s verdict. Journal of Consulting and Clinical Psychology, 59, 226-232. Beutler, L. E., & Berren, M. R. (1995). Integrative assessment of adult personality. New York: Guildford Press. Beutler, L. E., & Clarkin, J. F. (1990). Systematic Treatment Selection: Toward targeted therapeutic interventions. New York: Brunner/Mazel. Beutler, L. E., Clarkin, J. F., & Bongar, B. (2000). Guidelines for the systematic treatment of the depressed patient. New York: Oxford University Press. Beutler, L. E., & Consoli, A. J. (1992). Systematic eclectic psychotherapy. In J. C. Norcross & M. R. Goldfried (Eds.), Handbook of psychotherapy integration (pp. 264299). New York: Basic Books. Beutler, L. E., Engle, D., Mohr, D., Daldrup, R. J., Bergan, J., Meredith, K., & Merry, W. (1991). Predictors of differential response to cognitive, experiential, and selfdirected psychotherapeutic techniques. Journal of Consulting and Clinical Psychology, 59(333-340). Beutler, L. E., Frank, M., Scheiber, S. C., Calvert, S., & Gaines, J. (1984). Comparative effects of group psychotherapies in a short-term inpatient setting: An experience with deterioration effects. Psychiatry, 47, 66-76. Beutler, L. E., Goodrich, G., Fisher, D., & Williams, O. B. (1999). Use of psychological tests/instruments for treatment planning. In M. E. Maruish (Ed.), The use of psychological tests for treatment planning and outcome assessment (2nd ed., pp. 81113). Hillsdale, NJ: Lawrence Erlbaum.

Beutler, L. E., & Harwood, T. M. (2000). Prescriptive Psychotherapy: A practical guide to Systematic Treatment Selection. New York: Oxford University Press. Beutler, L. E., Kim, E. J., Davison, E., Karno, M., & Fisher, D. (1996). Research contributions to improving managed health care outcomes. Psychotherapy, 33, 197-206. Beutler, L. E., Machado, P. P. P., Engle, D., & Mohr, D. (1993). Differential patient X treatment maintenance aiming cognitive, experiential, and self-directed psychotherapies. Journal of Psychotherapy Integration, 3, 15-31. Beutler, L. E., & Mitchell, R. (1981). Differential psychotherapy outcome among depressed and impulsive patients as a function of analytic and experiential treatment procedures. Psychiatry, 44, 297-306. Beutler, L. E., Mohr, D. C., Grawe, K., Engle, D., & MacDonaled, R. (1991). Looking for differential treatment effects: Cross-cultural predictors of differential psychotherapy efficacy. Journal of Psychotherapy Integration, 1(121-141). Beutler, L. E., Moleiro, C., Malik, M., & Harwood, T. M. (2000). The UC Santa Barbara study of fitting therapy to patients: First results. Paper presented at the International Society for Psychotherapy Research, Chicago, June 2000. Beutler, L. E., Moleiro, C., Malik, M., Harwood, T.M., Romanelli, R., GallagherThompson, D., & Thompson, L. (2003). A comparison of the Dodo, EST, and ATI indicators Among Co-Morbid Stimulant Dependent, Depressed Patients. Clinical Psychology & Psychotherapy, 10, 69-85. Bilder, R. M., Goldman, R. S., Robinson, D., Reiter, G., Bell, L., Bates, J. A., Pappadopulos, E., Willson, D. F., Alvir, J. M. J., Woerner, M. G., Geisler, S., Kane, J. M., & Lieberman, J. A. (2000). Neuropsychology of first-episode schizophrenia: Initial characterization and clinical correlates. American Journal of Psychiatry, 157(4), 549-559. Billings, A. G., & Moos, R. H. (1985). Life stressors and social resources affect posttreatment outcomes among depressed patients. Journal of Abnormal Psychology, 94,

140-153. Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2009). Introduction to meta-analysis. West Sussex, UK: Wiley. Brown, T. A., & Barlow, D. H. (1995). Long-term outcome in cognitivebehavioral treatment of panic disorder: Clinical predictors and alternative strategies for assessment. Journal of Consulting and Clinical Psychology, 63, 754-765. Burvill, P. W., Hall, W. D., Stampfer, H. G., & Emmerson, J. P. (1991). The prognosis of depression in old age. British Journal of Psychiatry, 158, 64-71. Bussing, R., Zima, B. T., & Perwien, A. R. (2000). Self-esteem in special education children with ADHD: Relationship to disorder characteristics and medication use. Journal of the American Academy of Child & Adolescent Psychiatry, 39(10), 12601269. Butcher, J. N. (1990). The MMPI-2 in psychological treatment. New York: Oxford University Press. Calvert, S. J., Beutler, L. E., & Crago, M. (1988). Psychotherapy outcomes as a function of therapist-patient matching on selected variables. Journal of Social and Clinical Psychology, 6, 104-117. Cooney, N. L., Kadden, R. M., Litt, M. D., & Getter, H. (1991). Matching alcoholics to coping skills or interactional therapies: Two-year follow-up results. Journal of Consulting and Clinical Psychology, 59, 598-601. Cortina, J.M., & Nouri, H. (2000). Effect size for ANOVA designs. Thousand Oaks, CA: Sage Publications. Costa, P. T., Jr., & McCrae, R. R. (1985). The NEO Personality Inventory manual. Odessa, FL: Psychological Assessment Resources. Coyne, J. C., & Downey, G. (1991). Social factors and psychopathology: Stress, social support, and coping process. Annual Review of Psychology, 42, 401-425.

Cuijpers, P., & Van Lammeren, P. (1999). Depressive symptoms in chronically ill elderly people in residential homes. Aging & Mental Health, 3(3), 221-226. Dadds, M. R., & McHugh, T. A. (1992). Social support and treatment outcome in behavioral family therapy for child conduct problems. Journal of Consulting and Clinical Psychology, 60, 252-259. Eldredge, K. L., Locke, K. D., & Horowitz, L. M. (1998). Patterns in interpersonal problems associated with binge eating disorder. International Journal of Eating Disorders, 23(4), 383-389. Elkin, I., Gibbons, R. D., Shea, M. T., Sotsky, S. M., Watkins, J. T., Pilkonis, P. A., & Hedeker, D. (1995). Initial severity and differential treatment outcome in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical Psychology, 63(5), 841-847. Elkin, I., Shea, T., Watkins, J. T., Imber, S. D., Sotsky, S. M., Collins, J. F., Glass, D. R., Pilkonis, P. A., Leber, W. R., Docherty, J. P., Fiester, S. J., & Parloff, M. B. (1989). National Institute of Mental Health treatment of depression collaborative research program. Archives of General Psychiatry, 46, 971-982. Ellicott, A., Hammen, C., Gitlin, M., Brown, G., & Jamison, K. (1990). Life events and the course of bipolar disorder. American Journal of Psychiatry, 147(11941198). Eysenck, H. J. (1957). The dynamics of anxiety and hysteria. New York: Praeger. Eysenck, H. J. (1967). The biological basis of personality. Springfield, Illinois: Thomas. Eysenck, H. J. (1990). Genetic and environmental contributions to individual differences: The three major dimensions of personality. Journal of Personality, 58, 245261. Fahy, T. A., & Russell, G. F. M. (1993). Outcome and prognostic variables in bulimia-nervosa. International Journal of Eating Disorders, 14, 135-145.

Fisher, D., Beutler, L. E., & Williams, O. B. (1999). STS Clinician Rating Form: Patient assessment and treatment planning. Journal of Clinical Psychology, 55, 825-842. Folkman, S., Lazarus, R. S., & Dunkel-Schetter, C. (1986). Dynamics of a stressful encounter: Cognitive appraisal, coping, and encounter outcomes. Journal of Personality and Social Psychology, 50, 992-1003. Ford, J. D., Fisher, P., & Larson, L. (1997). Object relations as a predictor of treatment outcome with chronic postraumatic stress disorder. Journal of Consulting and Clinical Psychology, 65, 547-559. Fountoulakis, K. N., Tsolaki, M., & Kazis, A. (2000). Target symptoms for fluvoxamine in old age depression. International Journal of Psychiatry in Clinical Practice, 4(2), 127-134. Fremouw, W. J., & Zitter, R. E. (1978). A comparison of skills training and cognitive restructuring-relaxation for the treatment of speech anxiety. Behavior Therapy, 9, 248-259. Friedman, R. A., Markowitz, J. C., Parides, M., Gniwesch, L., & Kocsis, J. H. (1999). Six months of desipramine for dysthymia: Can dysthymic patients achieve normal social functioning? Journal of Affective Disorders, 54(3), 283-286. Garvey, M. J., Hollon, S. D., & DeRubies, R. J. (1994). Do depressed patients with higher pretreatment stress levels respond better to cognitive therapy than imipramine? Journal of Affective Disorders, 32(1), 45-50. Garvey, M. J., & Noyes, R. (1996). Association of levels of N-acetyl-betaglucosaminidase with severity of psychiatric symptoms in panic disorder. Psychiatry Research, 60(2-3), 185-190. Garvey, M. J., Noyes, R., Cook, B., & Blum, N. (1996). Preliminary confirmation of the proposed link between reward-dependence traits and norepinephrine. Psychiatry Research, 65(1), 61-64.

Garvey, M. J., Noyes, R., Woodman, C., & Laukes, C. (1995). Relationship of generalized anxiety symptoms to urinary 5-hydroxyindoleacetic acid and

vanillylmandelic acid. Psychiatric Research, 57(1). Garvey, M. J., & Tauson, V. B. (1996). Urinary levels of 3-methoxy-4hydroxyphenylglycol predict symptom severity in selected patients with unipolar depression. Psychiatry Research, 62(2), 171-177. Garvey, M. J., & Underwood, K. (1997). Association of N-acetyl-betaglucosaminidase levels with seriousness of suicide attempts. Biological Psychiatry, 42(4), 286-289. Gaw, K. F., & Beutler, L. E. (1995). Integrating treatment recommendations. In L. E. Beutler & M. Berren (Eds.), Integrative assessment of adult personality (pp. 280-319). New York: Guildford Press. George, L. K., Blazer, D. G., Hughes, D. C., & Fowler, N. (1989). Social support and the outcome of major depression. British Journal of Psychiatry, 154, 478-485. Gitlin, M. J., Swendsen, J., & Heller, T. L. (1995). Relapse and impairment in bipolar disorder. American Journal of Psychiatry, 152(1635-1640). Godding, P. R., McAnulty, R. D., Wittrock, D. A., & Britt, D. M. (1995). Predictors of depression among male cancer patients. Journal of Nervous and Mental Disease, 183(2), 95-98. Goldfried, M. R., & Padawar, W. (1982). Current status and future direction in psychotherapy. In M. R. Goldfried (Ed.), Converging themes in psychotherapy (pp. 352). New York: Springer Publishing Co. Gonzales, L. R., Lewishon, P. M., & Clark, G. N. (1985). Longitudinal follow-up of unipolar depressives: An investigation of predictors of relapse. Journal of Consulting and Clinical Psychology, 53, 401-469. Gray, J. A. (1981). A critique of Eysenck's theory of personality. In H. J. Eysenck (Ed.), A model for personality (pp. 246-276). New York: Springer-Verlag.

Green, C. R., Marin, D. B., Mohs, R. C., Schmeidler, M. A., Fine, E., & Davis, K. L. (1999). The impact of behavioral impairment of functional ability in Alzheimer's disease. International Journal of Geriatric Psychiatry, 14, 307-316. Haas, B. K. (1999). Clarification and integration of similar quality of life concepts. Image: Journal of Nursing Scholarship, 31, 215-220. Hardy, G. E., Barkham, M., Shapiro, D. A., Stiles, W. B., Rees, A., & Reynolds, S. (1995). Impact of Cluster C personality disorders on outcomes of contrasting brief psychotherapies for depression. Journal of Consulting and Clinical Psychology, 63, 9971004. Hays, J. C., Krishnan, K. R. R., George, L. K., Pieper, C. F., Flint, E. P., & Blazer, D. G. (1997). Psychosocial and physical correlates of chronic depression. Psychiatry Research, 72, 149-159. Hoencamp, E., Haffmans, P. M. J., Duivenvoorden, H., Knegtering, H., & A, D. W. (1994). Predictors of (non)-reponse in depressed outpatients treated with a threephase sequential medication strategy. Journal of Affective Disorders, 31, 235-246. Hoglend, P. (1993). Personality disorders and long-term outcome after brief dynamic psychotherapy. Journal of Personality Disorders, 7(2), 168-181. Hunter, J.E., & Schmidt, F.L. (2004). Methods of meta-analysis: Correcting error and bias in research findings, 2nd edition. Thousand Oaks, CA: Sage Publications. Imber, S. D., Pilkonis, P. A., Sotsky, S. M., Elkin, I., Watkins, J. T., Collins, J. F., Shea, M. T., Leber, W. R., & Glass, D. R. (1990). Mode-specific effects among three treatments for depression. Journal of Consulting and Clinical Psychology, 58, 352-359. Joyce, A. S., Ogrodniczuk, J., Piper, W. E., & McCallum, M. (2000). Patient characteristics and mid-treatment outcome in two forms of short-term individual psychotherapy. Presentation at the 31st annual meeting of the Society for Psychotherapy Research, Chicago, IL. .

Joyce, A. S., & Piper, W. E. (1996). Interpretative work in short-term individual psuchotherapy: An analysis usig hierarchical linear modeling. Journal of Consulting and Clinical Psychology, 64, 505-512. Judd, L. L., Paulus, M. P., Wells, K. B., & Rapaport, M. H. (1996). Socioeconomic burden of subsyndromal depressive symptoms and major depression in a sample of the general population. American Journal of Psychiatry, 153(11), 1411-1417. Kadden, R. M., Cooney, N. L., Getter, H., & Litt, M. D. (1989). Matching alcoholics to coping skills or interactional therapies: Posttreatment results. Journal of Consulting and Clinical Psychology, 57, 698-704. Kagan, J. (1998). Galens prophecy: temperament in human nature. NY: Basic Books. Karno, M. (1997). Identifying patient attributes and elements of psychotherapy that impact the effectiveness of alcoholism treatment. Unpublished doctoral dissertation, University of California, Santa Barbara. Karno, M., Beutler, L. E., & Harwood, T. M. (in press). Interactions between psychotherapy process and patient attributes that predict alcohol treatment effectiveness: A preliminary report. Addictive Behavior. Kawai, H. (1993). Monogatari to Ningenno Kagaku (Stories and Humans Science). Tokyo: Iwanami Books. Kawai, H. (1996). The Japanese Psyche: Major Motifs in the Fairy Tales of Japan. Dallas, TX: Spring. Keijsers, G. P. J., Hoogduin, C. A. L., & Schaap, C. P. D. R. (1994). Predictors of treatment outcome in the behavioral treatment of obsessive-compulsive disorder. British Journal of Psychiatry, 165, 781-786.

Kocsis, J. H., Frances, A. J., Voss, C., Mann, J. J., Mason, B. J., & Sweeney, J. (1988). Imipramine treatment for chronic depression. Archives of General Psychiatry, 45, 253-257. Koenig, H. (1998). Depression in hospitalized older patients with congestive heart failure. General Hospital Psychiatry, 20(1), 29-43. Koolhaas, J. M., Korte, S. M., De Boer, S. F., Van der Vegt, B. J., Van Reenan, C. G., Hopster, H., De Jong, I. C., Ruis, M. A. W., & Blokhuis, H. J. (1999). Coping style in animals: Current status in behavior and stress-physiology. Neuroscience and Biobehavioral Reviews, 23, 925-935. Koran, L. M. (2000). Quality of life in obsessive-compulsive disorder. The Psychiatric Clinics of North America, 23, 509-517. Latack, J. C., & Havlovic, S. J. (1992). Coping with job stress: A conceptual evaluation framework for coping measures. Journal of Organizational Behavior, 13, 479508. Lawton, M., Moss, M., Fulcomer, M., & Kleban, M. (1982). A research and service oriented multi-level assessment instrument. Journal of Gerontology, 37, 91-99. Lazarus, A. A. (1981). The practice of multi-modal therapy. New York: McGrawHill. Lazarus, R. S., & Folkman, S. (1984). Stress, apporaisal and coping. New York: Springer. Lipsey, M.W., & Wilson, D.B (2001) Practical Meta-Analysis. Applied Social Research Methods Series, Volume 49. Thousand Oaks, CA: Sage Publications Litt, M. D., Babor, T. F., DelBoca, F. K., Kadden, R. M., & Cooney, N. L. (1992). Type of alcoholics: II. Application of an empirically derived typology to treatment matching. Archives of General Psychiatry, 49, 609-614.

Littlefield, C. H., Rodin, G. M., Murray, M. A., & Craven, J. L. (1990). Influence of functional impairment and social support on depressive symptoms in persons with diabetes. Health Psychology, 9(6), 737-749. Longabaugh, R., Rubin, G. M., Malloy, P., Beattie, M., Clifford, P. R., & Noel, N. (1994). Drinking outcomes of alcohol abusers diagnosed as antisocial personality disorder. Alcoholism: Clinical and Experimental Research, 18, 778-785. Lueger, R. J. (1996). Using feedback on patient progress to predict the outcome of psychotherapy. Journal of Clinical Psychology, 55, 1-27. Lyons, J. P., Welte, J. W., & Brown, J. (1982). Variation in alcoholism treatment orientation: Differential impact upon specific subpopulations. Alcoholism: Clinical and Experimental Research, 6, 333-343. Maling, M. S., Gurtman, M. B., & Howard, K. I. (1995). The response of interpersonal problems to varying doses of psychotherapy. Psychotherapy Research, 5, 63-75. Mazure, C. M., Nelson, J. C., & Jatlow, P. I. (1990). Predictors of hospital outcome without antidepressants in major depression. Psychiatry Research, 33, 51-58. McGue, M., & Bouchard Jr, T. J. (1998). Genetics and environmental influences on human behavioral differences. Annual Review of Neuroscience, 21, 1-24. McLean, P. D., & Taylor, S. (1992). Severity of unipolar depression and choice of treatment. Behavior Research and Therapy, 30, 443-451. McLellan, A. T., Woody, G. E., Luborsky, L., O'Brien, C. P., & Druley, K. A. (1983). Increased effectiveness of substance abuse treatment: A prospective study of patient-treatment "matching". Journal of Nervous and Mental Disease, 171, 597-605. Mersch, P. P. A., Emmelkamp, P. M. G., & Lips, C. (1991). Social phobia: Individual response patterns and cognitive interventions. A follow-up study. Behavioral Research and Therapy, 29, 357-362.

Miller, W. R., & Joyce, M. A. (1979). Prediction of abstinence, controlled drinking, and heavy drinking outcomes following behavioral self-control training. Journal of Consulting and Clinical Psychology, 47, 773-775. Mintz, J., Mintz, L. I., Arruda, M. J., & Hwang, S. S. (1992). Treatments of depression and the functional capacity to work. Archives of General Psychiatry, 49, 761768. Moos, R. H. (1990). Depressed outpatients' life contexts, amount of treatment and treatment outcome. Journal of Nervous and Mental Diseases, 178, 105-112. Newsom, J. T., & Schulz, R. (1996). Social support as a mediator in the relation between functional status and quality of life in older adults. Psychology and Aging, 11(1), 34-44. Newton R.E., & Rudestam, K.E. (1999). Your statistical consultant: Answers to your data analysis questions. Thousand Oaks, CA: Sage Publications. Ogles, B. M., Sawyer, J. D., & Lambert, M. J. (1995). Clinical significance of the National Institute of Mental Health treatment of depression collaborative research program data. Journal of Consulting and Clinical Psychology, 63, 321-326. Ollendick, T. H. & King, N. J. (2007). Empirically supported treatments typically produce outcomes superior to non-empirically supported treatments. In J. C. Norcross, L. E. Beutler,, & R. f. Levant (Eds), Evidence-based practices in mental health: Debate and dialogue on the fundamental questions (pp. 308-317). Psychological Association. Oxman, T. E., Freeman, D. H., & Manheimer, E. D. (1995). Lack of social participation or religious strength and comfort as risk factors for death after cardiac surgery in the elderly. Psychosomatic Medicine, 57, 5-15. Piper, W. E., McCallum, M., Joyce, A. S., Azim, H. F., & Ogrodniczuk, J. S. (1999). Follow-up findings for interpretive and supportive forms of psychotherapy and Washington, DC: American

patient personality variables. Journal of Consulting and Clinical Psychology, 67(2), 267273. Prochaska, J. O. (1984). Systems of psychotherapy: A transtheoretical analysis. (2nd ed.). Homewood, IL: Dorsey Press. Prochaska, J. O. & DiClemente, C. C. (1992). The transtheoretical approach. In J. C. Norcross & M. R. Goldried (Eds.) Handbook of psychotherapy integration. (pp. 300-334). New York: Basic Books. Project Match Research Group. (1997). Matching alcoholism treatments to client heterogeneity: Project Match posttreatment drinking outcomes. Journal of Studies in Alcoholism, 58, 7-29. Prudic, J., Sackeim, H. A., Davanand, D. P., & Kiersky, J. E. (1993). The efficacy of ECT in double depression. Depression, 1, 38-44. Rapaport, M. H., & Judd, L. L. (1998). Minor depressive disorder and subsyndromal depressive symptoms: Functional impairment and response to treatment. Journal of Affective Disorders, 48(2-3), 227-232. Ravindran, A. V., Anisman, H., Merali, Z., Charbonneau, Y., Telner, J., Bialik, R. J., Wiens, A., Ellis, J., & Griffiths, J. (1999). Treatment of primary dysthymia with group cognitive therapy and pharmacotherapy: Clinical symptoms and functional impairment. American Journal of Psychiatry, 156, 1608-1617. Ray, C., Jefferies, S., & Weir, W. R. C. (1997). Coping and other predictors of outcome in chronic fatigue syndrome: A 1-year follow-up. Journal of Psychosomatic Research, 43(4), 405-415. Ray, J.W. & Shadish, W.R. (1996). How interchangeable are different estimators of effect size? Journal of Consulting and Clinical Psychology, 64, 1316-1325.

Reynolds, S., Stiles, W. B., Barkham, M., Shapiro, D. A., Hardy, G. E., & Rees, A. (1996). Acceleration of changes in session impact during contrasting time-limited psychotherapies. Journal of Consulting and Clinical Psychology, 64, 577-586. Roberts, R., Kaplan, G., Shema, S., & Strawbridge, W. (1997). Does growing old increase the risk for depression? American Journal of Psychiatry, 154(10), 1384-1390. Rosenblatt, A., & Rosenblatt, J. (2000). Demographic, clinical, and functional characteristics of youth enrolled in six California systems of care. Journal of Child & Family Studies, 9(1), 51-66. Sanders, N. E. J. (1999). An assessment of coping and adjustment in individuals with Parkinson's disease and their caregivers. Dissertation Abstracts International, 60, 842. Schneider, L. C., & Struening, E. L. (1983). SLOF: A behavioral rating scale for assessing the mentally ill. Social Work Research and Abstracts, 19(3), 9-21. Scogin, F., Bowman, D., Jamison, C., Beutler, L. E., & Machado, P. P. (1994). Effects of initial severity of dysfunctional thinking on the outcome of cognitive therapy. Clinical Psychology and Psychotherapy, 1, 179-184. Scogin, F., Hamblin, D., & Beutler, L. (1987). Bibliotherapy for depressed older adults: A self-help alternative. Gerontologist, 27, 383-387. Shapiro, D. A., Barkham, M., Rees, A., Hardy, G. E., Reynolds, S., & Startup, M. (1994). Effects of treatment duration and severity of depression on the effectiveness of cognitive-behavioral and psychodynamic-interpersonal psychotherapy. Journal of Consulting and Clinical Psychology, 62, 522-534. Shapiro, D. A., Rees, A., Barkham, M., Hardy, G., Reynolds, S., & Startup, M. (1995). Effects of treatment duration and severity of depression on the maintenance of gains after cognitive-behavioral and psychodynamic-interpersonal therapy. Journal of Consulting and Clinical Psychology, 63, 378-387.

Shea, M. T., Elkin, I., Imber, S. D., Sotsky, S. M., Watkins, J. T., Collins, J. F., Pilkonis, P. A., Beckham, E., Glass, D. R., Dolan, R. T., & Parloff, M. B. (1992). Course of depressive symptoms over followup: findings form the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Archives of General Psychiatry, 49, 782-787. Smithson, M. (2003). Confidence Intervals. Quantitative Applications in the Social Sciences, Series 140. Thousand Oaks, CA: Sage Publications Snyder, P. & Lawson, S. (1993). Evaluating results using corrected and uncorrected effect size estimates. The Journal of Experimental Education, 61, 334-349. Sotsky, S. M., Glass, D. R., Shea, T. M., Pilkonis, P. A., Collins, J. F., Elkin, I., Watkins, J. T., Imber, S. D., Leber, W. R., Moyer, J., & Oliveri, M. E. (1991). Patient predictors of response to psychotherapy and pharmacotherapy: Findings in the NIMH Treatment of Depression Collaborative Research Program. American Journal of Psychiatry, 148, 997-1008. Spangler, D. L., Simons, A. D., Thase, M. E., & Monroe, S. M. (1997). Response to cognitive-behavioral therapy in depression: Effects of pretreatment cognitive dysfunction and life stress. Journal of Consulting and Clinical Psychology, 65, 568-575. Sperry, L., Brill, P. L., Howard, K. L., & Grissom, G. R. (1996). Treatment outcomes in psychotherapy and psychiatric interventions. New York: Brunner/Mazel. Strupp, H. H., Horowitz, L. J., & Lambert, M. J. (Eds.). (1997). Measuring patient change after treatment for mood, anxiety, and personality disorders: Toward a core battery. Washington, D.C: American Psychological Association Press. Tasca, G. A., Russell, V., & Busby, K. (1994). Characteristics of patients who chose between two types of group psychotherapy. International Journal of Group Psychotherapy, 44, 499-508. Thalheimer, W., & Cook, S. (2002). How to calculate effect sizes from published

research articles: A simplified methodology. Retrieved March 15, 2009 from http://work-learning.com/effect_sizes.htm Thase, M. E., Simons, A. D., Cahalane, J., McGeary, J., & Harden, T. (1991). Severity of depression and response to cognitive behavior therapy. American Journal of Psychiatry, 148, 784-789. Vallejo, J., Gasto, C., Catalan, R., Bulbena, A., & Menchon, J. M. (1991). Predictors of anti-depressant treatment outcome in melancholia: Psychosocial, clinical and biological indicators. Journal of Affective Disorders, 21, 151-162. Veiel, H. O., Kuhner, C., Brill, G., & Ihle, W. (1992). Psychosocial correlates of clinical depression after psychiatric in-patient treatment: methodological issue and baseline differences between recovered and non-recovered patients. Psychological Medicine, 22, 425-427. Welsh, G. S. (1952). An anxiety index and an internalization ratio for the MMPI. Journal of Consulting Psychology, 16, 65-72. Weissman, M. M., & Bothwell, S. (1976). Assessment of social adjustment by patient self-report. Archives of General Psychiatry, 33, 111-115. Woody, G. E., McLellan, A. T., Luborsky, L., O'Brien, C. P., Blaine, J., Fox, S., Herman, I., & Beck, A. T. (1984). Severity of psychiatric symptoms as a predictor of benefits from psychotherapy: The Veterans Administration-Penn Study. American Journal of Psychiatry, 141, 1172-1177. Yost, E., Beutler, L. E., Corbishley, A., & Allender, J. (1986). Group cognitive therapy: A treatment approach for depressed older adults. New York: Pergamon. Zarit, S. H., Femia, E. E., Gatz, M., & Johansson, B. (1999). Prevalence, incidence and correlates of depression in the oldest old: the OCTO study. Aging and Mental Health, 3(2), 119-128.

Zlotnick, C., Shea, M. T., Pilkonis, P., Elkin, I., & Ryan, C. (1996). Gender, dysfunctional attitudes, social support, life events, and depressive symptoms over naturalistic follow-up. American Journal of Psychiatry, 153, 1021-1027.