Received 10/20/09

Revised 04/27/10
Accepted 08/27/10

Research in Counseling:
A 10-Year Review to Inform Practice
Dee C. Ray, Darrell M. Hull, Andi J. Thacker, Laura S. Pace,
Karrie L. Swan, Sarah E. Carlson, and Jeffrey M. Sullivan

Hrrhis study reviewed 4,457 articles from 1998 to 2007 in American Counseling Association division-affiliated journals
to identify research articles published in counseling; 1,139 articles (25.6%) were quantitatively research based. The
authors provide details related to quantitative research publicafions, including individual journal contribution fo the
research base, focus areas for research, and independent and dependent variables highlighted by researchers. One
summary finding was that only 6% of counseling research articles explored effectiveness of counseling interventions.

Research publication in the field of counseling is a current
subject of discussion, as demonstrated by recent attention
regarding research practices by the American Counseling
Association's (ACA)flagshipjournal. Journal of Counseling
& Development (JCD, see Bangert & Baumberger, 2005;
Marotta & Watts, 2007; Nilsson, Love, Taylor, & Slusher,
2007). The focus on research seems to be motivated by both
internal and external sources requiring data-driven methods for
choosing and evaluating counseling interventions. Researchers
have noted the benefits of practice based on empirical evidence.
Bangert and Baumberger (2005) reported that government
fimding has increasingly required the use of interventions based
on randomized experimental designs. Mate and Kelly (1997)
emphasized the need for the establishment of an empirical base
for the professional practice of mental health counseling. They
fiarther highlighted the need for original research to promote
the mental health counseling field as a developed professioa
Since the early 20th century, practitioners and researchers
have been aware of the benefits of incorporating research-
based facts to direct clinical services (Rubin, 2008). Rubin
(2008) suggested that the practice of incorporating best
practices includes a process by which one evaluates reliable
evidence and appoints certain interventions as empirically sup-
ported. Publications based on empirically supported evidence
serve as a guide to practitioners seeking to uphold ethical
obligations to provide counseling services based on rigorous
research methodologies.
Perhaps, most important, counselors are guided by ethi-
cal obligations to seek evidence-based practices. According
to the ACA Code of Ethics (ACA, 2005), "counselors have a
responsibility to the public to engage in counseling practices
that are based on rigorous research methodologies" (p. 9).
Therefore, if counselors are to behave ethically, they must
be aware of practices that are empirically supported and the
general research that is occurring in the field.
Sexton (1996) discussed the importance of research when
selecting relevant and effective treatments to use with clients.
He emphasized that "counselors must be confident that the
current status of outcome research is a valid and reliable source
of information that can guide clinical practice" (Sexton, 1996,
p. 590) and that counselors must draw upon research when
determining the most effective treatment. Furthermore, Sexton
discussed the importance of consistent research focused on
areas of counseling that are relevant to the field. One specific
aim of his research was to determine if the research litera-
ture being produced met the practical needs of counselors.
Although he found that a variety of research publications
existed about multiple topics on which counselors can draw,
he did not report on quality or relevance of research available.
This omission seemed to suggest that although many different
topics have been researched, the consistency of research that
can contribute to a substantial knowledge pool is lacking.
Nilsson et al. (2007) published a content analysis of
quantitative research published in JCD over a 10-year period.
They found the greatest amount of research in the areas of
academic/career, multicultural issues, symptoms/disorder,
and counseling process. Although the findings of this study
carried significant implications about counseling research, the
authors focused on only 1 out of a possible 15 ACA-affiliated
journals. In addition, Nilsson et al. reviewed research from
1991 to 2000, indicating the need for a review of more current
research publication practices.
Counselor educators contribute to the evidence-based lit-
erature through publication in journals that may be motivated
by career productivity requirements or contribution to the
field. Barrio Minton, Fernando, and Ray (2008) investigated
the publication patterns of counselor educators by examining
submitted manuscripts from faculty members of doctoral pro-
grams accredited by the Council for Accreditation of Counsel-
ing and Related Educational Programs. The authors examined
articles published between January 1997 and December 2006.
Findings from this study indicated that approximately half
of the reviewed publications were research oriented, whereas
the remaining articles focused on theory and practice (Barrio

Dee C. Ray, Andi J.Thacker, Laura S. Pace, Karrie L. Swan, Sarah E. Carison, and Jeffrey M. Suiiivan, Department of Counseling
and Higher Education, and Darreil M. Huil, Department of Educational Psychology, University of North Texas, Dentón. Correspon-
dence concerning this articie should be addressed to Dee C. Ray, Department of Counseling and Higher Education, University of
North Texas, 1155 Union Circle, Box 310829, Dentón, TX 76203 (e-mail: dee.ray@unt.edu).

© 2011 by the American Counseling Association. All rights reserved.

Journal of Counseling & Development • Summer 2011 • Volume 89 349
 Ray et al.

Minton et al., 2008). Although Barrio Minton et al. reported nais? (b) In relation to the field of counseling, what were the
the frequency of published research articles, they cited as a intentions ofthe research? (c) On what type of population, if
limitation the fact that the study did not address the quality any, did counseling research focus? (d) What were the primary
or content ofthe research publications. dependent variables? and (e) What areas of counseling will
In response to the need for empirically supported counsel- benefit from this research?
ing practices, JCD established a section regularly devoted to
research articles as early as 1993. The editors ofJCD identified •Method
a distinct need to increase the amount of relevant research
articles published consistently (Herr & Fabian, 1993). As the The research team consisted of one faculty member and five
flagship journal for ACA, JCD recognized the importance doctoral students in a counseling education graduate program.
of providing counseling professionals with relevant research The faculty member (first author) held a doctoral degree in
information. In 2003, JCD added a practical section, titled counselor education and had experience conducting multiple
Best Practices, asserting that best-practice manuscripts con- quantitative and qualitative published studies. The doctoral
tain information regarding eotmseling interventions centered students (second, third, fourth, fifth, and sixth authors) held
on outcome data. JCD further emphasized that best-practice master's degrees in counseling and had completed at least two
manuscripts use evidence-based data to speak to one or more courses in advanced statistics and research design. In addition,
aspects of what interventions work well with specific coun- the faculty member trained the doctoral students over a period
selors, with specific clients, with specific problems, within of several weeks on coding designs in the collection of research.
a specific setting (McGowan, 2003). JCD, as well as other We first reviewed the ACA website (www.counseling.org)
counseling joumals (e.g.. Professional School Counseling; to identify ACA divisions that publish peer-reviewed joumals
Lapan, 2005), called on researchers to contribute to tbe field accepting research articles. We identified 15 joumals for review
of counseling by submitting manuscripts that validate and including ACA's flagship joumal. Journal of Counseling &
contribute to relevant counseling research that will further Development, and 14 joumals associated with AC A divisions.
guide best counseling practices. We included only joumals that were published over the entire
The purpose ofthe present research study was to provide span of 1998-2007, thereby excluding joumals that were in
an overview of types of research articles being published in early production. The full list ofjoumals can be fotind in Table
counseling-related joumals and, more specifically, to provide 1. We attempted to access each joumal issue in paper form, and
a detailed description of research content within quantitative if unavailable, electronic databases were searched forfiillissues.
research related to the following five questions: (a) How much The research team developed a coding sheet addressing
research is being published in ACA division counseling jour- several aspects of publication, including number of authors.
TABLE 1
Research Articies and Totai Articie Publications for Counseling Division Journais
From 1988 Through 2007
No. of Quantita- No. of No. of Total % Research to
tive and Mixed- Quailtatlve Research No. of Total No. of Total
Journal iMethods Articles Articles Articies Articies Articies
Aduitspar) 14 3 17 62 27
Ttie Career Development Quarterly 105 30 135 291 46
Counseling and Values 56 24 80 223 36
Counselor Education and Supervision 80 35 115 243 47
Family Journal 64 12 76 742 10
Journai tor Specialists in Group Work 13 14 27 312 g
Journal ot Addictions and Offender
Counseling 3g 5 44 g3 47
Journal of College Counseling go 7 g7 203 48
Journal of Counseling & Development 167 3g 206 543 38
Journal of Employment Counseling 74 10 84 153 55
Journal of Humanistic Counseling,
Education and Development' 26 g 35 231 15
Journal of tVlental Heattti Counseling 65 g 74 276 27
Journal of ti^ulticultural Counseling
and Development 71 14 85 237 36
tVleasurement and Evaluation in
Counseling and Development 150 6 156 185 84
Professional Sctiool Counseling 125 26 151 663 23

Totai 1,13g 243 1,382 4,457 31

"Journal ot Humanistic Counseling as of Vol. 50, Issue 1, Spring 2011.

350 Journal of Counseling & Development • Summer 2011 • Volume 89

Research in Counseling
type of research (i.e., quantitative, qualitative, or mixed meth-
ods), counseling area to be benefited by research (e.g., couples
counseling, counselors in training), sample description (i.e.,
clinical or nonclinical), sample description of ethnicity and
gender, other sample descriptors reported (e.g., income,
education level), research focus (intention of research; i.e.,
to what counseling area does it make a contribution), and
an open answer for primary dependent variables and sample
subpopulations of focus. For each coding variable, the research
team decided on a clear definition of the variable and a de-
scription of the coding choices to guide the coding process.
The definitions and descriptors were included in the coding
forms for quick access. Coders then independently coded the
samefivearticles using the initial coding system. The research
team met to discuss variations and challenges in coding. Open
answers for primary dependent variables and subpopulations
were compiled into one database for review by the team. The
team then categorized dependent variables and subpopulations
for fiirther coding. Coding variable definitions and descriptors
were refined according to consensus. Following this process,
the team rated 10 articles individually and independently on
the finalized coding form to establish interrater reliability.
Once rater agreement was demonstrated to be substantial (see
Landis & Koch, 1977), researchers were permitted to code
independently. Although ideal methods include multiple raters
of single articles, the considerable number of articles located
for this study limited the application of such a process. On
the basis of a thorough survey of similar literature reviews,
the research team decided to use the method of establishing
agreement prior to independent coding of articles (Fouad &
Byars-Winston, 2005; Hsieh et al., 2005; Skowron & Reine-
mann, 2005; Wood, Crane, Schaalje, & Law, 2005). Joumal
issues published fi-om 1998 to 2007 were divided for coding
by joumal among the research team members. In addition,
each team member provided the number of research articles
and the number of total articles for each joumal issue.
Coding Variables
Several of the following coding variables for the present study
were subjectively identified by raters (i.e., type of research,
research population, sample description, research focus,
setting in which the research was conducted, subpopulation,
and a process variable). These seven coding variables were
analyzed for interrater agreement and subjected to a generaliz-
ability analysis, and they are referred to in the generalizability
analysis as different methods by which the raters rated the
articles. Scott's P. coefficient is provided for each of the seven
identified coding variables.
Number of authors. The number of authors was listed as
an open-ended continuous variable.
Ethnicity of sample. Sample populations were coded for
reported ethnicity for each study. Because of the specificity
and variation of ethnicities reported (e.g., immigrant Kore-
ans in the United States), researchers concluded that broad
Journal ofCounselingôC Development • Summer 2011 • Volume 89
categories of ethnicity would not reveal an accurate level of
diversity in studies. Thus, we chose to summarize data by
coding whether ethnicity was reported or not as a measure of
standard of practice for research publications.
Other characteristics of sample. Other characteristics of
samples were coded when reported, including income, age,
education level, marital status, sexual orientation, and profes-
sional status (e.g., working or unemployed).
Randomization of the sample. Each study was coded for
its use of randomization of the sample with the following
categories: randomly assigned, not randomly assigned, or not
applicable for research design.
Type of research. Each article was coded as quantitative,
qualitative, or mixed methods {P^ = 0.85). Because the focus
of this research was on exploration of quantitative research
and because mixed-methods research (n = 171) resulted in a
small number of studies, mixed methods was collapsed with
the quantitative category to accurately represent quantitative
studies. Thus, it should be noted that the total number of
qualitative research articles is underreported.
Research population. We identified eight categories of coun-
seling and assigned each study to an area of counseling that
the research most likely benefits {P. = 1.00). For example, if a
research study explored the attitudes of mathematics majors in
college, we categorized the study as benefiting adult counseling.
Categories included counselors in training, adult counseling,
child counseling, adolescent counseling, family counseling,
couples counseling, counselor educators, and counselors in
practice. During discussion of population categories, it was
noted that college populations would be categorized as adults
because being over age 18 years is the demarcation of adult-
hood legally in the United States. In addition, the research team
concluded that information regarding college research would
be better coded under the research setting variable.
Subpopulation. In coding the content of research studies,
there seemed to be a preference among counseling research-
ers to study specific types of populations beyond diagnoses
or clinical issues. These populations appeared to be indepen-
dent variables that were of particular interest in counseling
research. The groups identified in the literature were typically
considered multicultural populations. Articles were coded {P.^
= 1.00) if the researchers specified a particular group as an
independent variable or focus of the study, such as attitudes
of Native Americans. Because we found that authors some-
times specified two independent population variables, such as
experiences ofAfrican American women, we decided that this
category could be multicoded. Categories included females,
males, Afiican American, Asian American, White/Caucasian,
Hispanic, Native American, multiethnic, intemational popu-
lations, religious, GLBT (gay, lesbian, bisexual, and trans-
gender), socioeconomic status, disabilities, and older adults.
Clinical versus nonclinical sample population. Each article
was coded as clinical or nonclinical for its sample population
(P. = 0.77). Clinical samples were defined as participants
351

who had been referred for a clinical problem or were already
receiving clinical services. Nonclinical samples were defined
as participants who had been recruited for the study.
Research setting. Studies were coded for the setting in
which the research was conducted {P.^ = 0.76). Categori-
cal variables were listed including college, school (pre-
K-12), clinical (including counseling clinics, hospitals,
etc.), and other.
Intention of research related to counseling field. The
purpose of this coding variable was to identify what area of
counseling this research describes and to what area does it
make a contdbution. There were 11 categories in this area;
to provide more meaningful data, the coders were forced to
choose one that was most representative of the researchers'
intentions (P. = 0.85). The following is a list of descriptions for
each area. Effectiveness of counseling intervention was coded
if the research study was a design intended to investigate the
effect of a specific counseling technique or modality; this was
typically represented by experimental, quasi-experimental, or
single-group pretest-posttest design. The counselor effective-
ness category included studies that addressed the effectiveness
of the counselor, including building counselor competencies
such as multicultural competencies. The counselor variables
category was idendfied if the study focused on descdptors of
counselors, such as developmental level of counselors. The
client variables category included studies that used mea-
surements to descdbe clients, such as client personality or
behavioral characteristics. The diagnosis variables category
descdbed a diagnosis or population with a specific diagnosis.
The process of counseling category included research that
explored dynamics dudng counseling sessions. The counselor
educator vadables category included research that described
counselor educators, such as comparing attitudes of non-
tenured with tenured faculty. The measurement/instrument
category contained any studies in which the primary objec-
tive was to investigate the use of a particular instrument. The
population vadables category included research studies that
descdbed a group of people who were not clinical, such as
self-esteem of intemadonal college students. The supervision
category included any research on the process or outcome of
supervision of counseling students or counselors in practice.
And the "other" category was provided to categodze any study
that did not meet cdteda for any of the previously mentioned
categorizations.
Dependent variable. Studies were coded to identify content
areas of expected outcome {P^ = 0.81). We identified 22 areas
of content (see Table 6, p. 355) that incorporated areas of focus
for counseling researchers and then coded each study for one
of these content areas. In this process, we attempted to define
the single most important dependent vadable assessed by the
researcher, resulting in a forced-choice coding vadable (i.e.,
no multicategodcal coding).
Data analyses. Studies were coded for use of data analysis
methods, such as t tests, correlation, regression analysis, and
352 Journal of Counseling & Development • Summer 2011 • Volume 89
Ray et al.
so on. Because the coding team did not categodze analyses
into broader categodes but listed each type of data analysis
separately, interrater reliability was not established for this
vadable. Hence, results should be interpreted with caution. In
addition, individual studies were multicoded because of the
use of multiple types of analyses in a single article.
Reliability of Coding Procedures
Scott's Pj coefficient (Scott, 1955; analogous to Cohen's
kappa coefficient; Cohen, 1960) was used to examine inter-
rater reliability, whereby each rater was crossed with every
other rater on a subset of 10 randomly selected articles and
each article was rated on seven separate subjective methods
by all six raters. No coefficients were less than 0.75. However,
when measurement procedures, such as the one used in the
present study, introduce multiple sources of measurement
error, generalizability theory (Cronbach, Gleser, Nanda,
& Rajaratnam, 1972; Brennan, 2001; for an overview, see
Shavelson & Webb, 1991) uses classical analysis of vadance
(ANOVA) to estimate meastirement errors and G coefficients
for the entire measurement procedure. We identified three
facets, which can be thought of as independent vadables or
factors in ANOVA, that constitute the qualitative conditions
under which the data were obtained: methods (M), articles
(A), and raters (R). The observed ratings on one article with
one method of measurement is not fully dependable; that is,
it is unlikely that we would observe exactly the same rating
frequency on a different article or for a different method of
measurement. Implicit in this idea is that the ratings result
from a steady state within the person providing the radng, and
we assume that any differences in ratings from one article to
the next or between different measurement scales (methods)
are not attdbutable to systematic changes in the individual
providing the rating.
The value in applying generalizability theory is that mul-
tiple sources of measurement error can be estimated separately
in a single analysis. This allows the observation of relative
error vadance associated with each facet in the study and at
the same time provides a single coefficient refiecting the level
of dependability, analogous to a reliability coefficient (such as
Cronbach's alpha). In the present study, all facets (methods,
articles, and raters) wereftiUycrossed, thus there are no con-
founded or nested sources of measurement error. The aim of
the ANOVA, then, is to assess the relative importance of the
idendfied sources of rating vadation. This is accomplished
through the esdmadon of vadance components in a simple
three-facet crossed design MAR, in which methods (M) were
independently applied to articles (A) by different raters (R).
The score decomposidon equation would be
Y=a + M.+ A.+ RA- MA.+ MR A- AR.+ MAR.^e...
ijk f^ I J k y I* jk ijk' ¡jk
where 7..^ is the rating frequency obtained on Method / of
Article^' for Rater k, \i is the overall mean score (the average
Research in Counseling

of all ratings), M. is the effect of Method i (the deviation of the
ith measurement scale's average score fVom the grand mean),
Aj is the effect of Article 7 (the deviadon of theyth article's
average score from the grand mean), and Ä^ is the effect of
Rater k (the deviation of the kth rater's average score from the
grand mean). MA.., MR.^, and AR.^^ represent Method x Article,
Method X Rater, and Article x Rater interaction effects (e.g.,
for AR.1^, this is the degree to which Rater / deviates from his
or her normal severity/leniency from article to article). MAR..^^
represents completely confounded variance among methods,
articles, and raters and thus is synonymous with an error term
(e..^. Because effects are assumed to be independent, the
total score variance ( a p represents the sum of the variance
components: methods (CJ^), articles (a^), raters ( o p , and the
interactions between Methods x Articles (o^^). Methods x
Raters (o^^). Articles x Raters (CJ^^), and Methods x Articles
X Raters (o^^^), which includes the residual variance:
The G study was performed using EduG software (Car-
dinet, Johnson, & Pini, 2010), which is designed to carry
out generalizability analyses. The seven different methods
for examining articles were treated as a fixed facet, because
these are the only attributes of interest. Each facet or facet
interacdon is shown in the ANOVA source in Table 2. Sums
of squares, degrees of freedom, and mean squares are pro-
vided in the first three columns, in addition to three sets of
estimated variance components. Colutnns 4 through 6 contain
the estimated variance components for an entirely random
effects model, a mixed model, and corrected model using
Whimby, Vaughan, and Tatsuoka's (1967) correction factor.
Negative variances are not possible in theory, but the raters
variance and Methods x Raters variance contain sampling
fluctuations around small component values that resulted in
negligible negative component esdmates. While these nega-
tive values appear in the ANOVA table, they were replaced
by zeros in the computadon of generalizability parameters.
The most important column of this table (column 4) shows
the percentage of variance associated with each facet or
facet interaction. Note that the different methods contribute
the largest amount of variance to the total, combined with
the interaction of the different methods with the different
articles used for the reliability analysis. Both of these high
variance components should have high contributions to any
variance observed in this measurement procedure. Moreover,
the contribution of rater effects to variance is very low, as are
the rater interaction components.
In Table 3, the different error variances are displayed along
with the various facets of the study, and their interactions are
separated into differentiation facets (the first two columns)
and instrumentation facets. These are based on the data in
the ANOVA results from Table 2 and are calculated using the
estimation algorithms appropriate to the sampling status of
the different facets. Methods are treated as a differentiation
facet, which is the one single measurement component that
produces an effective difference between measured entities.
Two separate G study analyses are represented; the first rela-
tive study includes an a priori measurement design in which
the interactions with the differentiation source are examined,
and the second analysis, which is more important for our
purposes, is the absolute error variance showing the contri-
budon of all components of variance with the exception of
the differendadon variance (the component we expected to
produce the most variance). The differendation variance is
more than eight dmes that of the absolute error variance; all
other things being equal, this higher differendation variance
indicates that the sources of measurement error (articles and
raters) are highly consistent (reliable).
A ratio of the estimated differendation variance to estimat-
ed total variance of observed scores yields a G coefficient. The
absolute G coefficient is an indication of the global reliability
of the measurement procedure when the methods differentia-
tion variance (the different ways the articles were examined)

TABLE 2
ANOVA Source Table Demonstrating Percentage Contribution of Each Source to Totai Variance
Variable Components
Source ss df MS Random Mixed Corrected % SE
Methods 500.67 6 83.44 1.26 1.26 1.08 36.5 0.70
Articles 74.00 9 8.22 0.00 0.18 0.18 6.1 0.09
Raters 2.48 5 0.50 0.00 0.00 0.00 0.0 0.01
Methods and
articles 434.33 54 8.04 1.23 1.23 1.06 35.6 0.25
Methods and
raters 11.99 30 0.40 -0.02 -0.02 -0.02 0.0 0.01
Articles and
raters 29.14 45 0.65 0.00 0.09 0.09 3.1 0.02
Methods,
articles, and
raters 174.72 270 0.65 0.65 0.65 0.55 18.7 0.06

Note. ANOVA = analysis of variance.

Journal ofCounseling& Development • Summer 2011 • Volume 89 353

 Ray et al.

TABLE 3 in = 123), 2001 (« = 146), 2002 (n = 129), 2003 (n = 138),

2004 (n = 147), 2005 (n = 149), 2006 (n = 154), and 2007 (n
G Study Table (Measurement Design:
= 151). From 2004 to 2007, publication of research seemingly
Methods, Articles, and Raters)
stabilized for the final 4 years reviewed.
Source of Variance DV REV % Rei. AEV % Abs. Table 1 provides a ñill listing of journals as well as the
Methods 1.08 number of quantitative/mixed-methods articles and number
Articles 0.02 13.4 of qualitative articles in relationship to total publication.
Raters 0.00 0.0
Methods and articles 0.11 92.0 0.11 78.6 Measurement and Evaluation in Counseling and Development
Methods and raters 0.00 0.0 0.00 0.0 published the highest percentage of research to total articles
Articles and raters 0.00 1.1 with 84% of its articles dedicated to research, and the next
Methods, articles, and
raters 0.01 8.0 0.01 6.9 highest percentage was represented by Journal of Employ-
ment Counseling (55%). Journal for Specialists in Group
Sum of variances 1.08 0.12 100.0 0.13 100.0 Work published the lowest percentage with 9% of its articles
SD 1.04
reporting research. The percentage of the total number of
Note. Coefficient G relative = 0.90; Coefficient G absolute = 0.89. research articles in comparison to total number of published
Grand mean for levels used = 2.67. Variance error of the mean for articles is 31% (n - 1,382). If only quantitative and mixed-
levels used = 0.02. Standard error of the grand mean = 0.14. Relative methods articles are compared with total publication articles,
standard error = 0.34. Absolute standard error = 0.37. DV = differen-
the percentage drops to 26% (« = 1,139).
tiation variance; REV = relative error variance; % Rel. = percentage
relative; AEV = absolute error variance; % Abs. = percentage absolute.
Research Question 2: In Relation to the Field
is partitioned out of the model and can be interpreted similar of Counseling, What Were the Intentions of the
to an alpha coefficient in the case of a simple fully crossed Research?
design as shown here. The G coefficient certainly suggests Table 5 lists the intentional focus of research in relation-
that measurement precision is satisfactory, as .89 represents a ship to the field of counseling. As can be seen in Table 5,
good degree of measurement precision. Reliability coefficients the highest number of counseling research articles among
(relative and absolute) for each of the 10 articles and the six the categories listed was focused on population variables
raters are provided in Table 4. (36.8%), which is an exploration of nonclinical groups for
nonclinical problems. Examples of this type of research
•Results are characteristics of first-generation college students or
Research Question 1 : How Much Research Is attitudes of human resource employees. In a distant second,
Being Published in Counseling-Related Journals? measurement/instrument research accounts for approxi-
mately 15% of counseling research. The fewest number of
We first attempted to answer this question by listing the num- research studies were conducted on issues related to specific
ber of articles divided by year of publication. The following diagnoses and groups of people with specific diagnoses
is a breakdown of number of studies published for each year (diagnosis variables = 0.7%). Also, it is interesting to note
across all 15 journals: 1998 (n = 128), 1999 (n = 117), 2000 that the study of counseling interventions was ranked 6th of
TABLE 4 10 categories with 6% of research focused on this intention.
G-Facets Analysis
Facet and TABLE 5
Levei Coefficient G Reiative Coefficient G Absoiute Intention of Research Related to
Articies Counseling Field
1 0.88 0.86
2 0.87 0.85 No. of % Focus
3 0.88 0.86 Articies in toTotai
4 0.89 0.87 Research Focus Area Focus Area Research
5 0.91 0.90
6 0.88 0.86 Population variables 509 36.8
7 0.87 0.85 Measurement/instrument 210 15.2
8 0.89 0.88 Counselor variables 143 10.3
9 0.93 0.91 Client variables 128 9.3
10 0.92 0.90 Counselor effectiveness 114 8.2
Raters Effectiveness of counseiing interventions 83 6.0
1 0.90 0.89 Counselor educator variables 48 3.5
2 0.90 0.89 Process of counseling 42 3.0
3 0.90 0.89 Supervision 34 2.5
4 0.91 0.89 Diagnosis variables 10 0.7
5 0.90 0.89 Other 61 4.4
6 0.90 0.89 Note.N= 1,382.

354 Journal ofCounseling& Development • Summer 2011 • Volume 89

Research in Counseling

Research Question 3: On What Type of Population, TABLE 6

if Any, Did Counseling Research Focus?
Dependent Variables Identified in
Results addressing this research question were derived from Research Publications
three coding variables: subpopulation, clinical versus nonclini- % DV to
cal sample population, and research setting. Approximately No. of Total
70% (« = 959) of research publications did not identify a Dependent Variabie (DV) Studies Research
specific subpopulation. Thus, the vast majority of research Counseling 400 28.9
articles used study samples with general characteristics, Career 250 18.1
Development 141 10.2
ostensibly to allow generalization of findings to the general
Relationships gg 7.2
population. Of those studies that focused on a particular sub- Multiculturalism 73 5.3
group ofthe population (n = 423), females were studied the Academic 47 3.4
Substance use/abuse/addiction 43 3.1
most, in 109 studies, which represent approximately 8% of 3g 2.8
Crisis
all research articles. African Americans were the second most Self-esteem 38 2.7
studied population with 63 articles (4.6%). Asian Americans Spirituality/religion 33 2.4
were third (n = 59; 4.3%), intemational populations outside Depression ig 1.4
Attachment ig 1.4
ofthe United States were fourth (n = 55; 4%), and males were Disruptive behaviors 18 1.3
fiflh (n = 50; 3.6%). White/Caucasian (3%), Hispanic (3%), Body image/eating disorders 15 1.1
religious (2%), GLBT (1.9%), socioeconomic status (1.7%), Ethics/legal 14 1.0
Stress 14 1.0
disabilities (1.3%), older adults (1.2%), multiethnic (1.1%), Anxiety g 0.7
and Native American ( 1 %) were the subgroups of least focus. Attention-deficit/hyperactivity disorder 8 0.6
In another attempt to address this research question regard- Medical 6 0.4
Other Diagnostic and Statistical Manual
ing what populations are being studied, we coded two other disorders 5 0.4
pertinent variables; clinical versus nonclinical populations Grief/loss 4 0.3
Other 88 6.4
and settings in which the research was conducted. On the
question of clinical and nonclinical population, 134 stud- Note. N = 1,382.
ies (10%) reported investigating a clinical sample, whereas
the overwhelming majority, 1,248 studies (90%), reported which research made contributions. The following is a break-
investigating nonclinical study participants. We also coded down of counseling categories and their matched studies: adult
studies according to the setting in which the research was counseling (« = 642; 46.5%), counselors in practice (n = 244;
conducted. Ofthe 1,382 research studies, 609 (44%) were 17.7%), counselors in training (n = 152; 11%), adolescent
conducted in colleges/universities, 283 (21 %) in schools, 130 counseling (n = 134; 9.7%), child counseling (n = 79; 5.7%),
(9.4%) in clinics or clinical settings, and 360 (26%) in other counselor educators (n = 69; 5%), couples counseling (« = 36;
settings, which were typically workplaces such as studies in 2.6%), and family counseling (« = 26; 1.9%). In comparison
the employment literature. with other categories, the highest numbers of research studies
were being conducted to benefit adult counseling.
Research Question 4; What Were the Primary
Dependent Variables? Supplemental Results Data
Table 6 lists each of the categories of dependent variables Other results related to data collection were summarized as
and how many studies emphasized that variable. Not sur- potential points of interest to the counseling community. With
prisingly, processes related to counseling were identified as regard to number of authors, authorship ranged from 1 to 13
the dependent variable represented in the largest ntimber of authors for one article, with approximately 16% of articles
studies (n = 400; 28.9%). These could include a coimselor's published by single authors. Most articles were published
skills or dynamics in the counseling process. Second, career- by 2 to 4 authors, representing approximately 76% of all re-
related variables were highly represented with 250 publica- search publications. When research studies involved human
tions (18.1%). Development, which included Wellness and participants, most authors reported ethnicity of sample (n =
counselor development, was the third most likely dependent 936; 67.7%); however, a large number of publications did
variable to be studied (n = 141; 10.2%). Grief and loss were not report ethnicity (n = 446; 32.3%). Authors reported other
the least likely topics to be studied among the counseling sample characteristics at various rates, including income (« =
literature (n = 4; 0.3%). 126; 9.1%), age (« = 670; 48.5%), education level (n = 693;
50.1%), marital status (n = 173; 12.5%), sexual orientation
Research Question 5: What Areas of Counseling (n = 43; 3.1%), and professional status (n = 198; 14.3%).
Will Benefit From This Research? Randomization of the sample was rare, with frequency at
The final research question attempted to use the population 130 studies (9.4%). The most utilized statistical analysis was
coding variable to identify specific areas of counseling to correlation (n = 327; 23.7%), with ANOVA (« = 297; 21.5%),

Journal ofCounseling & Development • Summer 2011 • Volume 89 355


regression procedures (n = 282; 20.4%), multivariate analysis
of variance (« = 211; 15.3%), and t tests (« = 175; 12.7%)
rounding out the top five mostfi-equentlyused analyses. No-
tably, the nonparametric chi-square analysis was conducted
in 160 studies, representing 11.6% of all research studies.
•Discussion
Over a decade ago. Mate and Kelly ( 1997) stated, "Publication
of original research is a key criterion for judging the identity
development of professions" (p. 286). Results of the present
study indicate that the counseling profession is producing
research at a consistent rate. The rise of research publications
ti-om 2004 to 2007 demonstrates that the field of counseling
is attempting to produce research that progresses counseling
to an adequate level ofprofessional status. Interpretation of
the 31% of research articles to total articles found in this
research is challenging. Previous publications have reviewed
the research publication in individual journals (Mate &
Kelly, 1997; Nilsson et al., 2007), but there has never been a
comprehensive exploration of all counseling-relatedjoumals.
Although all ACA-affiliated joumals publish original research,
some concentrate on research at the expense of practice ar-
ticles, whereas others concentrate on practice at the expense
of research, leading to a wide variation of research publica-
tions in comparison to total publications across joumals. JCD
attempted to address this balance by adding a Best Practices
section to its joumal in 2003. The purpose of the Best Practices
section was to encourage counselors to "apply the research
evidence that undergirds the practice of counseling" (Marotta
& Watts, 2007, p. 491). However, this statement inspires the
question, "What research base do counselors have for the
practice of counseling?"
Evidence-Based Counseling: The Need for More
Intervention Research
Although evidence-based psychological interventions can
be defined through strict adherence to rigorous experimental
intervention designs and identified through concrete criteria
set forth by professional organizations such as the American
Psychological Association (APA; see www.psychology.sun-
ysb.edu/eklonsky-/divisionl2/faq.html), counseling literature
appears to link effective practice with a broader definition
of evidence. Evidence for counseling practices includes the
coconstruction of practice between counselor and client,
clinical practicalities, consideration of diverse populations,
and theoretical belief systems, along with empirical support
for practice (Marotta & Watts, 2007). The debate within the
mental health field regarding evidence-based practice has
grown to new heights over the last two decades and is often
a point of contention among professionals (Nathan, 2004;
Rubin, 2008; Schloss & Haaga, 2005). Some mental health
professionals believe a strict definition of evidence-based
practice is reductionistic and mechanical and propose more
356 Journal of Counseling & Development • Summer 2011 • Volume 89
Ray et al.
humanistic methods of evaluation (see Cain & Seeman,
2001); altematively, other mental health professionals view
as unethical counselors who do not exclusively use identified
evidence-based practices (see Nathan, 2004). And then there
are the majority of counselors who fall somewhere in the
middle of the debate.
However, the debate over evidence-based practice does
not appear to serve as a driving force in counseling literature
when one considers the results of the present study. Perhaps
the most conceming tinding of this study is that only 6% of
all counseling research articles are focused on exploring the
effectiveness of a counseling intervention. There are few
answers in the counseling literature to the most basic ques-
tions of the profession, "What works with clients?" and "Are
counselors doing what works?" The absence of counseling
literature on intervention motivates further questions regard-
ing counseling practice. Are counselors looking to other fields
such as psychology or social work for direction in practice?
Are counselors operating without empirical evidence regard-
ing practice? Do counselors believe that empirical evidence
is less relevant to practice? Do counselors rely on theory and
experience rather than empirical research to guide practice?
The low publication rate of intervention-based research leads
to more questions than answers. Future research might explore
the counseling practitioner's use of published research. In ad-
dition, a retrospective review of joumals to interpret the 6%
counseling intervention research publication rate is warranted.
Although counseling joumals experienced an increase in
research publications over the 10-year span, this may not be
specific to intervention research publications. Findings ftom
other disciplines indicate that intervention research may be on
the decline. Hsieh et al. (2005) reported that intervention re-
search in selected educational psychology joumals decreased
ti-om 40% to 26% over a 21-year period from 1983 to 2004.
In addition, intervention research published in the American
Educational Research Journal dropped trom 33% to 4% in
the same time span. Robinson, Levin, Thomas, Pituch, and
Vaughn (2007) further concluded that intervention studies in
teaching and leaming joumals declined trom 45% to 33%
over a 10-year period ti-om 1994 to 2004. Henson, Hull, and
Williams (2010) suggested that a lack of quality research
publications may be infiuenced by a lack of knowledge and
skills in doctoral-level education. In each of these studies,
authors emphasized the need for intervention research to
support practice.
There has been speculation that although counselors are
interested in research, they are often tied to practice or un-
skilled at conducting or understanding research (Bangert &
Baumberger, 2005; Lundervold & Belwood, 2000). If this
speculation is true, then it is unclear whether a lack of inter-
vention studies is due to counselors not conducting research
on intervention or joumal venues not publishing intervention
research. Although there are no verifiable answers to the many
questions regarding intervention publications, it does appear
Research in Counseling
that the field of counseling would benefit from a greater
concentration on increasing intervention research. Even if
counselors are not limited to pracdcing only those techniques
with empidcal evidence, most professionals would agree
that evidence for practice is helpful in several ways. Most
important, empidcal exploration of interventions provides
information descdbing under what conditions and with what
populations certain interventions are most effective.
Operating from empidcal evidence provides protection
from legal downsides of practice, including lending support
for current practices. And finally, empidcal evidence for in-
tervendon helps to gain funding from government and pdvate
sources and is helpful for marketing services to clients.
Utility of Current Counseling Research to the
Practitioner
Results of this review indicate that the typical research study
published in counseling joumals seemed to be a research
exploration descdbing aspects of general or specified popula-
dons that were not considered clinical and often using a col-
lege sample. This statement is not meant to be an indictment
of counseling research, only an explanation. There is certainly
value to the counseling profession to discover personality
and behavioral aspects of nonclinical populations. After all,
cotinselors concenfrate on the healthy development of people;
hence, the study of general developmental aspects of people
seems warranted. However, counselors also work with people
who have significant mental health problems. Of the research
articles examined, few focused on common clinical issues
such as depression, anxiety, and gdef and loss (2.4%). Given
reports that major depressive disorder is the leading cause
of disability in the United States for individuals ages 15-44
years and that anxiety disorders affect approximately 40
million adults each year (Kessler, Chiu, Demler, & Walters,
2005), the need for research on depression and anxiety ap-
pears significant. In addidon, gdef and loss are considered to
be universal human expedences, as well as dsk factors for the
development of depression and anxiety (Servaty-Seib, 2004).
Mental health counselors work with clients who are labeled
with diagnoses that are threatening to everyday funcdoning.
School counselors work with children who are dismptive to
their own leaming as well as others. Human resource counsel-
ors work with employees who may lose their career because
of mental health issues. These types of services call for an
increase in cotmseling research related to clients and client
problems. This study indicated that of 1,382 research studies,
only 134 explored a clinical population. In other words, less
than 10% of counseling research is dedicated to clients most
in need of counseling services.
One argument against increasing research related to client
and client issues is that as a cotmseling profession, counselors
are focused on development and Wellness. Yet, this appears to
be the very reason for increasing research in the positivistic
area of development to identify effective approaches and
Journal of Counseling & Development • Summer 2011 • Volume 89
detennine what positive approaches work with clients. The
medical model adopted by other mental health professions
concentrates on finding what is wrong with clients and then
fixing the problem. A counseling approach to client research
is based on looking at the whole person and how impaired
functioning is related to the whole, not just the problem.
This holisdc perspective contdbutes to finding new research
designs and ways of working with clients, as well as new im-
portance attdbuted to clinical significance of interventions. An
emphasis on conducting research from a holistic perspective
with real clients in real life may provide infinite possibilities
for the counseling practitioner.
Counseling Areas as a Focus
As previously descdbed, much of the counseling research re-
viewed for this study was committed to adding information to
the pracdce of adult counseling, which for the purposes of this
study included college coimseling. Informadon is provided
about the attdbutes or charactedsdcs of adult populadons or
possibly measurements that are designed for adult popula-
tions. Almost 47% of all research publicadons benefited adult
coimseling. Counselors in pracdce and counselors in training
were the second and third most frequent areas that benefited
from the research literature, combining for close to 29% of
all research publications. Research to inform child counsel-
ing made up only 5.7% and adolescent counseling made up
only 9.7% of the publications. These particular percentages
are disappointing considedng that Mellin (2009) referred to
children's mental health as a cdsis and summoned counsel-
ors to increase fraining and research in response to current
needs of children. Also, we were surpdsed to see the scarcity
of research related to couples and family counseling, which
made up fewer than 5% of all research publicadons. Certainly,
counseling pracddoners are seeing couples and families on
a consistent basis, and research in counseling should reñect
such practice. In faimess, it is noted that conducting research
on children, couples, and families is challenging and includes
many more barders than conducdng research with individual
adult populations. Barders may include legal challenges in-
volved in child research, informed consent and assent with
participants, and pracdcal considerations of organizing inter-
vendons for multiple participants simultaneously.
Limitations
Limitations of this study include the subjective nature of cat-
egodzation among independent raters. Although substantive
interrater reliability with dgorous statisdcal generalizability
analysis was established pdor to coding, each study was coded
by an individual rater who is subject to systemadc bias. It is
possible that another research team might come to different
conclusions regarding the categodzation process of the vad-
ous content areas. In addition, each rater attempted to access
hard copies of all issues of each joumal. However, in some
cases, raters relied on electronic databases, which may pos-
357

sibly result in missing individual articles within journal issues.
To address this issue, we cross-referenced each electronic
article with the electronic table of contents for each journal
issue. Regarding the immense scope of this research project,
it is doubtful that the review of 4,457 counseling articles
with meticulous reading of 1,382 identified research articles
resulted in error-fVee data.
In the context of extensive data collection, decisions regard-
ing presentation were made to aggregate data into categorical
findings. The purpose of aggregadng the data is to provide
some level of order for the reader's understanding. However,
aggregadon of data may also lead to more globalized presenta-
don, which may provide less meaning and possibly devalue
small qualitative differences.
Implications for the Counseling Profession
The good news is that the counseling profession is producing
original research that is intended to have an impact on the
pracdce of counseling. Close to one third of all counseling
publicadons over a 10-year period were research based. The
counseling profession produces research related to practitioners,
counselors in training, counselor educators, clients, diagnoses,
and the general population. However, the benefits of current
research to counseling practice are questionable. There seems
to be a strong emphasis on conducting research with nonclinical
populations to define characteristics and attributes, but there is
little focus on conducting research regarding coimseling inter-
ventions that will likely benefit a clinical population.
Conducting intervendon research is challenging at best.
Counseling researchers face hurdles from the beginning with
the inidal approval of human participants, where committees
are often hesitant to approve the use of specific techniques
for persons with mental health problems. Recruiting sample
participants who meet criteria and are willing to participate is
often where many researchers lose modvadon. When there is an
attempt to randomize the sample, the researcher may encounter
problems related to setting (Hsieh et al, 2005). In addidon, it
is often difficult to find measurements to appropriately isolate
dependent variables and then administer those measurements on
schedule. There are many reasons not to conduct intervendon
research. However, as demonstrated through this study, there
is a great need for intervention research.
Challenges related to intervention research can be met
through different options available to the counseling com-
munity. First, research teams are the best defense against
developing a sense of hopelessness and for developing creative
solutions to research problems. A research team involves
several people of similar professional backgrounds working
toward the same goal. Hence, when one member meets a
research challenge, there are others to provide inspiration for
condnuance. Findings from this study indicate that research
articles are more likely to be published by two to four authors
(76%) than by a single author (16%). This result is consistent
with that of Jones et al. (2010), who reported a mean of 2.6
358 Journal ofCounselingSc Development • Summer 2011 • Volume 89
Ray et al.
authors per article in the field of educational psychology for
the 2008 publicadon year. Second, acquisition of funding is
particularly helpful for assurance of research completion,
specifically to provide incentives to participants to complete
experimental or control conditions and/or complete mea-
surement administrations. Funding for interventions to work
with common societal problems such as depression, atixiety,
or disruptive behavioral problems is possible through both
govemment and private sectors. Setting reasonable expecta-
dons for research participation is another option to encour-
age intervention research. Often, counselors believe that
experimental research must be conducted with htmdreds of
participants to be considered of substance. However, rigorous
experimental designs can be conducted with a small number of
participants. For example, 20 participants might be randomly
assigned to each of two conditions, experimental and control,
thereby only requiring 40 participants in all and allowing for
statistical analysis. Although limited in power, small rigorous
designs lend credibility to specific interventions and can later
be used in meta-analyses and systematic reviews to contribute
to evidence-based practice.
Notably, research studies must include details regarding
sample characterisdcs, methodology, and data analysis to be
used for the broader purpose of meta-analyses and systematic
reviews. Results from the present study indicate that sample
characteristics such as ethnicity, gender, or other relevant
descriptors, as well as sampling procedures, frequently may
not be reported in published research. Joumal editors are
in a position to require reporting procedures as criteria for
publication.
Finally, cotmselor education programs can address the need
for more clinical research by initiadng the research process at
the student level. As described in Ray, Barrio Minton, Schot-
telkorb, and Brown (2010), students in counseling programs
can participate in research projects early on, from entering
data to midlevel contribudon of delivering intervendon to later
leadership in organization and administradon of an interven-
tion research design. Once cotmselors in training experience
the success of research completion, it is logical that they
will be more willing to conduct research as a practitioner or
counselor educator.
•References
American Counseling Associadon. (2005). ACA code of ethics.
Alexandria, VA: Author.
Bangert, A. W., & Baumberger, J. P. (2005). Research and stadsdcal
techniques used in the Journal of Counseling & Development:
1990-2001. Journal of Counseling & Development, 83,480-487.
Barrio Minton, C. A., Fernando, D. M., & Ray, D. C. (2008). 10 years
of peer-reviewed publications in counselor education: Where,
what, who? Counselor Education and Supervision, 48, 133-143.
Brennan, R. L. (2001). Generalizability theory. New York, NY:
Springer.
Research in Counseling
Cain, D., & Seeman, J. (Eds.). (2001). Humanisticpsychotherapies:
Handbook of research and practice. Washington, DC: American
Psychological Association.
Cardinet, J., Johnson, S., & Pini, G. (2010). Applying generalizability
theory using EduG. In G. Marcoulides (Series Ed.), Quantitative
methodology series. New York, NY: Routledge.
Cohen, J. (1960). A coefficient of agreement for nominal scales.
Educational and Psychological Measurement, 20, 37—46.
doi: 10.1177/001316446002000104
Cronbach, L. J., Gleser, G. C , Nanda, H., & Rajaratnam, N. (1972).
The dependability of behavioral measurements: Theory of gener-
alizability for scores and profiles. New York, NY: Wiley.
Fouad, N., & Byars-Winston, A. (2005). Cultural context of career
choice: Meta-analysis of raee/ethnicity differences. The Career
Development Quarterly, 53, 223-233.
Henson, R., Hull, D., & Williams, C. (2010). Methodology in
our education research eulture: Toward a stronger collective
quantitative proficiency. Educational Researcher, 39, 229-240.
doi:10.3102/0013189X10365102
Herr, E. L., & Fahian, E. S. (1993). The Journal of Counseling &
Development: Its legacy and its aspirations. Journal ofCounseling
& Development, 72, 3-4.
Hsieh, P., Acee, T, Chung, W, Hsieh, Y, Kim, H., Thomas, G., . . .
Robinson, D. (2005). Is educational intervention research on
the decline? Journal of Educational Psychology, 97, 523-529.
doi: 10.1037/0022-0663.97.4.523
Jones, S., Fong, C , Torres, L., Yoo, J., Decker, M., & Robinson, D.
(2010). Productivity in educational psychology journals from
2003 to 2008. Contemporary Educational Psychology, 35, 11-16.
doi:10.1016/j.cedpsych.2009.08.001
Kessler, R., Chiu, W, Demler, O., & Walters, E. (2005). Prevalence,
severity, and comorbidity of 12-month DÄW-/F disorders in the
National Comorbidity Survey Replication. Archives of General
Psychiatry, 62, 617-627. doi:10.1001/archpsyc.62.6.617
Lapan, R. (2005). An editor's top ten wish list. Professional School
Counseling, 8, ii-iv.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer
agreement for categorical data. Biometrics, 33, 159-174.
Lundervold, D. A., & Belwood, M. F (2000). The hest kept secret in
counseling: Single case (Af= 1 ) experimental designs. Jowrna/o/
Counseling & Development, 78, 92-102.
Marotta, S. A., & Watts, R. E. (2007). An introduction to the Best
Practices section in the Journal of Counseling & Development.
Journal of Counseling & Development, 85, 491-503.
Mate, R. L., & Kelly, K. R. ( 1997). Research publication in the Jour-
nal of Mental Health Counseling as an indicator of professional
identity development. Journal of Mental Health Counseling,
19, 286-294.
Journal ofCounseling& Development • Summer 2011 • Volume 89
McGowan, A. S. (2003). New and practical sections in the Journal
of Counseling & Development: Information for the prospective
author and the readership. Journal of Counseling & Develop-
ment, 81, 387-388.
Mellin, E. A. (2009). Responding to the crisis in children's mental
health: Potential roles for the counseling profession. Journal of
Counseling & Development, 87, 501-506.
Nathan, P. (2004). The evidence base for evidence-hased mental health
treatments: Four continuing controversies. BriefTreatment and Cri-
sis Intervention, 4,243-254. doi:10.1093/brief-treatment/mhh021
Nilsson, J. E., Love, K. M., Taylor, K. J., & Slusher, A. L. (2007).
A content and sample analysis of quantitative articles published
in the Journal of Counseling & Development between 1991 and
2000. Journal of Counseling & Development, 85, 357-363.
Ray, D., Barrio Minton, C , Schottelkorb, A., & Brown, A. (2010).
Single-case design in child counseling research: Implications
for counselor education. Counselor Education and Supervision,
49, 193-208.
Robinson, D., Levin, J.,Thomas, G., Pitueh, K., & Vaughn, S. (2007).
The incidence of "causal" statements in teaching and learning
research journals. American Educational Research Journal, 44,
400-413. doi:10.3102/0002831207302174
Ruhin, A. (2008). Practitioner's guide to using researchfor evidence-
based practice. Hohoken, NJ: Wiley.
Schloss, H., & Haaga, D. (2005). Nature of the evidence base for
evidenced-based mental health care. Journal of Cognitive and
Behavioral Psychotherapies, 5, 207-216.
Scott, W. (1955). Reliability of content analysis: The case of
nominal scale coding. Public Opinion Quarterly, 17, 321-325.
doi:10.1086/266577
Servaty-Seib, H. (2004). Introduction: Perspectives on counseling
the bereaved. Journal of Mental Health Counseling, 26, 95-97.
Sexton, T. L. (1996). The relevance ofcounseling outcome research:
Current trends and practical implications. Journal of Counseling
& Development, 74, 590-600.
Shavelson, R., & Webb, N. ( 1991 ). Generalizability theory: A primer.
Newhury Park, CA: Sage.
Skowron, E., & Reinemann, D. (2005). Effeetiveness of psycho-
logical interventions for child maltreatment: A meta-analysis.
Psychotherapy: Theory, Research, Practice, Training, 42, 52-71.
doi: 10.1037/0033-3204.42.1.52
Whimby, A., Vaughan, G. M., & Tatsuoka, M. M. (1967). Fixed ef-
fects vs random effects: Estimating variance components from
mean squares. Perceptual and Motor Skills, 25, 668.
Wood, N., Crane, D., Sehaalje, B., & Law, D. (2005). What works for
whom: A meta-analytic review of marital and couples therapy in
reference to marital distress. American Journal of Pamily Therapy,
33, 273-287. doi:10.1080/01926180590962147
359
Copyright of Journal of Counseling & Development is the property of American Counseling Association and its
content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for individual use.