Journal of Behavior Therapy

and Experimental Psychiatry 30 (1999) 1}14

Evidence-based clinical behavior analysis,

evidence-based medicine and the Cochrane
collaboration1
Eileen Gambrill*
School of Social Welfare, 120 Haviland Hall ~ MC 7400, University of California at Berkeley, Berkeley,
CA 94720-7400, USA

Abstract
Encouraging professionals in training and later to consider practice-related research "ndings
when making important clinical decisions is an on-going concern. Evidenced-Based Medicine
(EBM) and the Cochrane Collaboration (CC) provide a source of tools and ideas for doing so,
as well as a roster of colleagues who share this interest. Evidenced-based medicine involves
integrating clinical expertise with the best available external evidence from systematic research
as well as considering the values and expectations of patients/clients. Advantage can be taken of
educational formats developed in EBM, such as problem-based learning and critical-appraisal
workshops in which participants learn how to ask key answerable questions related to
important clinical practice questions (e.g., regarding e!ectiveness, accuracy of assessment
measures, prediction, prevention, and quality of clinical practice guidelines) and to access and
critically appraise related research. The Cochrane Collaboration is a world-wide network of
centers that prepare, maintain, and disseminate high-quality systematic reviews on the e$cacy
of healthcare. These databases allow access to evidence related to clinical practice decisions.
Forging reciprocal working relationships with those involved in EBM reciprocal and the CC
should contribute to the pursuit of shared goals such as basing clinical decisions on the
best-available evidence and involving clients as informed consumers.  1999 Elsevier Science
Ltd. All rights reserved.
Keywords: Evidenced-based practice; Cochrane Collaboration; informed choice; evidencedbased medicine; clinical behavior analysis

 Based on paper presented at the 16th Annual Conference of the Northern California Association for
Behavior Analysis, Oakland, CA. 29 January 1998.
* Tel.: 001 510 642 4450; Fax: 001 510 643 6126; E-mail: gambrill@uclink4.berkeley.edu.
0005-7916/99/$ } see front matter  1999 Elsevier Science Ltd. All rights reserved.
PII: S 00 0 5-7 9 16 ( 9 8) 0 0 03 5 - 4

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

Evidence-based practice (EBP) has had an uphill battle in the helping professions
including medicine and dentistry (see for example, Chalmers, 1983; Jarvis, 1990).
Consider the experience of Ignaz Semmelweiss who, around 1840, discovered that the
death rate of mothers from childbed fever markedly decreased if surgeons washed
their hands before delivering babies. Cleanliness was not taken seriously by the
medical profession until the end of the century. As McConnell (1990) notes &&Individual change can occur rapidly; social change typically moves with glacial speed''
(p. 148). Skinner has also addressed this concern (see for example, Skinner, 1981). Baer
(1998) recently incisively discussed contingencies that pose an obstacle to careful
program evaluation.
The purposes of this paper are twofold: (1) to examine what can be learned from
evidence-based medicine (EBM) and the Cochrane Collaboration (CC) about how to
reduce the research-practice gap; and (2) to encourage the cultivation of working
relationships with those involved in evidence-based medicine and the Cochrane
Collaboration. The key goal of clinical behavior analysis as well as EBM and the CC
is to closely link research and practice in relation to professional education and
everyday on-the-job decisions. All three enterprises represent a scienti"c approach
in which claims are rigorously tested. Forming such ties would have the following advantages: (1) provide additional encouragement for rigorous testing of
practice-related claims (e.g., regarding e!ectiveness, validity of assessment measures);
(2) provide additional routes for disseminating practice-related research "ndings;
(3) o!er opportunities for collaboration; and (4) increase opportunities for doing more
good than harm. Harms resulting from premature di!usion of untested methods are
amply illustrated in the histories of the helping professions (see for example, Jacobson,
Mulick and Schwartz, 1995; Silverman, 1997).

1. Evidence-based medicine (EBM)

&&Evidence-based medicine (EBM) is the conscientious, explicit, and judicious use of
current best evidence in making decisions about the care of individual patients''
(Sackett, Richardson, Rosenberg and Haynes, 1997, p. 2) (see also Gray, 1997; Sackett,
Rosenberg, Gray, Haynes and Richardson, 1996; Warren and Mosteller, 1993). It
involves integrating clinical expertise with the best available external evidence from
systematic research as well as considering the values and expectations of clients. It
&&promotes the collection, interpretation, and integration of valid, important and
applicable patient-reported, clinician-observed, and research-derived evidence. The
best available evidence moderated by patient circumstances and preferences, is applied to improve the quality of clinician judgements'' (McKibbon et al., 1995). The
kind of evidence needed to answer a question depends on the question. If the question
is one of service e!ectiveness, single-case experimental studies and/or randomized
control trials are essential. Answering other kinds of practice-related questions such as
&&What's the accuracy of this assessment measure?'' require other research methods.
The likelihood that a research method used can answer a question posed can be rated
on a scale ranging from 1 (none) to 6 (ideal) (see later discussion).

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

Steps in evidence-based practice include the following:

1. Convert information needs into answerable questions related to decisions that
must be made and identify &&educational prescriptions'' that help to answer these
questions.
2. Track down with maximum e$ciency the best evidence with which to answer key
questions.
3. Critically appraise that evidence for its validity and usefulness.
4. Apply the results of this appraisal (does the evidence apply to this client?).
5. Evaluate the outcome.
Evidence-based medicine is viewed as &&replacing the traditional medical paradigm
based on authority.2 There is also an emphasis on the dissemination of information,
as well as its collection, so that the evidence can reach clinical practice'' (Evidence
based medicine website, 8/30/97). Sackett and his co-authors (1997) suggest "ve
reasons in favor of EBP: (1) new types of evidence are being generated which can
increase our ability to help clients; (2) although it is clear that we often need this
evidence daily we usually do not get it; (3) as a result of the foregoing both our
up-to-date knowledge and our practice performance deteriorate with time; (4) attempts to overcome these de"ciencies via traditional continuing education programs
do not improve performance; and (5) a new approach to learning had been shown to
keep helpers up to date. The Center for Evidence-Based Medicine is located in
Oxford, England. A Center for Evidence-Based Social Services (see reference) has
recently been formed at the University of Exeter, England (see also Critical Appraisal
Skills Program, Bandolier, and Evidence-Based Medicine websites).
1.1. Evidence-based medicine is problem-focused
Problem-based learning has been actively explored in medical education for over 25
years (see for example, Albanese and Mitchell, 1993; Barrows, 1994; Berkson, 1993;
Chen, Cowdray, Kingsland and Ostwald, 1994; Nooman, Schmidt and Ezzat, 1990;
Schmidt, 1989; Vernon and Blake, 1993). Advantages of problem-based learning
(PBL) are suggested in Table 1 (Gambrill, 1997). A key one is multiple opportunities
for self-directed learning drawing on external evidence related to a particular problem.
Helping students to acquire e!ective self-directed learning skills is a key aim and
students are encouraged to take advantage of scienti"c knowledge in making practice
decisions. The goal is to produce practitioners who are e+ective, e.cient, and humane
in addressing clients' concerns and who learn in the process (Barrows, 1994, p. 5).
Barrows suggests that there is no need to identify possession of a certain kind of
knowledge as an objective because this is required to attain the three aims described.
Students are assigned to groups of 5}7. Each group is given a tutor. Education is
problem-focused from the very beginning of medical school. Students are encouraged
to come up with di!erent possible causes of a problem and to identify what information they need to choose among di!erent views and what particular underlying
processes are related to the problem (e.g, physiological). This links problem solving to
basic subjects such as physiology.

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

Table 1
Advantages of problem-based learning (Gambrill, 1997)
1. Encourage the integration of problem-related knowledge from the basic sciences (e.g., forge closer links
between diverse curricular areas such as research, "eld work, practice courses, policy, human growth
and behavior, and ethics).
2. Minimize irrelevant activities/discussion by grounding education on real-life problems of concern to
patients/clients and signi"cant others.
3. Emphasize the value of functional knowledge (knowledge that maximizes the likelihood of making
informed decisions).
4. Encourage development of e!ective, e$cient access skills for discovering problem-related research
"ndings.
5. Emphasize the importance of critically appraising practice-related research and enhance critical
appraisal skills.
6. Encourage evidence-based ethical practice by highlighting how ethical principles apply to particular
cases (e.g., regarding informed consent).
7. Provide multiple opportunities to practice an e!ective, e$cient problem-solving process and to
demonstrate how general principles apply to speci"c situations.
8. Emphasize the importance of thinking carefully about the allocation of scarce resources (e.g., what are
the opportunity costs of a decision about service provision?).
9. Estimate e!ectiveness of services in relation to the degree to which valued outcomes are achieved.
10. Encourage a candid, accurate appraisal of the current potential for resolving a given problem.
11. Increase the likelihood of fully informing clients/patients since helpers are informed by combining their
clinical expertise with problem-related research "ndings.
12. Encourage development of life-long, self-directed learning skills.
13. Help students to learn how to deal constructively with failure and uncertainty.

Problem-based learning is designed to take advantage of conditions that facilitate

learning. The following seven steps are involved (Schmidt, 1989). Step 1 involves
clari"cation of terms and concepts not readily comprehensive. Step 2 involves de"ning
the problem. This results in ideas and guesses about the structure of the problem. In
Step 3 (analyze the problem) students share opinions, knowledge, and ideas about
underlying processes and mechanisms. They are encouraged to share ideas freely
before carefully reviewing them (i.e., brainstorming). Step 4 involves preparing a systematic inventory of explanations collected in Step 3. Step 5 consists of identifying
learning goals.
Notice how student learning is built into this framework. Students are asked to
identify clear learning objectives (called &&educational prescriptions'') related to each
problem addressed. These relate to questions raised in the problem analysis phase
(e.g., to gain more knowledge of the processes related to a problem) (e.g., physiological
reactions). It is in this step that students relate problems to basic curricular domains.
For example, students could receive scores of learning opportunities over their
educational careers in asking: What do we know about behavior and how it is
developed and maintained that can be of value with this problem? A key concern in
this stage is to identify learning resources that might supply needed information. For
example, a problem may require an understanding of certain developmental trajectories. Decisions are then made about objectives to focus on, and, if necessary, students
agree on task distribution. In Step 6 students collect additional information outside

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

the group. Step 7 consists of sharing what has been found, synthesizing and testing the
newly acquired information, and supplementing this knowledge as needed and correcting it as necessary. Students recircle back to earlier steps as needed. Reviews of the
e!ectiveness of PBL di!er in their estimates of its e!ectiveness compared to traditional educational programs. (See for example, Albanese and Mitchell, 1993; Berkson,
1993; Vernon and Blake, 1993).

1.2. What can be learned from EBM

We can learn that we are not alone in our struggle to encourage professionals
to draw on practice-related research and to guide clinical decisions by ongoing,
accurate, problem-related data. For example, aims and activities of the Center for
Evidence-Based Dentistry can be seen in Table 2. We can add to our skills in using

Table 2
Aims and activities of the center for evidence-based dentistry, Institute of Health Services, Oxford
Aims of the Centre for Evidence-based Dentistry
To foster a spirit of enquiry within the dental team.
To encourage and enable the dental profession to make clinical decisions based on the best
available evidence.
To enable the dental team to "nd valid and up to date evidence on which to base treatment
decisions.
To assist in the development of better methods of disseminating quality evidence.
To encourage the development of evidence based professional and clinical standards and an
evidence based approach to clinical audit.
To facilitate evidence based continuing professional development (CPD) involving the whole
dental team.
To improve the public pro"le of the dental profession by moving towards evidence based
practice.
To assist in the production of clear evidence based patient information on dental treatments
and procedures.
To enable evidence based purchasing of materials and equipment.
Activities
Raising the pro,le of Evidence-based Dentistry: Papers outlining Evidence-based Dentistry as well as
workshops and lectures has raised the issue of Evidence-based Dentistry.
O+ering critical Appraisal Skills =orkshops: These were run initially with the assistance of the
Oxford based CASP (Critical Appraisal Skills Programme).
O+ering Finding the Evidence =orkshops: The Center is developing this with the CASP and
the Health Care Libraries Unit in Oxford.
Developing eaching Materials: The Evidence-based Dentistry Board have adapted and
developed CASP and evidence-based medicine materials for dentistry. A Evidence based
Workbook for dentists is being prepared.
Other: The Center is linked with Getting Research into Practice (GRiPP) and the PRISE
project concerned with information sharing in Primary Care.
Source: Evidence-based website, 1/14/91.

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

problem-focused learning in professional training programs. We can prepare user

friendly books such as EBM (1997) by Sackett and his colleagues which is in its "fth
printing. Appealing features of this book include: (1) size (convenient for slipping in
a pocket), (2) clear engaging writing, (3) use of icons, (4) engaging use of terms such as
&&educational prescription'' (referring to what students must "nd out to problem solve),
and (5) helpful lists describing teaching tips. Professional behavioral associations
could take a more active role in producing books such as this pertaining to behavioral
principles and their application. We can draw on related work in EBM to increase
options for connecting policy and practice (see for example, Gray, 1997) and for
helping practitioners to acquire and use self-directed, life-long learning skills via
participating in critical appraisal workshops (see also sources such as Gibbs
and Gambrill, 1999). Methods to encourage life-long self-directed learning skills,
life-long openness to peer review of practice decisions, and learning from mistakes can
be tested.
Medicine has taken the lead in encouraging identi"cation of adverse incidents (e.g.,
accidents) and errors (see for example, Bogner, 1994; Brennan et al., 1991; Ennis
and Vincent, 1990; Leape et al., 1991; McIntyre and Popper, 1983; Reason, 1995;
Vincent, 1989) and designing risk assessment methods based on the detailed analysis
needed to plan how to minimize avoidable mistakes in the future (see for example,
Vincent and Bark, 1995). There is a candid discussion of incompetent professionals
(see for example, Rosenthal, 1995) and reminders of harms caused by premature
di!usion of untested methods such as the blinding of about 10,000 babies between
1942 and 1954 by liberal use of supplemental oxygen (Silverman, 1997). We can test
the e!ectiveness of PBL in encouraging professionals to keep track of mistakes
(avoidable and unavoidable) to plan how to minimize them and encourage the design
of procedural audits (see for example, Allsop and Mulcahy, 1996; Kogan and Redfern,
1995).
Medicine has also taken a leading role in producing books that describe the
&&thoughts'' of practitioners as well as their actions as they problem-solve (see
for example, Kassirer and Kopelman, 1991). Comparisons are included between
novices and experts. Another lead we can take from medicine is to candidly discuss
limited resources and the importance of considering populations as well as individuals
(see for example, Eddy, 1996; Gray, 1997). Eddy argues that no matter what economic
system of health care services we have, resources will be limited. Hard choices will
have to be made and should be carefully and explicitly considered.
Some may think that medicine di!ers too much from other helping professions
such as psychology, social work, or counseling to make valuable comparisons.
A key advantage in medicine is the availability of signs (e.g., blood pressure) as
well as symptoms (e.g., feeling anxious). However, the similarities outweigh the
di!erences. Consider uncertainty. Medical experts argue that the typical physician
works in an atmosphere of uncertainty. Clinical behavior analysts use data to
guide decisions. This is one way in which clinical behavior analysis is evidence based.
We can ask for example: &&What is the evidence that a neutralizing activity will
decrease the rate of problem behaviors?'' and investigate the results (Horner, Day and
Day, 1997).

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

2. The Cochrane collaboration

Evidence-based medicine is closely tied to the Cochrane Collaboration (CC). The
CC is a world-wide network of centers designed to prepare, maintain and disseminate
high-quality systematic reviews of the e!ects of health care (Bero and Rennie, 1995;
Chalmers, Sackett and Silagy, 1997). It was founded in 1993 in Oxford, England by
about 80 people from a variety of countries. Organizational units of the CC include
Cochrane Centers (nine around the world) and collaborative review groups. Reviews
are entered on the Cochrane Database of Systematic reviews. This database is
available by subscription. At this point, of course, it is still incomplete in relation to
questions addressed. The UK Cochrane Center is part of the National Health Service
R & D's information systems strategy. Cochrane (1972) published E+ectiveness and
e.ciency, drawing attention to our ignorance about health care and suggested
randomized controlled trials (RCTs) as a guide to sound use of resources. The CC
evolved in response to his emphasis on systematic, up-to-date, critical reviews of all
RCTs in health care. The British government asked Cochrane to review medicine and
to estimate the extent to which di!erent areas were evidence based. In general,
Cochrane reviews are based on the results of randomized controlled trials, but
information derived from research using other research methods is used when appropriate. Reviews are updated regularly and electronically disseminated for public and
professional use. The Cochrane Collaboration starts with individuals * health
professionals, consumers and researchers * with interests in a particular health
problem or group of problems. A question might be: What is the best treatment for
stroke and how might we reduce the risk of its reoccurrence? How can people who
want to give up smoking be helped? Search methods and criteria for critically
appraising studies are clearly described.
The Cochrane Collaboration's potential derives from its commitment to prepare
and maintain reviews of research evidence which address questions of relevance to
people using the health services; its use of transparent methods in attempts to
minimize biases; and its openness to challenge. As a scienti"c enterprise it has at
least two features which are rare if not unique. Firstly, the protocols of Cochrane
reviews (that is, information about the Collaboration's &&research in progress'') are
routinely made available for public scrutiny and comment. Secondly, the Collaboration has established a system for incorporating new evidence in systematic
reviews prospectively, and improving or correcting them when ways of doing so are
identi"ed. . . '' (Chalmers et al., 1997, pp. 236}237).
There are a number of Cochrane Centers throughout the world that try to ensure
that people who have registered an interest with the Collaboration are placed in
contact with other like-minded individuals. The aim of review groups is to prepare
de"nitive reviews based on randomized controlled trials in relation to a particular
treatment. An intensive e!ort is made to locate all randomized controlled trials,
published and unpublished in all languages, worldwide, related to a question using
hand searches. Cochrane Collaboration reviews succeed in "nding 60% more

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

randomized controlled trials than are published on Medline. This is an astounding

di!erence. &&The shared responsibilities of the Cochrane centers include:
E Maintaining a register for people contributing to the Cochrane Collaboration, with
information about their individual responsibilities.
E Maintaining a register of people who have expressed interest in contributing to the
Cochrane Collaboration, with information about their speci"c interests.
E Maintaining a register of published reports of systematic reviews of the e!ects of
health care, so that the Collaboration can build on existing achievements.
E Helping to establish collaborative review groups, by fostering international collaboration among people with similar interests, participating in exploratory
discussions and meetings, helping to organize workshops, and in other ways
facilitating collaboration.
E Maintaining a register of systematic reviews currently being prepared or planned
by collaborative review groups, so that unnecessary duplication of e!ort can be
minimized and collaboration promoted.
E Coordinating the Collaboration's contributions to the creation and maintenance of
an international register of completed and ongoing RCTs, thus facilitating the "rst
phase of data collection for reviewers.
E Preparing and developing protocols and software * compiled in successive editions of the Collaboration's &&Tool Kit'' * to systematize and facilitate the preparation and updating of systematic reviews.
E Making arrangements for e$cient electronic transfer of reviews between reviewers
and editors, between editors and the Cochrane Database of Systematic Reviews,
and between the Cochrane Database management system and electronic dissemination media.
E Developing policies and setting standards to maximize the reliability of information
disseminated through the Cochrane Database of Systematic Reviews.
E Promoting and undertaking research to improve the quality of systematic reviews.
E Exploring ways of helping the public, health service providers and purchasers,
policy makers and the press to make full use of Cochrane Reviews.
E Organizing workshops, seminars and colloquia to support and guide the development of the Cochrane Collaboration''. (The Cochrane Collaboration, updated.)
Software developed to expedite e!orts include Review Manager.
Cochrane Review Groups (CGRs) make particular use of two software packages.
The "rst * Review Manager * provides both an organizational and analytic
framework for assembling Cochrane Reviews in electronic format; the second
* Module Manager * enables administrators and editors to assemble up-to-date
reviews prepared by the members of their CRG, as well as information about the
CRG itself, for example, the scope of its work, and the strategy used to assemble
a specialized register of relevant studies.
This collection of Cochrane Reviews and information about the CRG, known as
a module, is the CRG's principal contribution to the Cochrane Collaboration.
Together with modules from all the other CRGs, it contributes to the Parent

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

Database of Cochrane Reviews. It is from this continuously updated Parent

Database that Cochrane Reviews (and titles and protocols of those reviews that are
in the process of being prepared) are disseminated.
Dissemination takes place through a regularly updated electronic journal called
The Cochrane Database of Systematic Reviews. Research review protocols are also
available on the Cochrane Database.
2.1. What can be learned from the Cochrane Collaboration
Behavioral clinicians can learn how to set up international review groups in relation
to a question and discover the importance of conducting intensive hand searches of
published and unpublished material in all languages. Behavioral clinical organizations
could set up their own review groups or, better yet, could nurture ties with the
Cochrane Collaboration and participate in CC review groups. The latter option seems
best, given the head start the Cochrane Collaboration has in preparing review
protocols, designing procedures for facilitating the work of review groups, and
establishing on-line electronic communication systems. We can also learn from the
Cochrane Collaboration how to get research "ndings into the hands of consumers
that will allow them to make informed choices. The two volume book describing
research related to obstetrical care prepared as a result of Archie Cochrane's review
had little e!ect on the everyday practice of gynecologists. Enkin and his colleagues
then prepared a book for laypeople entitled A guide to e+ective care in pregnancy and
childbirth (1995). This describes the following:
(1) Bene"cial forms of care-e!ectiveness demonstrated by clear evidence from controlled trials.
(2) Forms of care likely to be bene"cial. (The evidence in favor of these forms of care is
not as "rmly established as for those in category 1.)
(3) Forms of care with a trade-o! between bene"cial and adverse e!ects. (Women and
caregivers should weigh these e!ects according to individual circumstances and
priorities.)
(4) Forms of care of unknown e!ectiveness. (There are insu$cient or inadequate
quality data upon which to base a recommendation for practice.)
(5) Forms of care unlikely to be bene"cial. (The evidence against these forms of care is
not as "rmly established as for those in category 6.)
(6) Forms of care likely to be ine!ective or harmful. (Ine!ectiveness or harm demonstrated by clear evidence.)
This book has been popular and is in its second edition. Behavioral clinical might
prepare similar books based on critical reviews of the literature in relation to key areas
of concern describing what has been tested to what e!ect and what has not been
tested. Examples that come to mind include: (1) behavioral disabilities; (2) psychiatric
rehabilitation programs; and (3) parent training. Such books will help both
providers and consumers of professional services to be more informed, for example by

10

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

encouraging them to ask questions such as, &&Are there any harmful side e!ects of this
service?''
A notable feature of both EBP and the Cochrane Collaboration is the concern for
client interests and the involvement of consumers of health services as active participants in health-related decision-making processes. Consumers have access to CC
internet communication networks via which they can raise questions and give comments. Attention to the values and expectations of clients is one of the key hallmarks
of evidence-based medicine (Sackett et al., 1997). This is also re#ected in the attention
given to the development of accessible, accountable, complaint procedures (see for
example, he citizen's charter (1991) and Complaints do matter (1994)). Evidenceinformed patient choice (EIPC) entails three criteria: (1) the decision involves which
health care intervention or care pattern a person will or will not receive; (2) the person
is gliven research-based information about e!ectiveness (likely outcomes, risks and
bene"ts) of at least two alternatives (which may include the option of doing nothing);
and (3) the person provides input into the decision-making process (Entwistle,
Sheldon, Sowden and Watt, 1998). A concern for involving clients in making decisions
that a!ect their lives highlights the importance of informed (in contrast to uninformed
or misinformed) consent. This concern is also emphasized in applied behavior analysis
in its attention to social validity * the acceptability of goals, methods and outcomes to
clients and signi"cant others (see Baer, Wolf and Riley, 1968; Schwartz and Baer, 1991).
Another means of involving clients is including them in the critical appraisal of research
reports including reviews of research "ndings in a given area. Consumer involvement
may reveal that outcomes of concern to clients are not addressed (Oliver, 1997).
3. What clinical behavior analysts can o4er
We can o!er support and help, including empirically based principles of behavior
that can guide the analysis of contingencies that in#uence the use of evidence in
everyday practice. We can o!er guidelines for conducting descriptive and functional
analyses regarding problem-related contingencies (including those related to encouraging practitioners to consider practice-related research when making decisions)
and for gathering on-going data regarding outcome and social validity (see for
example, Carr, Levin, McConnachie, Carlson, Kemp and Smith, 1994; Goldiamond,
1984; Malott, 1994; Schwartz and Baer, 1991; Van Houten and Axelrod, 1993). Thus
the suggested collaboration should bene"t all involved, especially clients. We can o!er
an echoing voice encouraging accurate rather than in#ated claims regarding what
claims have been rigorously tested to what e!ect. As the term &&evidence-based''
becomes more popular, it is likely to be misused more often; i.e., methods or reviews
that are not evidence-based will be described as evidence-based. Readers of research
reviews will have to be discerning in regard to the rigor of critical appraisal of research
reports and accuracy of related claims (e.g., of e!ectiveness). Reviewers should clearly
describe how they searched, what they searched for, and what criteria were used in the
critical appraisal of studies found (Oxman and Guyatt, 1993). A justi"cation point of
view (seeking support for a claim) in contrast to a falsi"cation approach (seeking to
falsify guesses about what may be true by critical tests, Popper, 1972; Gomory, 1997),

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

11

encourages in#ated claims about &&what works'' that may result in considerable harm
to clients (see for example, Silverman, 1997, 1998).
In summary, we can join our compatriots in other disciplines and professions in
striving to involve clients as informed participants in decisions that a!ect their lives
and to close the research-clinical practice gap. (See Table 3.) We can speak out against
editorial censorship of unpopular views (see for example, Lang, 1998). We can make
transparent our review methods, e.g., by clearly describing how we searched for
material regarding a question, what criteria were used to review material found, and
what synthesis methods were used to combine "ndings of research studies.

Table 3
Recommendations
1. Form collaborative ties with the Cochrane Collaboration (CC) and those in evidence-based medicine.
For example join and/or initiate review groups in collaboration with the Cochrane Collaboration;
participate in International Cochrane Colloquium.
2. Subscribe to the Cochrane Collaboration database.
3. O!er user-friendly workshops on the critical appraisal of research * how to critically appraise and use
evidence in decisions about health care (see for example Critical Appraisal Skill Program (CASP) home
page. P.O. Box 777, Oxford OX3 7LF. Fax: 01865 22 6959. E-mail: cspittlehouse@casp.phru.org.
Website: http://www.ihs.op.dc.uk/cdsp/.
4. Experiment with the e!ectiveness of problem-based learning in professional education (See for example,
Barrows, 1994).
5. Continue to design and implement procedural audit review procedures for all areas of practice. Develop
risk audit methods focusing on providing e!ective services.
6. Design methods for identifying mistakes and carefully analyze their context to identify related factors,
both latent and active toward the goal of minimizing mistakes (see Reason, 1995; Vincent and Bark,
1995). Draw on behavioral principles to identify contributors to adverse events and mistakes.
7. Continue to search for ways to increase accessibility of practice-related research "ndings to both
professionals and clients.
8. Prepare books for consumers in all problem areas describing what methods have been tested to what
e!ect (i.e., found to be helpful, ine!ective, or harmful) and o!er critical appraisal workshops to users of
services.
9. Prepare a user-friendly search system akin to the &&grateful med'' that is accessible to both professionals
and clients.
10. Include clients in the critical appraisal of research studies and reviews of studies.
11. View complaints as opportunities to improve the quality of services and develop timely, easy-to-use,
accountable, complaint procedures to harvest complaints and to learn from them how to improve the
e!ectiveness and e$ciency of services.
12. Use a falsi"cation approach to the investigation of claims in contrast to justi"cation approach which
encourages in#ated claims about what does more good than harm.
13. Participate in the preparation of high quality systematic critical reviews of research regarding speci"c
questions which avoid in#ated claims about what is known and which clearly describe search, appraisal
and synthesis methods used and arrange for the routine updating of reviews.

12

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

References
Albanese, M. A., & Mitchell, S. (1993). Problem-based learning: A review of literature on its outcomes and
implementation issues. Academic Medicine, 68(1), 52}81.
Allsop, J., & Mulcahy, L. (1996). Regulating medical work: Formal and informal controls. Buckingham: Open
University Press.
Baer, D. M. (1998). Program evaluation: Arduous, impossible, or political? Paper presented at the 16th
Annual Conference of the Northern California Association for Behavior Analysis, Oakland, CA.
Barrows, H. S. (1994). Practice-based learning: Problem-based learning applied to medical education. Spring"eld, IL: Southern Illinois University School of Medicine.
Berkson, L. (1993). Problem-based learning: Have the expectations been met? Academic Medicine, 68(10),
579}587.
Bero, L., & Rennie, D. (1995). The Cochrane Collaboration. Preparing, maintaining, and disseminating
systematic reviews of the e!ects of health care. Journal of the American Medical Association, 274(24),
1935}1938.
Bogner, M. S. (Ed.) (1994). Human error in medicine. Hilllsdale, NJ: Erlbaum.
Brennan, T. A., Leape, L. L., Laird, N. M., Herbert, L., Localio, A. R., Lawthers, A. G., Newhouse, J. P.,
Weiler, P. C., & Hiatt, H. H. (1991). Incidence of adverse events and negligence in hospitalized
patients * Results of the Harvard Medical Practice Study I. he New England Journal of Medicine, 324,
370}376.
Carr, E. G., Levin, L., McConnachie, G., Carlson, J. I., Kemp, D. C., & Smith, C. E. (1994). Communicationbased intervention for problem behavior: A user's guide for producing positive change. Baltimore: Paul H.
Brookes.
Center for Evidence-Based Social Services. Armory Bldg., University of Exeter, England, EX4 4RJ. Fax:
01392 263324. E-mail: S.E.Bosley@exeter.ac.uk. Director: Brian Sheldon.
Chalmers, I. (1983). Scienti"c inquiry and authoritarianism in perinatal care and education. Birth, 10(3),
151}155.
Chalmers, I., Sackett, D., & Silagy, C. (1997). The Cochrane Collaboration. In A. Maynard, & I. Chalmers
(Eds.), Non-random re-ections on health services research: On the 25th anniversary of Archie Cochrane's
e+ectiveness and e.ciency (pp. 231}249). London: BMJ.
Chen, S. E., Cowdray, R., Kingsland, A., & Ostwald, M. (Eds.) (1994). Re-ections on problem-based learning.
Sydney, Australia: Wild & Wooley.
Cochrane, A. (1972). E+ectiveness and e.ciency: Random re-ections on health services. London: Nu$eld
Provincial Hospitals trust. (See also next reference.)
UK Cochrane Centre, NHS R & D Program, Summertown Pavilion, Middle Way, Oxford, OX2 7LG,
U.K. E-mail address: general@cochrane.co.uk. Fax: #44 - (0) 1865-516311, Telephone: #44 - (0) 1865
- 516300.
Complaints do matter (1994). A consultive paper on future NHS [National Health Service] complaints
arrangements. (1994). National Association of Health Authorities, NAHAT, 26 Chapter St. London.
SW1P 4ND.
Eddy, D. M. (1996). Clinical decision making: From theory to practice. Sudbury, MA: Jones & Bartlett
Publishers.
Enkin, M., Keirse, M. J. N., Renfrew, M., & Neilson, J. (1995). A guide to e+ective care in pregnancy and
childbirth (2nd ed.). New York: Oxford University Press.
Ennis, M., & Vincent, C. A. (1990). Obstetric accidents: a review of 64 cases. British Medical Journal, 300,
1365}1367.
Evidence-Based Medicine Website. (See for example Learning Resources Center. Tool-kit.)
Gambrill, E. (1997). Social work practice: A critical thinker's guide. New York: Oxford University Press.
Gibbs, L., & Gambrill, E. (1999). Critical thinking for social workers: Exercises for the helping professions
(2nd ed.). Thousand Oaks, CA: Pine Forge Press.
Goldiamond, I. (1984). Training parent trainers and ethicists in nonlinear analysis of behavior. In R. F.
Dangel, & R. A. Polster (Eds.), Parent training: Foundations of research and practice (pp. 504}546). New
York: Guilford.

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

13

Gomory, T. (1997). Social work and philosophy. In M. Reisch, & E. Gambrill (Eds.), Social work in the 21st
century (pp. 300}310). Thousand Oaks, CA: Pine Forge Press.
Gray, J. A. M. (1997). Evidence-based health care: How to make health policy and management decisions. New
York: Churchill Livingstone.
Horner, R. H., Day, M. M., & Day, J. R. (1997). Using neutralizing routines to reduce problem behaviors.
Journal of Applied Behavior Analysis, 30, 601}614.
Jacobson, J. W., Mulick, J. A., & Schwartz, A. A. (1995). A history of facilitated communication. American
Psychologist, 50(9), 750}765.
Jarvis, W. T. (1990). Dubious dentistry: a dental continuing education course. Loma Linda University, School
of Dentistry, Loma Linda, CA 92350.
Kassirer, J. P., & Kopelman, R. I. (1991). earning clinical reasoning. Baltimore: Williams and Wilkins.
Kogan, M., & Redfern, J. C. (1995). Making use of clinical audit: a guide to practice in the health professions.
Philadelphia: Open University Press.
Lang, S. (1998). Challenges. New York: Springer-Verlag.
Leape, L. L., Brennan, T. A., Laird, N., Lawthers, A. G., Localio, A. R., Barnes, B. A., Hebert, L.,
Newhouse, J. P., Weiler, P. C., & Hiatt, H. (1991). The nature of adverse events in hospitalized
patients * Results of the Harvard Medical Practice II. he New England Journal of Medicine, 324,
377}383.
Malott, R. W. (1994). Rule-governed behavior, self-management, performance management. Kalamazoo, MI:
Western Michigan University, Department of Psychology.
McConnell, J. (1990). Bias in outcome reviews. he Behavior herapist, 13(7), 148}150.
McIntyre, N., & Popper, K. (1983). The critical attitude in medicine: The need for a new ethic. British
Medical Journal, 287, 1919}1923.
McKibbon, K. A. et al. (1995). The medical literature as a resource for evidence-based care. Working Paper
from the Health Information Resources Unit, McMaster University, Ontario, Canada.
Nooman, Z. M., Schmidt, H. G., & Ezzat, E. S. (1990). Innovation in medical education: An evaluation of its
present status. New York: Springer.
Oxman, A. D., & Guyatt, G. H. (1995). The science of reviewing research. In K. S. Warren & F. Mosteller
(Eds.), Doing more good than harm: he evaluation of health care interventions (pp. 125}133). New York:
New York Academy of Sciences.
Popper, K. R. (1972). Conjectures and refutations: he growth of scienti,c knowledge (4th ed.). London:
Routledge and Kegan Paul.
Reason, J. (1995). Understanding adverse events: Human factors. In C. Vincent (Ed.), Clinical risk management. London: BMJ Pub. Group.
Rosenthal, M. M. (1995). he incompetent doctor: Behind closed doors. Buckingham, UK: Open University
Press.
Sackett, D. L., Rosenberg, W. M., Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based
medicine: What it is and what it isn't. British Medical Journal, 312, 71}72.
Sackett, D. L., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (1997). Evidence-based medicine: How to
practice and teach EMB. New York: Churchill Livingstone.
Schmidt, H. G. (1989). The rationale behind problem-based learning. In H. G. Schmidt, M. Lipkin, Jr.,
M. W. De Vries, & J. M. Greep (Eds.), New directions for medical education: Problem-based learning and
community-oriented medical education (pp. 105}111). New York: Springer-Verlag.
Schwartz, I. S., & Baer, D. M. (1991). Social validity assessments: Is current practice state of the art? Journal
of Applied Behavior Analysis, 24, 189}204.
Silverman, W. A. (1997). Equitable distribution of the risks and bene"ts associated with medical innovations. In A. Maynard and I. Chalmers (Eds.), Non-random re-ections on health services research. London:
BMJ.
Silverman, W. A. (1998). =here's the evidence?: Controversies in modern medicine. Oxford: Oxford
University Press.
Skinner, B. F. (1981). =e happy few, but why so few? Invited address to the Association for Behavior
Analysis, Milwaukee, WI.
he citizen's charter: Raising the standard. White paper presented to Parliament, July 1991. London,
England.

14

E. Gambrill/J. Behav. Ther. & Exp. Psychiat. 30 (1999) 1}14

Van Houten, R., & Axelrod, S. (Eds.) (1993). Behavior analysis and treatment. New York: Plenum.
Vernon, D. T., & Blake, R. L. (1993). Does problem-based learning work? A meta-analysis of evaluative
research. Academic Medicine, 68, 550}563.
Vincent, C. A. (1989). Research into medical accidents: a case of negligence? British Medical Journal, 299,
1150}1153.
Vincent, C., & Bark, P. (1995). Accident investigation: Discovering why things go wrong. In C. Vincent
(Ed.), Clinical risk management. London: BMJ.
Warren, K. S., & Mosteller, F. (1993). Doing more good than harm: he evaluation of health care interventions.
(Vol. 703). New York: Academy of Science.