Methodological Challenges in Meditation Research
Opher Caspi, MD PhD, and Katharine O. Burleson, MD
Abstract
Like other complex, multijaceted interventions in medicine,
meditation represents a mixture oj specijic and not-so-specijic
elements oj therapy. However, meditation is somewhat unique
in that it is dijjicult to standardize, quantijy, and authenticate
jor a given sample oj research subjects. Thus, it is ojten chal-
Ienging to discern its specijic ejjects in order to satisfy the scien-
tific method oj causal injerences that underlies evidence-based
medicine. Therejore, it is important to consider the key method-
olo^cal challenges that ajject both the design and analysis oj
meditation research. The goal oj this paper is to review those
challenges and to ojjer some practical solutions. Among the
challenges discussed are the mismatches between questions and
desiffls, the variability in meditation types, problems associated
with meditation implementation, individual dijjerences across
meditators, and the impossibility oj double-blind, placebo-con-
trolled meditation studies. Among the design solutions ojjered
are aptitude x treatment interaction (ATI) research, mixed
quantitative-qualitative methods, and practical (pragmatic)
clinical trials. Similar issues and solutions can be applied more
generally to the entire domain oj mind-body therapies.
INTRODUCTION
Meditation in its original form is the "attuning of the
mental and physical body to its spiritual source."' It is the
process of self-regulation of attention.^ For practical pur-
poses, meditation may be construed as one specialized
form of several active relaxation techniques for strengthen-
ADVANCES Spring 2005, VOL. 2 1 , NO. 1
ing the mind, emotions, and body. Within the NIH's
National Center for Complementary and Alternative
Medicine (NCCAM) classification system, meditation is
considered part of the domain of mind-body therapies.-*
These therapies are intended to enhance health, change
symptomatic states, and promote relaxation. They include
such diverse practices as guided imagery, mindfulness-
based stress reduction (MBSR), progressive relaxation.
Transcendental Meditation (TM), and other therapies.
Like other complex, multifaceted interventions in med-
icine (eg, psychotherapy), meditation represents a mixture
of specific and not-so-specific (incidental) elements of ther-
apy •* However, meditation is somewhat unique in that it is
difficult to standardize, quantify, and authenticate for a
given sample of research subjects. Thus, it is often challeng-
ing to discern its specific effects in order to satisfy the scien-
tific method of causal inferences that underlies
evidence-based medicine. Therefore, it is important to con-
sider the key methodological challenges tbat affect both the
design and analysis of meditation research. The goal of this
paper is to review those challenges and to offer some practi-
cal solutions. Similar issues can be applied more generally
to the entire domain of mind-body therapies.
DEFINING THE RESEARCH QUESTION
Studies of meditation can be grouped into two main
categories based on their overarching goals. Every medita-
tion study should therefore be carefully designed to specif-
ically address the underlying question of interest. The
Abbreviations:
NCCAM = National Center for Complementary and
Alternative Medicine
MBSR = mindfulness-based stress reduction
TM = Transcendental Meditation
EEG = electroencephalogram
qEEG = quantitative EEG
fMRt = functional magnetic resonance imaging
PET = positron emission tomography
ATI = aptitude (or attribute) by treatment interaction
HRV = heart rate variability
RCT = randomized controlled trial
NNT = number needed to treat
PCT = practical (or pragmatic) controlled trial
Methodological Challenges in Meditation Research
classical ejjicacy-ejjectiveness model of meditation research
tests meditation against an acceptable comparator, with the
goal to assess its relative efficacy or effectiveness for a given
condition. There are different formats for efficacy studies.
One can compare meditation to placebo, or to another
treatment, or even compare two different types/durations
of meditation. These studies can take the form of a
between-groups design in which the comparison, for
example, is between experienced meditators and non-med-
itators; or within a subject design in which the comparison
is made in the same individuals between when they were
not meditating and when they were meditating. Efficacy-
effectiveness research aims at answering the question, "to
what extent does meditation work?"
The explanatory model of meditation research, on the
other hand, focuses on discovering the mechanisms
believed to underlie the putative causal relationships
between meditation and health outcomes. That is, the pur-
pose of explanatory meditation research is to determine
how meditation works, or what plausible biological mech-
anisms account for the meditation experience, rather than
whether it works. In this approach, one measures what
actually happens physiologically during meditation, or as a
result of the meditation practice. This type of research is
useful in addressing research questions of basic science
and physiological mechanisms related to meditation.
Thus, the research question has direct bearing on the
design chosen. Only by using theory as a guiding frame-
work is meditation research able to appropriately address
the choice of one design or form of intervention over oth-
ers. Yet, examples unfortunately exist in both mind-body
and meditation research for instances in which there was
an apparent mismatch between theory, the research ques-
tion, and the design chosen. The case of hypertension, for
example, illustrates this point. Mind-body therapies for
hypertension aim at enhancing cognitive and behavioral
stress-coping strategies and reducing sympathetic arousal.
Both theory and practice suggest that mind-body medi-
cine works best as part of an "integrative" package of care,
not as stand-alone therapies. Yet, most of the mind-body
research done to date in hypertension compared standard-
ized, stand-alone, mind-body interventions to conven-
tional antihypertensive medication.^ As we explain later,
the expectation that meditation (or any other mind-body
practice for that matter) as a stand-alone treatment would
prove efficacious for the entire range of severity of hyper-
tension and for all patients is unrealistic. It is, therefore,
not surprising that the sixth report of the Joint National
Committee on Prevention, Detection, Evaluation, and
Treatment of High Blood Pressure QNC 6) concluded,
"The available literature does not support the use of relax-
ation therapies for definitive therapy or prevention of
hypertension," and the JNC 7 ignored research findings
from mind-body research altogether.^ Table 1 provides
Methodological Challenges in Meditation Research
some guiding principles on how to best match the
research question, the goals of the study, and the design in
meditation research.
NOT ALL MEDITATION PRACTtCES ARE ALIKE!
In day-to-day life, meditation is thought of as a prac-
tice where any consistent effort would allegedly result in a
positive effect on health and well-being. The emphasis
within that day-to-day life context is on "just doing it," not
on a specific threshold for effect. In mindfulness medita-
tion, for example, the object is not about getting anywhere
or fixing anything, but rather on being present in a non-
judgmental way. This emphasis on non-attachment to out-
come is a radical departure from most clinical
interventions.' When meditation is studied as a therapeutic
modality, the standard research questions of "product"
quality, threshold for effect, duration of effect, and signifi-
cance of effect are all examined, even though this is in dis-
tinct contrast to day-to-day practice of meditation.
We metaphorically refer to meditation within that con-
text as a "product" in order to stress how imperative it is to
define exactly what we mean by "meditation." Just labeling
an intervention as "meditation" is not enough because it has
been shown that different forms of meditation may result in
distinctly different physiological responses." Furthermore,
since no two meditative sessions are likely to be similar, just
labeling the practice as "meditation" may represent only a
remote approximation for the infinite richness of the experi-
ence. Labels, in short, do not make treatments what they
are. Patients and subjects, we note, may each experience
meditation differently at different times, while the label
"meditation" remains the same. Therefore, investigators
must clearly define and standardize the particular medita-
tion technique used in meditation research.
To illustrate the importance of clearly defining the
meditative technique, it is useful to look at the history of
meditation research. The classic studies on meditation
were done prior to 1980. In these studies, meditation was
seen as a method to produce a state of relaxation character-
ized physiologically by decreased heart rate, respiration
rate, and oxygen consumption.' However, recent research
suggests that meditation is more than simply an "awakeful
hypometabolic integrated response."'"'^ Indeed, Shapiro"
suggested that meditation practices are of three kinds:
those that focus on the field or background perception (eg,
Zen meditation); those that focus on a specific object (eg.
Transcendental Meditation); and those that shift the focus
between the field and the object (eg, mindfulness-based
stress reduction or MBSR).
The fact that not all meditations are the same adds to
the methodological challenge of conducting scientifically
sound meditation research because study design must
reflect thoughtful consideration of key aspects of medita-
tion that currently do not have definite answers. For
ADVANCES Spring 2005, VOL. 2 1 , NO. 1
 TABLE 1 How to Match The Research Question, The Goals, and The Design in Meditation Research

Research Question Goal of Study Best Design Suggested Comparator

"Can medilalion Establishing efficacy, Randomized con- If assay sensitivity* is established—^an active control; if assay
really help these (ie, demonstrating an effect trolled trials (RCTs) sensitivity Is questionable—either a placeho control or a 3-arm
palienls?" in a controlled setting) design with both placebo and active controls

"How does medita- Establishing effectiveness Observational trials Standard of care (including a multifaceted package of care), or
tion compare with (ie, demonstrating an effect or practical (prag- other real-world altemative therapeutic options (including other
other altematives?" in a real-world setting) matic) controlled forms and protocols of meditation administered either alone or in
trials (PCTs)t combination with other treatmenis)

"Who might henefii Helping decision-makers to Aptitude by treat- Multi-arm trials that represent both different levels/forms of
from (or be harmed make the best decisions by ment interaction intervention and different levels o(a-priori explicated patient char-
by) different forms maximizing treatment safety, (ATI) research! acteristics (eg, hypnotizability or absorption)
and protocols of efficiency, and effectiveness
meditation?"

"Which type Establishing "dose-response" Randomized con- Different protocols of the same form of meditation, (eg, the same
and how much curve; finding the most trolled trials (RCTs) type of meditation with varying duration and/or frequency of
meditation should appropriate intervention or meditation). Note that because it is unknown whether subjects
we prescribe?" research protocol can truly retum to baseline, crossover designs are inappropriate.

"How does Understanding the mecha-
meditation help?" nism(s) by which medita-
tion elicits its effects
Factorial designs Depending on the most plausible rival hypothesis. The concept
in which different of "the most plausible rival hypothesis" can be explained, in shon,
"non-specific" using the following simple question: What are some other com-
constructs peting causal models or altemative explanations that can plausibly
(eg, expectancy) account for the outcome, independently of the meditation inter-
are manipulated vention? Eor example, placebo control would assess the extent to
which placebo effects might account for the observed effect.

* Assay sensitivity is a property of a clinical trial defined as the ability to distinguish an effective treatment from a less effective or ineffective treatment."
t See text for explanation of these terms.

example, it is currently unknown what session length and
frequency of a meditation practice are necessary to
achieve a result; how many weeks', months', or years'
duration of meditation practice produce a clinically mean-
ingful result; whether meditation effects are linear or
cumulative; and what the most appropriate sample size
for hypothesis testing should be. Ethical and economic
considerations influence study design to keep the size and
duration of clinical trials as small and short as possible.
Yet, we admit that we do not know whether this strategy
is appropriate for meditation research.
What is more, since different forms of meditation
seem to have different therapeutic effects, it is completely
possible that some meditators might benefit from some
forms or protocols of meditation, while others might bene-
fit from different forms or protocols of meditation, tf this
individual variability is indeed true, then another key
methodological challenge is to find what the most appro-
priate ways are to address the healthy tension between
standardizing the intervention protocol (which would sup-
port internal validity) and individuahzing it (which would
support external validity). We maintain that not only does
a need exist for more basic and clinical research comparing
the effects of each of the major meditation practices, but
also for more careful examination of how those practices
interact with individual patients' characteristics. This
knowledge would support both therapeutic and policy
decision-making at the point of care by maximizing treat-
ment safety, efficiency, and effectiveness.
DEFINING THE INTERVENTION—MEDITATION
INTEGRITY AND FIDELITY
Since studies that fail to implement an appropriate
interventional protocol may lead us to erroneous conclu-
sions about treatment effects (namely, type tt error, ie, con-
cluding that there is no effect when indeed there is one),
the importance of treatment integrity and fidelity in most
cases of clinical research needs no further elaboration. And
yet, ensuring both treatment integrity and fidelity in medi-
tation research appears to be uniquely challenging. As dis-
cussed below, this is mainly due to three factors: 1)
pragmatic—meditation is a self-practiced technique rather

ADVANCES Spring 2005, VOL. 21, NO. 1 Methodological Challenges in Meditation Research
than a provider-administered treatment; 2) conceptual—
compliance with meditation instructions does not necessar-
ily mean quality meditation; and 3) methodological—there
are currently no gold standard physiological measures that
would serve as a signature for the quality of meditation.
Meditation is a Self-administered
and Setf-practiced Technique
An important factor to consider in all meditation stud-
ies is the baseline experience or practice level of the sub-
jects in the experimental and control groups (analogous to
intercept). Some meditation studies involve teaching
novices how to meditate, while others examine the effect of
a long-term meditation practice by comparing experienced
to not-so-experienced meditators. Although it is ultimately
the subject who meditates and controls the process of med-
itation, not the facilitator, the choice of both subjects and
facilitators needs to be carefully considered in the design of
meditation studies and in their analyses. This is because the
competence of the facilitator, both as a practitioner and an
instructor, as well as the level of motivation, commitment,
determination (and perhaps "talent," see the following dis-
cussion) of subjects may have a profound impact on the
findings of these studies, tn the absence of objective ways
to verify the level of proficiency of both meditators and
instructors, it is often the case that researchers have to rely
on each candidate's self-report of "years of experience with
meditation" or lack thereof, as if meditation is unidimen-
sional and can be quantified using time units. Yet, time
spent in meditation and experience with meditation are
imperfect proxies for true competency. It is the quality of
meditation, not necessarily its quantity that may matter in
terms of health outcomes. Thus, as Dr Aiastair
Cunningham of the University of Toronto often argues, the
emphasis on the research question shifts from the medita-
tion intervention delivered by the protocol to the quality or
mastery of the meditation achieved by the subject. This
suggests that qualitative research assessments need to be
part of any clinical studies on meditation. Furthermore, a
threat to validity might exist in the form of a maturation
effect if some subjects would become "more proficient" in
the meditative practice sooner than others, a phenomenon
that would cause a curvilinearity that would need accurate
modeling (analogous to differential trajectories).^
Compliance Does Not Necessarily
Mean Meditation
Many meditation studies report data on subject com-
pliance in doing the daily practice. To the extent that sub-
ject self-reports provide reliable measures of actual
behavior (a questionable contention,'0 compliance with
meditation self-practice techniques is often relatively high.
For example, in one study of TM that used a protocol of 6
hours of instruction over a 4-day period, and then t2
Methodological Challenges in Meditation Research
weeks of self-practice of meditation for 20 minutes twice
daily, the authors reported 91% compliance based on sub-
ject diary entry. "^ Another investigation on treatment for
blood pressure had subjects use a music/breath interaction
device at home for 10 minutes daily over a period of 8
weeks. In this study, the breath interaction device kept its
own data log, and the mean compliance rate was 71%."
However, compliance rate, even when perfect, tells us
very little about the quality and durability of the medita-
tive state. Indeed, it has been suggested that there is no
evidence that MBSR actually produces a state of mindful-
ness.'" We, therefore, contend that adopting the pharma-
ceutical adherence/compliance approach to treatment
integrity and fidelity in meditation research is likely to be
misleading since compliance is, at best, a proxy for the
subject's motivation, not necessarily for "product" quality.
Lack of Objective Measures May Impair
Our Ability to Make Causal Inferences About
Meditation Effects
Despite recent advances in both neuroscience and
neuroimaging (eg, qEEG, fMRt, PET), there are currently
no reliable ways by which researchers can assess and verify
the extent to which subjects actually achieve a state of
meditation. This is mainly because many of the changes
that occur in physiological measures during meditation
(eg, heart rate variability, respiratory rate, and skin con-
ductance) are not unique to meditation. Furthermore, dif-
ferent forms of meditation can result in either change in
alpha or theta EEG."" But what those changes mean with
respect to the causal link between meditation and out-
comes is unknown.
The current absence of objective physiological mark-
ers or signatures of quality for meditation leads to, among
other things, the fact that it is hard to establish a dose-
response curve for meditation. Yet, dose-response curves
are among the most important indicators of causal rela-
tionship between interventions and outcomes. This state of
affairs makes it exceedingly difficult to distinguish between
truly negative and failed (ie, false negative or type II error)
meditation trials." Put another way, inferential conclusions
about both the efficacy and effectiveness of meditation,
either as a health promotion practice or as a treatment for
certain conditions, might be uncertain in many cases.
We, therefore, wholeheartedly agree with Scott and
Sechrest who noted, "We need to develop ways of quanti-
fying the strength of interventions so that it may be related
to outcome and so that results of different interventions
may be more usefully compared. Better, more systematic
theory is required in order to derive measures of strength.
Only if theory is incorporated at the very beginning into
planning of interventions and their evaluation can we
expect to benefit from the many advantages that good esti-
mates of the strength of intervention would bring.""
ADVANCES Spring 2005, VOL. 2 1 , NO. 1
Indeed, more theory-driven exptoratory research is needed
to examine issues related to meditation integrity and fideli-
ty in preparation for confirmatory research that would
assess meditation efficacy and effectiveness.
INDIVIDUAL DIFFERENCES ACROSS MEDITATORS
I-towever, more than just the features of the treatment
ought to be considered in the design and analysis of medi-
tation research. We also believe that individual differences
across meditators should be studied. Here we challenge the
assumption of subject homogeneity that underlies random-
ized controlled trials (RCTs). First, selection bias based on
personal preference to undertake a meditation practice and
to participate in meditation research might occur. When
selection bias occurs, it weakens the study credibility,
internal validity, and external validity Second, it is entirely
possible that the ability or predisposition for learning
and/or practicing meditation techniques varies among
individuals. Thus, the study design and analysis plans
must take into consideration any and all plausible system-
atic differences across subjects. This recommendation goes
beyond just addressing potential confounders and system-
atic differences between those who elect to meditate on a
regular basis and those who do not. It suggests that, even
among those who elect to practice meditation, some sys-
tematic differences might exist that would account for
some of the variance associated with the therapeutic effect.
Indeed, many studies assessing the cumulative effects
of meditation compared experienced meditators to
novices, or self-selected meditators to non-meditators,
tlowever, as Canter pointed out, results from many such
trials may not be generalizable to the larger population
because the ability to learn and maintain a meditation
practice have not been accounted for in the study design."
Croup means, in other words, might hide important indi-
vidual differences that have direct bearing on the decision
of whether, for whom, when, and how meditation should
be integrated into the therapeutic plan. Research on the
effect of group psychotherapy on cancer survival provides
another example of the potential for an RCT to obscure
meaningful individual responses. A recent RCT, for exam-
ple, showed no significant result in overall survival, where-
as a prospective longitudinal study showed a highly
significant relationship between psychological attributes
and survival duration.'" One potential solution to this
problem can be found in a research paradigm called apti-
tude (or attribute) by treatment interaction research, a full
discussion of which can be found in Caspi and Bell."-'°
APTITUDE BY TREATMENT INTERACTION
AND MEDITATION RESEARCH
Aptitude (or attribute) by treatment interaction
research (or ATI research) is a research paradigm that is
designed to systematically take into account individual dif-
ADVANCES Spring 2005, VOL. 2 1 , NO. 1
ferences in the process of treatment evaluation. It attempts
to examine how outcomes depend on the match or mis-
match between patients' specific characteristics and the
treatments they receive, so as to examine whether particu-
lar treatments can be chosen or adapted to optimally fit a
particular person. In the ATI paradigm, the "A" (aptitude
or attribute) stands for any individual difference variables
or patient characteristics that may moderate the effects of a
treatment (T) on an outcome (O). The "T" may represent
any type of intervention, from a whole ("integrated") thera-
py package to a specific, even single intervention. The "1"
(interaction) term is used in a statistical (not a social)
sense, referring to the moderating effect of A on the rela-
tionship between T and O."
The methodological challenge in ATI research is to
avoid the trap of inferring spurious causal relationships
between certain aptitudes and treatments. We, therefore,
advocate that plausible aptitudes should be explicated a pri-
ori and tested in a confirmatory way, rather than being
explored in post hoc fashion. As a guiding principle, we sug-
gest including as aptitudes only those variables that, on the
one hand, are most pervasive while, on the other hand, are
differentially predictive of outcomes. That is where theory
should come into play. Indeed, Karuparthy and
Vepachedu,^' for example, suggested that the negative find-
ings of a recent clinical trial could be explained by a mis-
match between patients' constitution as defined by Ayurveda
theory and the treatment prescribed, and Canter and Ernst"
used the methodology of multiple n=l trials to explore and
profile subjects' heterogeneity in response to treatment.
Two examples of markedly different theory-based apti-
tudes that are likely to be relevant to meditation research
are absorption and expectancy. Absorption refers to an
individual's proclivity toward total attentional involvement
in perceptual and imaginative experience.^"* Neff et al,^^ for
example, reported that level of absorption could predict
how patients respond to biofeedback treatment for
headache, and Shea'" found that patients low in absorption
achieved better voluntary control of heart rate with hypno-
sis, as compared with imagery. Mindfulness meditation
seems also to be related to absorption, although it does not
necessarily involve complete immersion in the experience,
in contrast to what might be expected with TM. We pro-
pose that absorption is an important aptitude to examine
in future ATI meditation research.
Expectancy, on the other hand, is an aptitude that
seems to be related to all clinical research, not just medita-
tion studies. In recent years, evidence that both the degree
and direction of expectancy are strong predictors of health
outcomes has emerged. In a study of various interventions
for back pain, Kalauokalani et al found that after adjustment
for baseline characteristics, improved function was observed
for 86% of the participants with higher expectations for the
treatment they received, as compared with 68% of those
MeLhodological Challenges in Meditalion Research
with lower expectations (P=.O1).-" Furthermore, patients
who expected greater benefit from massage than from
acupuncture were more likely to experience better outcomes
with massage than with acupuncture, and vice vei"sa
(P=.O3). tn contrast, general optimism about treatment,
divorced from any specific treatment, was not strongly asso-
ciated with outcome. The results of this study suggest that
patient expectations may specifically influence clinical out-
come through the interaction with the treatment itself
DOUBLE-BLtND PLACEBO-CONTROLLED
MEDITATION STUDIES ARE NOT EEASIBLE
Meditation research is methodologically challenging in
at least two other ways, both of which are related to strate-
gies meant to minimize the possibility of systematic biases.
While randomization serves as the major means to handle
the threats of selection bias, double-blinding has been
advocated as the major means to minimize performance
bias (ie, systematic differences in the provision of care
apart from the treatment under evaluation) and detection
bias (ie, systematic differences in the assessment of out-
come, also known as evaluation bias).^^ Research has
shown that effect-size estimates under masking conditions
(ie, when neither the subjects nor the research team know
what treatment has been administered to whom) are likely
to be more valid than in open-label studies.^' Thus, main-
taining the blind is an important feature of both efficacy-
effectiveness and explanatory research. Placebos, in that
regard, have been proposed as one of the major design
strategies to maintain the blind.
The problem is that neither true double-blinding nor
true placebo control are feasible in meditation research.
First, although the investigator or an assessor can be blind-
ed as to which subject did what, especially when the out-
come is relatively objective (eg, HbAtC), subjects in
meditation research cannot possibly be blinded to the
meditative practice.''" Thus, inferences about the specific
causal relationship between the practice of meditation and
disease outcomes are potentially confounded with such
"nonspecific" factors as subjects' expectancy, belief systems,
motivation, and suggestibility.'" Such confounders might
result in type t error (ie, concluding that there is a specific
effect of the treatment on the condition, whereas indeed
there is none).
Second, establishing a true sham or placebo-control
condition for meditation research is particularly difficult, tt
is relatively easy to match the subjects and controls for
physical activity and whether eyes during meditation
should remain open or closed. The difficulty arises in
assessing and matching the mental activity to be either
meditating or non-meditating. For example, when medita-
tion is defined as "...an intentional regulation of attention
from moment to moment,'"'^ then the question arises as to
how this is different from a common control condition
Methodological Challenges in Meditation Research
where subjects are instructed to do counting exercises.
This similarity between the two conditions is even more
apparent when a concentration form of meditation is being
studied and the meditators restrict their attention to a sin-
gle repetitive stimulus, such as a word, phrase, or visual
object. One would think that some of the basic physiologi-
cal effects of meditation, such as changes in respiration
and heart rate variability (HRV) might be good indicators
of meditation.''^ However, designing appropriate controls
and comparing outcomes with these parameters has
become even more difficult now that a recent study by
Bernardi showed that even reading silently caused changes
in the respiratory rate and affected HRV^
POTENTIAL SOLUTIONS
AND EUTURE DIRECTIONS
So what can be done to substantially enhance both
the validity and applicability of meditation research find-
ings? Earlier in this paper we discussed important design
features that we believe should be incorporated into all
future meditation studies, and by way of extrapolation,
into all mind-body research. Our recommendations thus
far included:
• Better matching among the research question, the
goals of the study, and the design chosen, especially
as related to subject eligibility criteria and the
choice of the comparator (see Tablet).
• Clear explication of the particular meditation tech-
nique to be used in the study, with due considera-
tion to issues related to standardization vs
individualization.
• Monitoring and assessment beyond the descriptive
level of compliance of the issues related to medita-
tion integrity and fidelity (ie, the quality of medita-
tion achieved as a self-practiced technique) that may
have an impact on therapeutic effects.
• Examination of individual differences across sub-
jects using the ATt approach, in order to gain
knowledge that can be used to better match individ-
uals and treatments, thereby maximizing treatment
safety, efficiency, and effectiveness.
Two other important design options that promise to
enhance the validity and applicability of meditation
research are the integration of qualitative methods into
quantitative clinical studies and the use of practical (or
pragmatic) clinical trials.
Qualitative Methods
Contrary to popular belief, qualitative methods can
produce valuable evidence.''' Hence, it has been suggested
that the insistence on RCTs as the sole source of evidence
for efficacy is probably too limiting.'"* Whereas in the past
ADVANCES Spring 2005, VOL. 2 1 , NO. 1
 quantitative and qualitative methods were considered
mutually exclusive, the two have been combined more
recently to achieve more robust methodology."" The added
qualitative layer sheds light on what is happening behind
the scenes of the study, enriches the discussion around the
study findings, may affect causal inferences regarding the
relative efficacy of the intervention, and can be used to
inform and improve future research. An example of this is
the work by Cunningham, who demonstrated a correlation
between the level of involvement in self-help and cancer
survival.'"' This analysis emphasized assessments of indi-
vidual responses and subgroups, rather than the usual
homogenous approach of the randomized control trial.
We, therefore, believe that all future meditation stud-
ies should use mixed quantitative-qualitative methods. We
suggest that whether those studies focus on efficacy-effec-
tiveness or are explanatory in nature, subjects should be
1) encouraged to reflect on their experience with medita-
tion during the trial, and 2) invited to make attributions
about what in their minds might have led to any change
in their outcome as a result of participation in the study
This approach, when done appropriately, can provide par-
tial solution to two paramount methodological challenges
of meditation research. First, subjects can rate their per-
ceived "quality" of the meditation after each session. Once
validated, those ratings, which complement such proxy
variables as compliance, can then be used in the analysis
as covariates. Such analysis enhances our ability to distin-
guish between failed (type tt error) and negative trials in
situations where the study findings are not indicative of
any meaningful clinical effects. Second, it is important to
remember that what we label as simply "meditation" is, in
fact, a mixture of specific and not-so-specific elements of
therapy* It often involves a spiritual construct, changes in
breathing patterns, a relaxation response, and in most
cases a hypometabolic state. Moreover, meditation may be
confounded by elements such as group support and cog-
nitive therapy during the instruction process if the sub-
jects are taught to change their way of looking at their
illness or stress."" Therefore, even if the study findings are
suggestive of meaningful clinical effects, a question always
exists as to which of the above elements account for the
meditation's positive effects."" Here again, qualitative
methods could help. For example, if most subjects, while
reflecting on what happened to them, would indicate that
they believe the changes they have experienced during the
study are best explained by either the relaxation response,
increased self-efficacy, or social support, rather than by
the altered awareness, it would be hard to argue that med-
itation has a specific causal effect on clinical outcomes.
Indeed, a study by Lazarus and Mayne found no differ-
ence in the response to progressive relaxation, meditation,
or an attention-placebo group.^° We, therefore, suggest
including qualitative methods as a means to explore com-
1.0 ADVANCES Spring 2003, VOL. 2 1 , NO. 1
peting explanatory models for meditation effects. These
competing explanatory models could then be further
studied in future confirmatory, explanatory research so as
to avoid the traps of type I/II errors.
Practical (or Pragmatic) Clinical Trials
Despite ongoing efforts to enhance effective translation
of research findings to clinical practice and health policy,
neither explanatory research nor placebo-controlled RCTs
support decision-making at the point of care. This is
because clinicians are interested in head-to-head compar-
isons of the benefits, risks, and costs of viable altemative
clinical strategies, rather than abstract parameters such as
number needed to treat (NNT) that are based on placebo
comparisons. This is all the more true in the case of mind-
body interventions where clinicians may be interested in
the body of evidence that supports or refutes their integra-
tion into what is already a very complex healthcare system.
To begin to address this issue, Tunis, Stryer, and
Clancy have recently introduced the concept of practical
(or pragmatic) clinical trials (PCT).'' PCTs are clinical tri-
als for which the hypothesis and study design are devel-
oped specifically to answer the questions faced by
decision-makers. The characteristic features of PCTs are
that they: 1) select clinically relevant altemative interven-
tions to compare, 2) include a diverse population of study
participants, 3) recmit participants from diverse practice
settings, and 4) collect data on a broad range of health
outcomes. Thus, PCTs help to shift the discussion from
efficacy to effectiveness. Furthermore, the idea behind
PCTs is congruent with the idea that underlies ATI
research. That is, they both attempt to not just answer,
"Which treatment is the best?" but more importandy "Best
for whom, using what protocol, and under what condi-
tions?" We, therefore, recommend that future meditation
research adopt both paradigms in order to maximize treat-
ment safety, efficiency, and effectiveness.
SUMMARY
Meditation is a complex, multifaceted intervention
that represents a mixture of challenges to evidence-based
research. Those challenges should force investigators to
pay particular attention to issues of research design and
intervention integrity. The basic distinctions between
explanatory and efficacy studies are the first step in
appropriate design strategies. We have identified addi-
tional issues more unique to mind-body medicine
research and meditation studies, in particular. These
issues include t) "product" quality (not all meditation
practices are alike), 2) delivery and personal mastery of
the meditation intervention (a self-practiced technique
that often lacks objective physiological markers), 3) indi-
vidual variability in the response to the intervention (pro-
ficiency and aptitude), and 4) feasibility issues related to
Methodological Challenges in Meditalion Research
double-blinding and a true placebo control. Merely iden-
tifying these methodological challenges is the first step in
improving research design. In addition, we have delineat-
ed specific suggestions for research design, using both
quantitative and qualitative approaches. The use of the
ATI approach is described in detail because of tbe
emphasis on individual variations in the meditation apti-
tude. Finally, the future role of PCTs was discussed in
order to make meditation research findings more applica-
ble to clinical decision-making.
References
1. Puryear H, Thurston M. Meditation and the Mind oJ Man. Virginia Beach, VA;
A.R.E. Press; 1975.
2. Barrows KA, Jacobs BR Mind-body medicine; an introduction and review of
the literature. Medical Climes of North America. 2002;86;l 1-31.
3. http;//nccam.mh.gov/health/whatiscam/index.
4. Bootzin RR, Caspi O. Explanatory mechanisms for placebo effects; cognition,
personality, and social learning. In; Guess HA, Kleinman A, Kusek JW, Engel
LW, eds. The Science oJ the Placebo: Toward an Interdisciplinary Research Agenda.
London, UK; BMJ Books; 2002.
5. Astin JA, Shapiro SL, Eisenberg DM, Forys KL. Mind-body medicine; state of
the science, implications for practice. J Am Board Fam Pract. 2003;f6; 131-147.
6. Caspi O. The JNC 7 hypertension guidelines. 7AAM. 2003;290;1313.
7. Kabat-Zinn J. Mindfulness-based interventions in context; past, present, and
future. Oinical Psychology: Science and Practice. 2003; 10; 144-156.
8. Dunn BR, Hartigan JA, Mikulas WL. Concentration and mindfulness medita-
tions; unique forms of consciousness? Appl Psychophysiol Biojeedback.
I999;24;147-165.
9. Wallace RK, Benson H, Wilson AR A wakeful bypometabolic physiologic state.
AmJPhysiol. 1971;221;795-799.
10. Jevning R, Wallace RK, Beidehach M. The physiology of meditation; a review.
A wakeful hypometabolic integrated response. Neurosci Biobehav Rev.
1992;16;4f5-424.
11. Travis F, Olson T, Egenes T, Gupta HK. Physiological patterns during prac-
tice of the Transcendental Meditation technique compared with patterns
while reading Sanskrit and a modern language. Ind j Neurosci.
2001;109;71-80.
12. Davis PA, Gold EB, Hackman RM, Stem JS, Gershwin ME. The use of comple-
mentary/alternative medicine for the treatment of asthma in the United States.
J lnvestigAllergol Clin Immunol 1998;8;73-77.
13. Shapiro DH, Jr. Overview; clinical and physiological comparison of meditation
with other self-control strategies. Am] Psychiatry. 1982;139;267-274.
14. Shadish WR, Cook TC, Campbell DT. Experimental and Quasi-experimental
Designs for Generalized Causal Inference. Boston, Mass; Houghton Mifflin
Company; 2002.
15. Stone AA. The Science of Self-report: Implications for Research and Practice.
Mahwah, NJ; Lawrence Eribaum; 2000.
16. Cunningham C, Brown S, Kaski JC. Effects of transcendental meditation on
symptoms and electrocardiograpbic changes in patients with cardiac syndrome
X. AmJ Cardiol. 2000,85;653-655, A610.
17. Schein MH, Gavish B, Herz M, et al. Treating hypertension with a device that
slows and regularises breathing; a randomised, double-blind controlled
study J Hum Hypertens. 2001;15;271-278.
18. Bishop SR. What do we really know about mindfulness-based stress reduction?
Psychosom Med. 2002;64;71-83.
19. Davidson RJ, Kabat-Zinn J, Schumacher J, et al. Alterations in brain and
immune function produced by mindfulness meditation. Psychosom Med.
2003;65;564-570.
20. Travis F, Wallace RK. Autonomic patterns during respiratory suspensions; pos-
sible markers of Transcendental Consciousness. Psychophysiol. 1997;34;39-46.
21. Travis F, Wallace RK. Autonomic and EEG patterns during eyes-closed rest and
transcendental meditation (TM) practice; tbe basis for a neural model of TM
practice. Conscious Cogn. 1999;8;302-318.
22. Lehmann D, Faber PL, Achermann P, et al. Brain sources of EEG gamma fre-
quency during volitionally meditation-induced, altered states of consciousness,
and experience of the self. Psychiatry Res. 2001; 108; 111-121.
23. Aftanas LI, Golocheikine SA. Human anterior and frontal midline theta and
lower alpha reflect emotionally positive state and internalized attention; high-
resolution EEG investigation of meditation. Neurosci Leu. 2001;310;57-60.
24. Kubota Y, Sato W, Toichi M, et al. Frontal midline theta rhythm is correlated
with cardiac autonomic activities during the performance of an atiention
demanding meditation procedure. Ccgn Brain Res. 200l;l 1;281-287.
25. Otto MW, Nierenberg AA. Assay sensitivity, failed clinical trials, and the con-
duct of science. Psycholher Psychosom. 2002;71;241-243.
Methodological Challenges in Meditation Research
26. Scott AG, Sechrest L. Strength of theory and theory of strength. Evalualion and
Program Planning. 1989;12;329-336.
27. Canter PH. The therapeutic effects of meditation. BMJ. 2003;326;1049-l050.
28. Cunningham AJ. Healing through the mind; extending our theories, research
and clinical practice. Adv Mind-Body Med. 2001; 17;214-227.
29. Caspi O, Bell IR. One size does not fit all; Aptitude x Treatment Interaction
(ATI) as a conceptual framework for CAM outcome research. Part 1 - What is
ATI research? J Aliern Complem Med. 2004;10;580-586.
30. Caspi O, Bell IR. One size does not fit all; Aptitude x Treatment Interaction
(ATI) as a conceptual framework for CAM outcome research. Part II -
Research designs and their applications. J Altern Complem Med.
2004;10;698-705.
31. Shoam V, Rohrbaugh M. Aptitude x treatment interaction (ATI) research;
Sharpening the focus, widening the lens. In; Aveline M, Shapiro. DA, eds.
Research Foundation for Psychotherapy Practice. Sussex, England; John Wiley &
Sons; 1995;73-95.
32. Karuparthy VR, Vepachedu S. Guggulipid and cholesterol levels. JAMA.
2003;290;2800-2801.
33. Canter PH, Ernst E. Multiple n = 1 trials in the identification of responders and
non-responders to the cognitive effects of Ginkgo biloba. IntlJ Clin Pharmacol
Therapeutics. 2003;41;354-357.
34. Tellegan A. Openness to absorbing and self-altering experiences ("absorption");
a trait related to hypnotic susceptibility J Abnorm Psychol. 1974;83;268-277.
35. Neff DF, Blanchard EB, Andrasik E The relationship between capacity for
absorption and chronic headache patients' response to relaxation and biofeed-
back treatment. Biofeedback &• Self Regulation. 1983;8;177-183.
36. Shea JD. Effects of absorption and instructions on heart rate control. J Mental
Imagery. 1985;9;87.
37. Kalauokalani D, Cherkin DC, Sherman KJ, Koepsell TD, Deyo RA. Lessons
from a trial of acupuncture and massage for low back pain; patient expecta-
tions and treatment effects. Spine. 2001;26;14l8-1424.
38. Schulz KF, Chalmers 1, Altman DG. The landscape and lexicon of blinding in
randomized trials. Ann Intern Med. 2002;136;254-259.
39. Schulz KF, Chalmers L, Hayes RJ, Altman DG. Empirieal evidence of bias;
dimensions of methodological quality associated with estimates of treatment
effects in controlled trials. JAMA. 1995;273;408-412.
40. Caspi O, Millen C, Sechrest L. Integrity and research; Introducing the concept
of dual blindness. How blind are double-blind clinical trials in alternative
medicine?; Aliern Complem Med. 2000;6;493-498.
41. Caspi O, Bootzin RR. Evaluating how plaeehos produce change; Logical and
causal traps and understanding cognitive explanatory mechanisms. Evalualion
6- The Heallh Pro/essions. 2002;25;436-464.
42. Kutz 1, Borysenko JZ, Benson H. Meditation and psychotherapy; a rationale for
the integration of dynamic psychotherapy, the relaxation response, and mind-
fulness meditation. AmJ Psychiatry. 1985;142;L-8.
43. Peng CK, MietusJE, Liu Y, et al. Exaggerated heart rate oscillations during two
meditation techniques. InllJ Cardio/. 1999;70;101-107.
44. Bernardi L, Wdowczyk-Szulc J, Valenti C, et al. Effects of controlled breathing,
mental activity and mental stress with or without verbalization on heart rate
variability J Am College Cardiol. 2000;35; 1462-1469.
45. Thomas J, Harden A, Oakley A, et al. Integrating qualitative research with trials
in systematic reviews. BMJ. 2004;328;1010-1012.
46. Pincus T. Limitations of randomized clinical trials in chronic diseases; explana-
tions and recommendations. Adv Mind-Body Med. 2002;I8;l4-21.
47. Donovan J, Mills N, Smith M, et al. Quality improvement report; improving
design and conduct of randomised trials hy embedding tbem in qualitative
research; ProtecT (prostate testing for cancer and treatment) study. BMJ.
2002;325;766-770.
48. Cunningham AJ. A new approach to testing the effects of group psychological
therapy on length of life in patients with metastatie cancers. Adv Mind-Body
Med. 2002;18;5-9.
49. Baer RA. Mindfulness training as a clinical intervention; A conceptual and
impirical review. Clin Psychol: Sci Pract. 2003; 10; 125-143.
50. Lazarus AA, Mayne TJ. Relaxation; some limitations, side effects and proposed
solution. Psychother. 1990;27;261-266.
51. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials; increasing the value of
clinical research for decision making in clinical and health policy. JAMA.
2003;290;1624-1632.
52. ICH Harmonized Tripartite Guideline. E-10; Choice of control group and relat-
ed issues in clinical trials. 2000.
Grant support
At the lime this paper was written, Dr Caspi was on ihe faculty of ihe
Progrann in Inlegralive Medicine at the University of Arizona, and Dr
Burleson was supported by an NIH fellowship (T32 AT001287-01) from the
National Center for Complementary and Altemative Medicine (NCCAM).
The contents of this article are solely the responsibility of the authors, and
do not necessarily represent the official views of NCCAM or the NIH.
ADVANCES Spring 2005, VOL. 21, NO. 1 11