668298

research-article2016
QHRXXX10.1177/1049732316668298Qualitative Health ResearchVandermause et al.

Methods
Qualitative Health Research

Qualitative Methods in Patient-Centered

2017, Vol. 27(3) 434­–442
© The Author(s) 2016
Reprints and permissions:
Outcomes Research sagepub.com/journalsPermissions.nav
DOI: 10.1177/1049732316668298
journals.sagepub.com/home/qhr

Roxanne Vandermause1, Frances K. Barg2, Laura Esmail3,

Lauren Edmundson3, Samantha Girard4, and A. Ross Perfetti2

Abstract
The Patient-Centered Outcomes Research Institute (PCORI), created to fund research guided by patients, caregivers,
and the broader health care community, offers a new research venue. Many (41 of 50) first funded projects involved
qualitative research methods. This study was completed to examine the current state of the science of qualitative
methodologies used in PCORI-funded research. Principal investigators participated in phenomenological interviews to
learn (a) how do researchers using qualitative methods experience seeking funding for, implementing and disseminating
their work; and (b) how may qualitative methods advance the quality and relevance of evidence for patients? Results
showed the experience of doing qualitative research in the current research climate as “Being a bona fide qualitative
researcher: Staying true to research aims while negotiating challenges,” with overlapping patterns: (a) researching
the elemental, (b) expecting surprise, and (c) pushing boundaries. The nature of qualitative work today was explicitly
described and is rendered in this article.

Keywords
methodology; research evaluation; research design; philosophical hermeneutics; United States

The Patient-Centered Outcomes Research Institute
(PCORI) is an example of a relatively new federal initia-
tive that funds research that may include mixed and quali-
tative methodologies. The PCORI, an independent
nonprofit, nongovernmental organization was authorized
by Congress in 2010 “to improve the quality and relevance
of evidence available to help patients, caregivers, clini-
cians, employers, insurers, and policy makers make
informed health decisions” (PCORI, 2015). During the
first year, PCORI funded a majority of research proposals
that included qualitative components in the studies, given
the apparent utility of mixed method and qualitative meth-
odologies in answering research questions that prioritize
patient needs, voices, and perspectives. This is important
because funding for qualitative research has been long
regarded as a challenge. Expanded approaches in patient-
centered research and funding opportunities have
increased (Miller, Crabtree, Harrison, & Fennell, 2013),
yet investigators report continued barriers to funding,
implementing, and disseminating qualitative research.
(Willis, 2007). The current opportunity to explore the
qualitative research experience in the context of the avail-
ability of a new federal funding source offered the poten-
tial to better inform the community of researchers about
the evolving field of qualitative research. In this study, we
set out to examine a specific subset of qualitative research,
the proposals and experiences of inaugural researchers
funded by PCORI in 2012. We were particularly focused
on the current climate of qualitative research, with the
entrance of PCORI as an epochal period.
Decades of discussion in the field of qualitative
research have produced commentary, methods descrip-
tions, and debates between “quantitative” and “qualita-
tive” researchers and among qualitative researchers using
various forms. Many approaches to qualitative research
have been subjected to critiques centered on lack of clarity
for evaluating rigor and concerns about whether qualita-
tive methods have the capacity to answer important
research questions. Increasing recognition for the need to
address problems relevant to practice and renewed interest
in health care science to include patient and stakeholder
voices have emphasized the importance of revisiting past
debates and clarifying understandings and current use of
qualitative methodologies.
1
University of Missouri–St. Louis, Missouri, USA
2
University of Pennsylvania, Philadelphia, Pennsylvania, USA
3
Academy Health, Washington, District of Columbia, USA
4
Washington State University, Pullman, Washington, USA
Corresponding Author:
Roxanne Vandermause, University of Missouri–St. Louis, One
University Drive, Seton 315, St. Louis, MO 63121-4400, USA.
Email: vandermauserk@umsl.edu
Vandermause et al. 435
The literature on qualitative methodologies is exten-
sive. There are numerous books and peer-reviewed arti-
cles that speak to the evolution, types, strengths, and
challenges of using qualitative methods. Some authors
take readers through the process of “doing” the research
(Smythe & Giddings, 2007), although such literature also
integrates epistemological discussions as well. Decades
of scholarship attest to the breadth of methodologies in
this paradigm and chronological descriptions of epochal
periods show the climate of qualitative research is chang-
ing (Lincoln, Lynham & Guba, 2011; Munhall, 2012;
Willis, 2007). Various methodological forms have sharp-
ened the field (Wertz, McMullen, Josselson, Anderson, &
McSpadden, 2011). We were interested in gaining a
cross-sectional focus on the nature of qualitative research
at a point when the context of such research had shifted.
Our collaboration began through the PCORI Pilot
Project Learning Network (PPPLN), a collaborative
learning community initiated by PCORI in partnership
with AcademyHealth for pilot project investigators. We
proposed, during this association, two studies to examine
the current state of the science of qualitative research, one
to examine the components of funded studies using quali-
tative methods (Barg, Vandermause, Edmundson, Esmail,
& Perfetti, 2015) and the current study to analyze the
experience of doing qualitative research in today’s
research climate, with a particular emphasis on what
PCORI describes as patient-centered outcomes research
(PCOR). The long-term goal was to strengthen the under-
standing of qualitative research in conventional health
care science and to increase the acceptability of using
multiple methodologies to answer research questions.
In our recent study (Barg et al., 2015), investigators
examined the research plans of PCORI pilot project
investigators (41 of 50 funded projects used qualitative
methods) and queried them about the resources and chal-
lenges involved in planning for and using qualitative
methods in their studies. Investigators described numer-
ous challenges (e.g., lack of understanding of qualitative
approaches by the scientific community; lack of time for
recruitment, stakeholder engagement, and analysis; prob-
lems locating dissemination venues). Despite these chal-
lenges, investigators also described benefits in qualitative
approaches (e.g., understanding the patient perspective;
learning unexpected findings; calling upon community
and patient stakeholders for practical, relevant informa-
tion). This information helped to discern some of the
challenges that are experienced in today’s research envi-
ronment and set the conditions for the current study, an
examination of the day-to-day experiences of PCORI
pilot project qualitative investigators.
The current study, using the information obtained
from the descriptive results of our preliminary work,
focused on how researchers incorporated patients’ voices
and patient-centered outcomes in their work. This
research is intended to broaden understanding and use of
qualitative research paradigms in PCOR and current
health care research.
Method
The specific aims of this project were to (a) Generate an
interpretation of the state of the science of qualitative
research methods, specifically to illuminate current
experiences with funding, implementing and disseminat-
ing qualitative findings in PCOR, and (b) Identify the
unique contribution that qualitative methodology offers
to PCOR.
An ontological approach was engaged to conduct this
study, which was supported by PCORI Engagement and
Implementation funds. It included in-depth interviews,
ongoing hermeneutic analyses, and presentation to sev-
eral researcher groups, including research faculty at two
participating research institutions at opposite sides of the
country and participants at an international qualitative
research conference. These presentations were designed
to share developing results of the interpretive findings
and to elicit responses that could be incorporated into the
findings. Such conversations are consistent with philo-
sophical hermeneutic approaches which are used to inter-
pret underlying meaning through analysis of transcribed
interviews and interpretive commentary (Diekelmann &
Ironside, 2006). Dissemination events served as auditing
mechanisms and added to the interpretive development of
findings.
Recruitment and Data Collection
Investigators from the inaugural group of PCORI-funded
pilot projects, who participated in our preliminary study
(Barg et al., 2015), were contacted for willingness to par-
ticipate in interviews. A total of 19 investigators responded
to the invitations and provided in-depth telephone inter-
views, lasting 0.5 to 1.5 hours. Incentives were not
offered. Interviews were conducted by two investigators
experienced in conducting hermeneutic interviews
(Vandermause & Fleming, 2011). Questions were open-
ended, though guided by findings from the preliminary
study of qualitative research proposals and investigators’
one-page summaries of the challenges and opportunities
they faced in enacting their work. Participants were asked
to describe their experiences generating research find-
ings, applying findings to practice, and disseminating the
pilot project work. Interviews were transcribed verbatim,
de-identified and shared with the research team via secure
servers. This study was deemed exempt by the Institutional
Review Boards at the University of Pennsylvania and
Washington State University.
436 Qualitative Health Research 27(3)
Data Analysis
The transcribed texts were analyzed via a philosophical
hermeneutic approach to generate an interpretation of
meaning, using a graduated data collection and analysis
process (Diekelmann & Ironside, 2006). Texts were ana-
lyzed in iterative fashion, using in-depth written interpre-
tive summaries of each text, followed by discussions
across texts between two hermeneutic analysts and
expanding the sessions to include all co-investigators for
various select transcripts. A focus on principles of herme-
neutic analysis was maintained (Crist & Tanner, 2003;
Vandermause, 2012). That is, texts were read for struc-
ture, language, content, and inference, resulting in the
emergence of various common manifestations of experi-
ence, described below. Data were managed via Microsoft
Word files. Initial patterns, named “Being genuine:
Researching as elemental” and “Being safe: Researching
to make a difference” centered the description of the
emerging phenomena in the study on integrity and risk.
Member checking is not used in this form of hermeneu-
tics. Rather, a series of validation steps and consensus
discussion among all members of the research team,
some with qualitative expertise and some without, led to
the naming of recurrent patterns. After further analysis of
the interpretations alongside co-investigator commen-
tary, reviewed transcript analyses, and elicitations at dis-
semination events, the following patterns were refined
and explicated, comprising the study results.
Results
The multi-layered text that comprised data for these find-
ings was anchored by the verbatim interview transcripts
of 19 primary investigators of studies funded by PCORI
during its inaugural year. These investigators had various
levels of expertise, self-describing as having years of
qualitative experience, very little qualitative experience,
or some experience with qualitative methods. They repre-
sented western, midwestern, eastern, and southern regions
of the United States. They were in the final stages of data
analysis and dissemination of their 2-year projects.
From these texts and the interpretive process that fol-
lowed, the experience of doing qualitative research in the
current research climate can be understood as “Being a
bona fide qualitative researcher: Staying true to research
aims while negotiating challenges.” This overarching
pattern (Benner, 1994) subsumed three general, overlap-
ping areas of experience (patterns) that describe the phe-
nomenon of interest (the experience of doing qualitative
research in PCOR): (a) researching the elemental, (b)
expecting surprise, and (c) pushing boundaries. All of the
accounts signal these patterns, interpret meaning from the
textual data, and guide research implications. Findings
are described below with exemplars and interpretation
embedded in the discerned patterns.
Pattern: Researching the Elemental
A frequently voiced idea was that qualitative research is
done to understand very basic human phenomena, to
know patients and those who care for them better, and to
get “underneath” happenstance to get to the “real.” There
were numerous references to the manner in which quali-
tative research designs are necessary to answer questions
about primal human concerns. Matters essential or intrin-
sic to human nature, constitutive aspects of experience, or
practices grounded in everyday activities are addressed.
For example, one researcher states,
When I do an interview, I want to get very involved in
hearing the other’s experience. I was able to get underneath
some of the things that were important to people. You know,
get into their source of identity and who they are—who they
were as a person in relation to their disease.
For many, these essential discussions were extended, iter-
ative processes involving intensive day-long interviews
and focus group conversations. There was also the desire
to “drill down” for persons’ reactions, as in the following
example.
And we wanted to do it in a nuanced way. We wanted to hear
what words they used. We wanted to hear what their reactions
were. We wanted to hear if they changed their minds back
and forth—as we thought people might, depending on what
other people at the table were saying. And we thought we
would get a much fuller picture than if we only did a survey.
It was important to these researchers to get to the essence
of the patient and stakeholder experience to fully under-
stand. Taking the time, circling the data to get to the prime
or key matters, was required. “We kept asking ourselves,
how is this helping us understand the patient’s perspec-
tive?” What a patient says, elementally, literally, matters,
and this was conveyed simply.
What does hearing the patient’s voice mean? Just like—it’s
just hearing their perspective and not making an assumption
that you know their perspective.
The pursuit of the prime or elemental idea was often
discussed in terms of the populations of interest, the
patients and those around them. Thus, these interviews
reflected the inseparability of qualitative research and
stakeholder engagement. Many commented on the way
PCORI is shaping the landscape for qualitative research
by prioritizing stakeholder engagement. An example of
this was expressed by a researcher who identified as
Vandermause et al. 437
having a well-established base of community partners
brought into the PCOR project. In the project, persons
with a stake in the issue were “thrilled to have a voice,” to
be heard. This participant told a story about how those
gathered in a discussion group expressed themselves,
revealing emotions that they wanted to register and
changing their emotions when they recognized that
researchers really wanted to hear what they had to say.
And . . . having the face-to-face discussion provided a format
for people to talk about what they were passionate about.
What wasn’t on the survey or just what was most important
to them. And that really—you could feel the kind of
prevailing feeling in the room change as people had a chance
to talk.
In one instance, a stakeholder-partner held a negative
opinion about particular survey questions. Nevertheless,
the group continued a respectful, productive conversa-
tion. This example showed the value in stakeholder
engagement because it ultimately led to a practical under-
standing of issues that mattered to the population of inter-
est, which the participant distinguished as different from
traditional data collection methods.
People were really engaged in our topic. You know, everyone
who came was—there was a lot of positive group dynamic.
Everyone who came finished the survey and stayed and
talked a long time.
Positive moods surrounding patient and other stakeholder
discussions commonly anchored knowledge acquisition
and the sense that they were getting to essential matters,
generating excitement for the researchers. This idea will
be discussed further in the following section.
In “researching the elemental,” there is also a sense
that the work must be done well, thoroughly, enacted with
scholarly rigor. Thus, the experience, even identity of the
researcher as “bona fide” with regard to qualitative exper-
tise, is basic. Researchers often described themselves by
their methodologies and years of experience or lack of
experience. Furthermore, there is an accompanying idea
that such research is “genuine” and those who research in
this way must be genuine, attesting to their basic and
“elemental” nature.
I chose qualitative methods because I feel strongly that it is
a powerful research paradigm that is underutilized,
undervalued, and often critiqued . . . it was my career goal to
bring as much rigor and validity to qualitative methods.
This excerpt illustrates a researcher with a “career goal”
of elevating the science of qualitative methods, assuming
an identity as a qualitative researcher. Others were careful
NOT to identify as such, expressing a fidelity to matching
research questions and methods on a case by case basis.
The unsolicited tendency to clarify personal and profes-
sional associations with methodology was commonly
demonstrated. This underscored the idea that, for the
researcher, qualitative research is a serious and genuine
endeavor, requiring devotion to principles that reflects on
the researcher as a person, and is substantial in the realm
of science.
In some accounts, researchers underscored the value
of learning what patients valued, even if it ran counter to
providers’ wisdom. In the following example, a researcher
heard from the patient why he had made a particular
health care decision.
. . . we show here’s one perspective and here’s another
perspective. What’s right for you? We used a patient
perspective [that was different than what is usually
recommended]. But hearing his story and digging deeper in
it would probably be a shock to some of those doctors who
got angry at him for not getting a (medical device reference).
In this excerpt, the researcher points out the natural and
true, elemental, perspective from that of the patient,
emphasizing the genuine nature of the data obtained by
getting to a core issue.
In summary, “researching the elemental” involves the
research process and outcome as well as the researcher
identity—an investment in quality research. The first
part, involving the process and outcome, relates to the
many references to getting to the bottom of things, under-
standing what really happens in everyday life, eliciting
the patient and stakeholder voice. These references are
found and can be exemplified in nearly every transcript.
Sometimes researchers talked about ways these methods
enhanced or complemented statistical research methods;
sometimes they conveyed the way new and surprising
ideas came forth. Bonus findings occur, those that the
researcher could not have anticipated, and this makes the
discovery process genuine. The idea of being genuine is
extended to focusing on things that matter to patients and
those involved with them, illuminating what’s important
as well as what is elementally learned. The basic human
experience, ontological as well as practical, is recognized
using qualitative methods. The second part of being gen-
uine in researching the elemental is related to the
researcher’s ability to adequately process and synthesize
the information. Researchers who have used qualitative
methods over time identify with their approaches.
Pattern: Expecting Surprise
In most accounts, there is a sense that qualitative research
results in findings that could not have been anticipated.
This is said literally in many cases and insinuated
438 Qualitative Health Research 27(3)
throughout the data. As noted above, the idea that ele-
mental human experiences must be addressed using qual-
itative methods is a common understanding. Furthermore,
researchers who use these methods may be “devoted” to
the approaches they use for this very reason. There were
frequent accounts of joyful, surprising, and meaningful
events in the doing of qualitative research.
And a couple of the folks in that [focus] group said that they
were just so (I hate to use this term, but) grateful—or pleased
about being asked their opinions about something that was,
you know, a policy question. And they were just—everybody
who heard that on the team was like tearing up. So—I know
that happens in qualitative research sometimes, but it was
also I think speaking to—we pulled out all the stops in terms
of recruiting from low-income and underserved communities,
and that was the emphasis of our project.
Researchers’ expressions of joyfulness were unexpected
and distinct findings in the research, in that the pattern
was conveyed so strongly in many of the transcripts. Part
of this was attributed to the relevance of the findings.
Also, the possibility of learning something new or com-
pletely unexpected was serendipitous and welcomed by
researchers.
The pattern, “Expecting surprise” is worded as such
because the semantics imply a paradoxical element that is
characteristic of the research. There is an attitude of open-
ness that is underscored by an expectation that what is
questioned is not yet known. Stories patients tell often
reveal “a bonus finding that happens in qualitative
research.” This is different from experimental designs
where the research question carries with it a hypothesis
that is to be accepted or rejected. Thus, from the onset, a
qualitative researcher must assume there is a result she or
he cannot entirely foresee.
If I want to get into something that I don’t have any idea
what I might find—if I want to get into something that’s
completely new, or if I want to open my mind to the
possibility that there is a phenomenon, let’s say, or an
experience that I haven’t heard about and learned, I really
have to use these kinds of techniques—a technique of
interpretive interviewing, a deeper kind of approach . . .
You know, [as a participant] I have to kind of think about
things to tell you about that I might not have thought about
and I wouldn’t necessarily answer in a little survey because
I’m having to dig a little bit deeper to think about the
issues . . .
This attentiveness to being open necessarily invites
new and unexpected results. One researcher described
patient responses that were not what clinicians expected,
for example, when asking patients whether they had any
outstanding experiences during their hospitalization.
And the patient did say, “Well, I understand there was a
problem with my heart during surgery, so I had to stay
overnight, and I didn’t expect to have to stay overnight.”
And then he said, “Oh, and the door of my room was really
squeaky, so I didn’t get any sleep all night long.” And we
asked each reporter to rate the severity of the event they’re
reporting, and that patient rated the severity of the squeaky
door much higher than the severity of their cardiac arrest in
the operating room.
This response differed from the clinicians’ expectations.
Patients and providers had different agendas and, thus,
different perceptions on care.
But the biggest take-home is that patients and clinicians . . .
in the same place and experiencing the same system of care
have two different worldviews of what that’s all about. And
if one of the goals of healthcare is to deliver patient centered
care, clinicians are going to need to learn about this patient
worldview and presumably address it.
Researchers heard that complications that occurred in
surgery or treatment were less significant to patients than
a “squeaky door” or getting food on time. “I think what I
just said is what I learned, which is that the patient view
was so orthogonal to the clinician view . . .” Other
researchers told similar stories. “Nobody had ever told
me that when I had them in the exam room as a patient.”
The inclusion of patients and stakeholders into the
research process meant that expectations were up-ended.
I can’t think of a single project where I haven’t been
surprised. Bringing stakeholders in-they’re going to look at
it in a different way, and you’re going to learn from that. And
that’s so important . . .
Ultimately, unanticipated findings in these studies
were revelatory and inspired the teams, both not only in
terms of promulgating interventions but also in terms of
doing something joyful in research. The research that is
done is so close to the perspectives of those involved and
often includes asking questions that have not been asked
previously, making the research findings novel. The
unexpected nature of this form of scholarship brings an
aspect of joy even alongside the challenges. For many,
this aspect makes the challenges worth continuing quali-
tative work.
Pattern: Pushing Boundaries
The changing research climate is inclusive of qualitative
research, despite lingering issues related to its acceptance
in the scientific domain. In these accounts, investigators
voiced challenges specific to qualitative research. Many
challenges related to scientific acceptability; others related
Vandermause et al. 439
to the nature of the work itself. “Story-telling is a weapon
of the weak and not used by the powerful.” Qualitative
research can be ambiguous in nature (e.g., changing
research questions and methods, nonspecific or conflict-
ing evaluation criteria, misconceptions about the proper
execution of the work) and consequently might meet with
reviewer (of funding or dissemination venues) bias or
naiveté, according to experienced qualitative researchers.
This challenges them as they write proposals.
Sample size and set procedures are the default assumption,
so we go with what is traditional. Yet, there are times in a
qualitative study that we might change our way of asking
questions or shift the analysis a bit. But we have to present a
proper plan upfront to be taken seriously.
There is, concomitantly, a risk to engaging in qualita-
tive methods. Some researchers did not want to be identi-
fied as “qualitative” researchers, implying that there may
be a career disadvantage in identifying as such. For oth-
ers, being a “bona fide” or genuine qualitative researcher
was considered important for the enactment of good
research.
Concerned because reviewers might not understand
their methodologies, researchers struggled with how to
present their proposals.
As an example, I am used to putting in sample size
determinations and other measured approaches so that the
procedure fits into a grant format or provides what the reviewer
expects. In this case, we didn’t know what the reviewers
might think about procedures so we put in a set sample size
anyway. The idea is to let them know we are being serious
about the project.
Numerous excerpts show that researchers are troubled by
the ambiguous and sometimes marginalized approaches
they must use to answer questions pertaining to patients’
perceptions or experience. “When you push the boundar-
ies a little bit [beyond] established views and approaches
to research, you should expect not to be treated with
acclaim.” When the research questions require qualitative
methods, researchers move past the risks, even if reluc-
tant. “I think a lot of researchers worry that when they
bring in a patient representative and advocate, that it’s
going to mean that they won’t be able to do science any-
more.” Some researchers expressed concern that they
could be leaving some stakeholders out of the discussion
and were hearing from more eloquent and not all voices.
This dilemma shows the paradoxes qualitative research-
ers face, adding to the burden related to acceptability of
their methods.
Participants spoke about legitimacy of qualitative
research. There were repeated intimations in most tran-
scripts that unacceptability by scientists persists and that
demonstrating rigor is a contributing problem. Many
researchers understood that openness in methods leaves
one open to criticism, and that the value in qualitative
methodologies makes this vulnerability tolerable. The
idea that qualitative methods push boundaries and create
risk is strong in today’s qualitative research climate. The
same concerns that showed themselves decades ago
remain active, even though various qualitative method-
ologies are somewhat more acceptable in the scientific
community.
Finally, additional challenges of qualitative work
exacerbate the risk associated with the lingering threat of
scientific unacceptability. The issue of time surfaced in
these interviews, which did not come up in our previous
work examining research plans. In many of the inter-
views, researchers talked about the extra time needed for
quality analysis and the challenges of data collection
methods like interviews.
Well, I think one of the challenges, if you’re not experienced
in the area, you need to understand is that once you’ve
actually done a data collection, you’re really maybe a quarter
of the way done (Laughter), if you’re really doing a good job
with your analysis. And that takes much more time than
folks realize. And, it’s really important to dedicate and
devote that time. I think the other thing, you know, as I’ve
said on study sections and grant review panels, I think
sometimes folks don’t always recognize both on the
reviewer’s side and sometimes you’ll see this in proposals
too, that it’s kind of obvious that somebody just threw in a
few qualitative—oh, we’re going to do some interviews . . .
but they hadn’t really thought it through.
The element of time and experience make for quality.
This participant asserted that the quality is not always
there, that the “standards” for qualitative work have not
yet reached those of experimental methods. It is interest-
ing that, in the discussion of quality, it is often that other
empirical forms of research are brought into the conver-
sation. The traditional understanding that use of statistics
contributes to rigor in approach tends to move the conver-
sation to the need for quantitative components, mixed
methods that complement one another as a preferred
design. Many of these researchers wanted to illuminate
their experience or respect for quantitative methods.
At the same time, there was a pervasive recognition
that the qualitative components of research are the
designs needed to learn what patients and stakeholders
really need. All the participants emphasized the value in
qualitative methods, despite challenges of time, funding,
and dissemination. “Where the hell to publish?” was an
outstanding excerpt because, though pungent, it under-
scores some of the frustration experienced by research-
ers wanting to make a difference in their work for
improved health care.
440 Qualitative Health Research 27(3)
These practical challenges show the nature of qualita-
tive work as demanding. At the same time, the results can
transform patient outcomes. Standing outside traditional
boundaries of science while contributing to extant scien-
tific knowledge, qualitative researchers broaden and
deepen the research enterprise while they struggle for
appropriate funding and dissemination avenues. The
PCORI was recognized as a welcome resource for
advancing the quality and relevance of evidence for
patients.
Discussion
This study focused on the experience of doing qualitative
research. The literature on the experience of doing quali-
tative research is often intertwined with discussions of
how to do such research. In other words, studies of the
experience itself are not distinct from the guiding intel-
lectual foundations of qualitative research. Participants in
this study, likewise, not only talked about their experi-
ence but also spoke at length about how they did their
work and what their work revealed in the context of
PCOR. References to self-reflection, reflexivity, and
identifying or de-identifying with the research focus,
present in the literature over decades, surfaced in this
study. Recent literature continues to bring the researcher
and the research together, personalizing the act of
research. For example, Holloway and Biley (2011) dis-
cuss the necessity of the involvement of the person in a
thoughtful way, balancing what is learned from partici-
pants with what is known about the self. Thoreson and
Öhlén (2015) link the personal experience to knowledge
development, resurrecting Merleau-Ponty’s (1962) dis-
cussions on embodied experience as an aspect of integrat-
ing the subjective and objective worlds. This goes along
with van Manen’s (1991) extensive work, carried forward
today. The bona fides manifested in the experiences of the
participants in this study underscored the importance of
the researchers’ identities as rigorous qualitative research-
ers. This identification worked for and against them,
depending upon the perceptions of others in the research-
ing community. Therefore, moving through challenge
was personal and necessary and showed itself in the pat-
terns of researching the elemental or direct prime experi-
ence, expecting surprise or feeling joyfully open, and
pushing boundaries or doing research despite the chal-
lenges. In many ways, these findings are not new. They
corroborate what qualitative researchers have been writ-
ing about over time.
The research climate of today includes a renewed
focus on patient-centered care. The PCORI has created
new opportunities for funding research that place the
patient and other stakeholders at the center of inquiry. In
this study we have seen that the response to the first
PCORI funding cycle yielded a majority of studies using
qualitative methodologies. By explicitly incorporating
patient-centered outcomes and stakeholder engagement
into the research enterprise, PCORI, as a funding source,
has exerted a strong influence over the nature of the work
that is being done (Forsythe et al., 2015). The funding
focus on patient engagement attracted investigators who
are skilled in eliciting “insider” perspectives on the expe-
rience of illness. Patient-centeredness, along with stake-
holder engagement in the research enterprise, may inspire
more qualitative and mixed methods, even in compara-
tive effectiveness research, advancing the quality and rel-
evance of evidence in health care research.
Qualitative research that has been shaped by the
PCORI’s focus manifested in this study as addressing the
elemental, answering genuine questions and making a
difference. Researchers enjoy and care deeply about such
work and are willing to push boundaries and take risks in
their methodological decisions. The challenges of doing
qualitative research are offset by the rewards experienced
in using such methods. Researchers are willing to commit
extra time and unusual efforts, but they have concerns
about acceptability by reviewers, time required, and
ambiguity in procedures. Thus, “Being a bona fide quali-
tative researcher: Staying true to research aims while
negotiating challenges” reflects a persistent characteriza-
tion, even in a research climate that welcomes attention to
patient-centered needs and stakeholder engagement.
Good elemental research requires transparency. For
qualitative researchers, it is important to show the process
of the work to be trusted in the result. As in all research,
attention to ethics and scholarly rigor is essential. The
nature of these methods is dynamic and flexible. Frequent
protocol changes and shifting views during data collec-
tion and analysis are contrary to scientific assumptions.
This is part of the experience of risk—claiming and
showing expertise while taking a position of being open.
As part of the scientific minority, qualitative researchers
push boundaries and therefore must defend their pro-
cesses and their professional identities to be well regarded
in the scientific community.
Most qualitative researchers are aware of guidelines
established by national groups, including The National
Institute of Health’s (2010) “Best Practices for Mixed
Methods Research in the Health Sciences” and
“Qualitative Methods in Health Research: Opportunities
and Considerations in Application and Review”; The
Robert Wood Johnson Foundation’s documents’
“Qualitative Research Guidance project” and “Guidelines
for Reviewing Qualitative Reports and Grant
Applications”; and numerous publications in the general
health care literature written for the scientific community
on qualitative research best practices (Green et al., 2007;
Maggs-Rapport, 2001; Marshall, Cardon, Poddar, &
Vandermause et al. 441
Fontenot, 2013; Nelson, 2008). Many of these documents
are useful, though various qualitative strategies require
specific guidelines. There are nuances in the variety of
methodological forms that researchers use. Such nuances
are neither widely understood nor consistently referenced
by researchers, making evaluation and critique difficult.
Good research projects may lack credible dissemination
venues. Our study adds to the literature by focusing on
ways that the very qualities of qualitative research, when
done well, extend the power of patient and stakeholder
centered research, although the challenges to funding,
enacting, and disseminating the work remain.
Finally, in recent years, the emphasis on stakeholder
engagement has raised new questions about how to con-
duct patient-centered research in the age of PCOR. Early
work in Community-based Participatory Research (Israel,
Eng, Schultz, & Parker, 2005; Minkler, 2000; Reason,
1994; Small, 1995) has initiated the understanding and
acceptance of methods that include patient and stake-
holder voices. Various community partners are being
brought into the research enterprise. The value of stake-
holder engagement has yet to be thoroughly evaluated in
research (Esmail, Moore, & Rein, 2015), though there is
a strong perception, evidenced by our findings and the
mere presence of a broad literature on the subject, that it
is valuable. Mechanisms for incorporating voices outside
the academic community are being refined in today’s
research climate. This study underscores researchers’
desires to maintain rigor while opening the boundaries of
methodological practices.
Our study has several limitations. We only queried
investigators from the first round of PCORI-funded proj-
ects, and these projects may differ in important ways
from subsequent PCOR studies. We did not query inves-
tigators about their specific backgrounds or the content of
their studies. Relevant information was expected to show
itself essentially, and the phenomenological approach
elicited participants’ priorities, but differences related to
investigator background or content could not be ascer-
tained. Two different interviewers conducted the study;
realistically, this set differing interview scenarios, even
given similar methodological preparation, so common
findings should be considered alongside this fact.
The questions we are led to ask: “What does it mean to
identify as a qualitative researcher? How is it done well?”
are issues raised in this study. Most of the researchers
talked about the challenges of publishing and funding
qualitative work, even today. These concerns were offset
by the conviction that making a difference made the work
worth doing. The extensive time needed to do qualitative
work well was a part of this conundrum, as well as the
desire for guidelines that would make such work accept-
able to a broad audience. The aspect of surprise and joy
that accompanies these methods for many researchers is
an added value, especially as the experience of doing
qualitative research meshes with the experience of doing
a good job, producing revelatory and relevant results that
can be applied to practice.
Conclusion
This study illuminates the importance of qualitative
research in PCOR, the challenges in designing, funding,
implementing, and disseminating qualitative work, and
the need to recognize and establish appropriate mecha-
nisms for enacting the research. The relevance of qualita-
tive research in PCOR is evident, and the institution of
the PCORI as a mechanism for improving patient out-
comes is highlighted. The results of this study emphasize
the profound role that funders play in the course of health
care delivery and the generation of new knowledge. The
experiences of this group of investigators explicated the
practice of current federally funded qualitative research-
ers, updating the literature in this important methodologi-
cal area.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) disclosed receipt of the following financial sup-
port for the research, authorship, and/or publication of this arti-
cle: This work was partially supported through an Engagement
Award (EAIN): Supporting Dissemination and Implementation
Activities of the PCORI Pilot Projects Learning Network
(PPPLN) #1IP2PI000669-01 and AcademyHealth’s Pilot
Project Monitoring Services.
References
Barg, F., Vandermause, R., Edmundson, L., Esmail, L., &
Perfetti, A. (2015). The power of qualitative research:
Implementing the patient voice. Unpublished manuscript.
Benner, P. (Ed.). (1994). Interpretive phenomenology:
Embodiment, caring, and ethics in health and illness.
Thousand Oaks, CA: Sage.
Crist, J. D., & Tanner, C. A. (2003). Interpretation/analysis
methods in hermeneutic interpretive phenomenology.
Nursing Research, 52, 202–205.
Diekelmann, N. L., & Ironside, P. M. (2006). Hermeneutics.
In J. J. Fitzpatrick & M. W. Kazer (Eds.), Encyclopedia of
nursing research (pp. 260-262). New York: Springer.
Esmail, L., Moore, E., & Rein, A. (2015). Evaluating patient
and stakeholder engagement in research: Moving from
theory to practice. Journal of Comparative Effectiveness
Research, 4, 133–145.
Forsythe, L. P., Ellis, L. E., Edmundson, L., Sabharwal, R., Rein,
A., Konopka, K., & Frank, L. (2015). Patient and stakeholder
442 Qualitative Health Research 27(3)
engagement in the PCORI pilot projects: Description and
lessons learned. Journal of General Internal Medicine, 31,
13–21. doi:10.1007/s11606-015-3450-z
Green, J., Willis, K., Hughes, E., Small, R., Welch, N., Gibbs,
L., & Daly, J. (2007). Generating best evidence from
qualitative research: the role of data analysis. Australian
and New Zealand Journal of Public Health, 31, 545–550.
doi:10.1111/j.1753-6405.2007.00141.x
Holloway, I. & Biley, F. C. (2011). Being a Qualitative
Researcher. Qualitative Health Research, 21(7), 968–975.
Israel, B. A., Eng, E., Schultz, A. J., & Parker, E. A. (Eds.).
(2005). Methods in community-based participatory
research for health. San Francisco: Jossey-Bass.
Lincoln, Y. S., Lynham, S. A., & Guba, E. G. (2011).
Paradigmatic controversies, contradictions, and emerging
confluences, revisited. The Sage Handbook of Qualitative
Research, 4, 97–128.
Maggs-Rapport. (2001). “Best research practice”: In pursuit of
methodological rigour. Journal of Advanced Nursing, 35,
373–383.
Marshall, B., Cardon, P., Poddar, A., & Fontenot, R. (2013).
Does sample size matter in qualitative research? A review
of qualitative interviews. The Journal of Computer
Information Systems, 54(1), 11.
Merleau-Ponty, M. (1962). The phenomenology of perception.
London: Routledge & Kegan Paul.
Miller, W. L., Crabtree, B. F., Harrison, M. I., & Fennell, M. L.
(2013). Integrating mixed methods in health services and
delivery system research. Health Services Research, 48(6,
Pt. 2), 2125–2133.
Minkler, M. (2000). Using participatory action research to
build healthy communities. Public Health Reports, 115,
191–198.
Munhall, P. (2012). Nursing research: A qualitative perspective
(5th ed). Sudbury, MA: Jones & Bartlett Learning.
National Institute of Health Office of Behavioral and Social
Sciences Research (2010). Qualitative methods in health
research: Opportunities and considerations in application
and review. National Institute of Health.
Nelson, A. M. (2008). Addressing the threat of evidence-based
practice to qualitative inquiry through increasing attention
to quality: A discussion paper. International Journal of
Nursing Studies, 45, 316–322.
Patient-Centered Outcomes Research Institute. (2015).
Retrieved from http://www.pcori.org/about-us
Reason, P. (1994). Three approaches to participatory inquiry. In
N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of quali-
tative research (pp. 324–329). Thousand Oaks, CA: Sage.
Small, S. A. (1995). Action-oriented research: Models and
methods. Journal of Marriage and Family, 57, 941–955.
Smythe, L., & Giddings, L. S. (2007). From experience to
definition: Addressing the question: “what is qualitative
research?” Nursing praxis in New Zealand, 23, 37–57.
Thoresen, L., & Öhlén, J. (2015). Lived observations: Linking
the researcher’s personal experiences to knowledge devel-
opment. Qualitative Health Research, 25, 1589–1598.
Vandermause, R. K. (2012). Being wholesome: The paradox of
methamphetamine addiction and recovery-A hermeneutical
phenomenological interpretation within an interdisciplin-
ary, transmethodological study. Qualitative Social Work,
11, 299–318.
Vandermause, R. K., & Fleming, S. (2011). Philosophical her-
meneutic interviewing. International Journal of Qualitative
Methods, 10, 367–377.
van Manen, M. (1991). Researching lived experience: Human
Science for an action sensitive pedagogy (2nd ed.). London:
The Althouse Press.
Wertz, F. J., McMullen, L. M., Josselson, R., Anderson, R.,
& McSpadden, E. (2011). Five ways of doing qualitative
analysis. New York: Guilford Press.
Willis, J. (2007). Foundations of qualitative research:
Interpretive and critical approaches. Thousand Oaks, CA:
Sage.
Author Biographies
Roxanne Vandermause is an associate professor at the
University of Missouri-St. Louis College of Nursing. Her role
as Donald L. Ross Endowed Chair for Advancing Nursing
Practice focuses her work on academic-clinical partnerships
and her research involves community based and mental health
concerns in vulnerable groups.
Frances K. Barg is a medical anthropologist and an associate
professor of family medicine and community health at the
Perelman School of Medicine at the University of Pennsylvania.
Her research centers around using qualitative methods to con-
trast patient and expert views on illnesses and their treatment,
including depression, congestive heart failure, diabetes, preg-
nancy loss, ADHD, and Parkinson’s disease.
Laura Esmail is a program officer for the Clinical Effectiveness
Research team at the Patient-Centered Outcomes Research
Institute (PCORI). Previously, she was a Senior Manager with
AcademyHealth where she helped lead and manage a variety of
PCOR projects, including the PCORI Pilot Projects Learning
Network and PCORI’s Research Portfolio Development
program.
Lauren Edmundson is a quality improvement specialist at
Swedish Medical Group in Seattle, WA where she works to
improve cancer screening care for primary care patients. She
previously worked at AcademyHealth, a PCORI partner, focus-
ing on stakeholder engagement in research.
Samantha Girard is graduate of Washington State University
College of Nursing and a Jonas Center for Nursing Excellence
scholar alum. She lectures at the University of Washington in
Tacoma, WA and is a Nursing Education and Practice Specialist
at Group Health Cooperative in Seattle, WA. Her research
focuses on academic progression in nursing education, and
Global and Population Health.
A. Ross Perfetti completed her undergraduate training in
Modern Middle Eastern Studies at the University of
Pennsylvania, and is now a postgraduate student in Medical
Anthropology at Durham University. Her interests are in quali-
tative health research and clinical medicine, with a focus on
refugee health.