University of Nebraska - Lincoln

DigitalCommons@University of Nebraska - Lincoln

DBER Speaker Series

Discipline-Based Education Research Group

11-14-2013

Steps in Conducting a Scholarly Mixed Methods

Study
John W. Creswell
University of Nebraska - Lincoln, jcreswell1@unl.edu

Follow this and additional works at: http://digitalcommons.unl.edu/dberspeakers

Creswell, John W., "Steps in Conducting a Scholarly Mixed Methods Study" (2013). DBER Speaker Series. Paper 48.
http://digitalcommons.unl.edu/dberspeakers/48

This Presentation is brought to you for free and open access by the Discipline-Based Education Research Group at DigitalCommons@University of
Nebraska - Lincoln. It has been accepted for inclusion in DBER Speaker Series by an authorized administrator of DigitalCommons@University of
Nebraska - Lincoln.

AbstractforDBERGroupDiscussionon20131114

Presenter(s),Department(s):
JohnCreswell
Professor
DepartmentofEducationalPsychology
UniversityofNebraskaLincoln

Title:
StepsinConductingaScholarlyMixedMethodsStudy

Abstract:
Mixedmethodsresearchisarapidlyexpandingmethodologyinthesocialandhumansciencesin
theUSandaroundtheworld.InthispresentationIwillfirstdefinemixedmethodsresearch(combining
bothquantitativeandqualitativemethodsofresearch)anddiscusswhatitisandwhatitisnot.ThenI
willreviewabriefhistoryofitsdevelopment,andwhyitisimportanttoday.Iwilldiscussseveralofthe
scientificdevelopmentsinmixedmethodsthathaveoccurredoverthelasttenyears,suchasthe
specificationoftypesofdesigns,theformationofmixedmethodsquestions,andtheuseofinnovative
approachestojointlydisplayquantitativeandqualitativeresults.Finally,Iwilltalkaboutthefutureof
thismethodologywhereitisheadedandsomeimportantworldwidedevelopmentsthathave
encouragedmixedmethodsresearch.

StepsinConductingaScholarlyMixed
MethodsStudy
JohnW.Creswell
UniversityofNebraskaLincoln

HowWeMakeSense

Numbers

PersonalExperiences

AMixedMethodsApproach

ASimpleIdea
Letsusequantitativeandqualitativedata(orquantitativeresearch
andquantitativeresearch)together togainamorecompleteunderstanding
ofourresearchquestions.

Mixedmethodsisaresearchapproach,popularinthesocial,behavioral,and
healthsciences,inwhichresearcherscollect,analyze,andintegrateboth
quantitativeandqualitativedatainasinglestudyorinasustainedlongterm
programofinquirytoaddresstheirresearchquestions.

POPULARITY FEDERALFUNDING

New NIH funding awards that self-identify as

mixed methods (N = 226) (1997-2008)
70

Number of New Awards

60
50
40
30
20
10
0
1996

1998

2000

2002

2004
Year

Source:CRISPdatabase;
PlanoClark (2010)

2006

2008

2010

DISSERTATIONS

Haines,C.,2011unpublisheddissertation,UniversityofNebraskaLincoln

MixedMethods
SocialScience
Articles

MixedMethodsHealthScienceArticles

SelectMixedMethodsBooks

When did mixed methods develop in

the social sciences and health/behavioral
sciences?
(1985-90)
Jennifer Greene USA- University of
Illinois evaluator
John Hunter and Allen Brewer USA Northwestern and Boston College
sociologists
Alan Bryman - England management
John Creswell USA education
Nigel and Jane Fielding sociologist in UK
Jan Morse nursing in Canada

Ifyouwalkedintomyoffice,withthe
ideaofconductingamixedmethods
study,Iwouldtakeyouthrougha
seriesofquestionstohelpyoudesign
yourproject.

Iwilltakethestepsoutoforderofwhatyoutypicallysee
inaresearchproject,butwewillputtheminorderattheend.

Question1.Doyouhavea
mixedmethodsstudy?(or
simplyaquantitativestudy
oraqualitativestudy)

WhatIamlookingfor(corecharacteristics
ofamixedmethodsstudy):

1. Doyouhaveaquantitativedatabase?(closed
ended)
2.Doyouhaveaqualitativedatabase?(open
ended)
3. Doyouplanonbringingthetwodatabases
together?(integration)
4.Whatprocedures(design)willyouuse?

WhatIamNOTlookingfor:
Keepingthedatabasesseparate
ThecollectionofBOTHquantitative
andqualitativedata(notcontentanalysis)
Multipleformsofquantitativedataormultiple
formsofqualitativedata(multimethod)
Onlythetransformationofqualitative
dataintoquantitative(inanalysis)
Onlyformative/summativeevaluation
Onlyaddinginqualitativedataintoan
experiment
Doingmixedmethodsbecauseitistrendy

Question2.Doesyour
researchproblemorquestion
meritamixedmethodsstudy?

Theysay,chooseyourmethodbasedonyour
problem.

Whenismixedmethodssuitablefor
a problemorquestion?

Whenqualitativeresearchor
quantitativeresearchisinsufficient
tofullyunderstandtheproblem

Othertypesofproblems,when:

Weneedtoexplorebeforeweadminister
instruments
Weneedtoexplainourstatisticalresults
bytalkingtopeople
Weneedtoseeifourquantitativeresults
andourqualitativeresultsmatch
Weneedtoenhanceourexperimentsby
talkingwithpeople
Weneedtodevelopnewinstrumentsby
gatheringqualitativedata

Question3:Whatisthe
overallintentofyourstudy?

Mygoal:
Tolearnwhetheryouare
quantitatively orqualitatively
orientedbythewayyoustate
yourintent

K
e
y
Quantitative
w
ofactors
rdeterminants
relationship
d
causes
sinfluences
effects

i
n
y
o

Qualitative
explore
generate
develop
create
meaning
personalexperiences

YourKeywords:

1. Willhelpmethinkaboutwhatmixed
methodsdesignyouwilllikelyuse
2. Willhelpmethinkaboutyourdiscipline
fieldbackgroundthatyoubringtomixed
methods
3. Willhelpmethinkaboutwhatskillsyou
havetobeginconductingamixedmethods
study

ExamplesofGeneralIntentStatements

Thisarticlereportsonthestudyconductedtounderstand
studentspersistenceintheDistributedDoctoralProgramin
EducationalLeadershipinHigherEducation(ELHE)offeredbythe
UniversityofNebraskaLincoln(UNL).(Ivankova &Stick,2007)

Weconductedaninterviewbasedstudyofterminallyill
individualswhowerereceivingpalliativecare.(Kutner,Steiner,
Corbett,etal.,1999)

tounderstandpatientsexperiencesoftransitionsfrom
hospitaltoahomelessshelteranddetermineaspectsofthese
experiencesassociatedwithperceivedqualityofthese
transitions.(Greysen,Allen,Lucas,etal.,2012)

Question4:Whatquantitative
andqualitativedatawillyou
collecttoaddressthisintent?

Iwillaskyoutolistyourquantitativeand
qualitativedatasources:

QuantitativeData
Siteandsample?
Howmany?
Typesofdatato
collect
Specific
instruments/protoc
ols
Typesof
scales/questions

QualitativeData

Iwillaskyoutolistyourquantitativeand
qualitativedataanalysissteps:

Quantitative
Analysis
Getting dataready
foranalysis
Preliminaryanalysis
Stepsinanalysis
(including statistics,
themes)

Qualitative
Analysis

ListMultipleTypesofQuantitativeand
QualitativeDatatobeCollectedandAnalyzed
QUANTITATIVERESEARCH
Quantitativedata
collection(closed
ended)
Instruments
Behavioralchecklists
Records

Quantitativedata
analysis
Usenumericdata,
Fordescription
Forcomparinggroups
Forrelatingvariables

QUALITATIVERESEARCH
Qualitativedatacollection
(openended)

Interviews
Observations
Documents
Audiovisualmaterials

Qualitativedataanalysis
Usetextandimagedata,
Forcoding
Forthemedevelopment
Forrelatingthemes

27

AreTheseProceduresRealisticandFeasible?

Time

Economically
Skills
Usefulforstakeholders

Question5.Whyareyou
collectingbothquantitative
andqualitativedata?

Basicreasonsforusingmixedmethods
(calledtherationaleinmixedmethods):

Needdifferent,multipleperspectives,ormorecomplete
understandings
Needtoconfirmourquantitativemeasureswithqualitative
experiences
Needtoexplainquantitativeresults
Needbettercontextualizedinstruments,measures,or
interventionstoreachcertainpopulations
Needtoenhanceourexperiments
Needtogathertrenddataandindividualperspectivesfrom
communitymembers
Needtoevaluatethesuccessofaprogrambyusinganeeds
assessmentANDatestofthesuccessoftheprogram

Questions6.Doyouplanon
usingatheory?Doyouplan
onadvancingyourphilosophical
stance?

Frame the study within theory/philosophy:

Paradigm/Worldview (epistemology, ontology, axiology,
methodology)
Theoretical lens
(e.g., feminist, racial, social science theories)
Methodological approach (e.g., ethnography,
experiment, mixed methods)
Methods of data collection (e.g., interviews,
checklists, instruments)

Adapted from Crotty M. (1998)

32

Furtherquestionsaboutphilosophy:

Whatisaphilosophy?
Whichphilosophiesaretypically
usedasthefoundationinmixed
methodsresearch?

Philosophiestoconsiderinmixedmethods:

Pragmatism
Transformativeworldview
Criticalrealism
Dialecticpluralism

Multipleworldviews/paradigms
Matchworldviewtodesign
Worldviewcomesfromyourscholarly
community

Furtherquestionsabouttheory:

Whatisatheory?
Howdoyoufindatheory?
Howwillyouuseitinyourmixedmethods
study?

Question7.Whatmixed
methodsdesignwillyouuse?

Parsimonioussetofdesigns:BasicandAdvanced
Basic Designs
Convergent Design
Explanatory Sequential Design
Exploratory Sequential Design
Advanced Designs
Intervention Design
Transformative Design
Multiphase Design

AdvancedDesigns
Basic
Designs

Basicmixedmethodsdesigns
ConvergentParallelDesign
QuantitativeData
Collectionand
Analysis

Quantitative
Results
Merge
Results
ForComparison

QualitativeData
Collectionand
Analysis

InterpretorExplain
Convergence/Divergence

Qualitative
Results

ExplanatorySequentialDesign
QuantitativeData
Collectionand
Analysis

Quantitative
Results

Determine
Quantitative
Resultsto
Explain

ExploratorySequentialDesign
QualitativeData
Collectionand
Analysis

Qualitative
Results

UseResultsto
FormVariables,
Instruments,
Interventions

QualitativeData
Collectionand
Analysis

QuantitativeData
Collectionand
AnalysisBasedon
Variables,
Instruments,
Interventions

Qualitative
Results

Quantitative
Results

InterpretHow
QualitativeData
Explains
QuantitativeResults

InterpretHow
QuantitativeResults
ProvideNewResults,
New,BetterInstruments,
andBetterInterventions

Advancedmixedmethodsdesigns
InterventionMixedMethodsDesign
ExperimentalStudy

Qualitative
DataCollection,
Analysis,and
Results
(Exploratory)

ExperimentGroup

Pretest

ControlGroup

Pretest

PostTest

Intervention

PostTest

Qualitative
DataCollection,
Analysis,andResults
(Explanatory)

QualitativeDataCollection,
Analysis,andResults(Convergent)

SocialJusticeDesign(usinganExplanatorySequentialDesignexample)
Theory

Research
Questions

Quantitative
DataCollection
e.g.,survey

Quantitative
Results

QualitativeData
Collectionand
Analysis

Qualitative
Results

InterpretHow
QualitativeData
Explains
QuantitativeResults
AndCallsforAction

PromoteSocialJustice
MultistageEvaluationDesign(usinganExploratorySequentialDesignexample)
SingleProgramObjective
FormativeNeeds
Assessment
(qualitative
datacollection)

Theory/
ConceptualFramework
(basedonqualitative
results)

Instrument
Development
(basedon
quantitativetests)

Formative
Program
Assessment
(qualitative
datacollection)

Summative
Program
Evaluation
(prepost
quantitative
tests)

Program
Revision

Howdoyouchooseyourdesign?

Firstselectyourbasicdesign,then
addifyouhaveanadvanceddesign
Consideryourscholarlycommunity:
quantitativeorqualitative?(howtobegin
yourstudy)
Consideryourskills:quantitativelystrong
orqualitativelystrong(whatwillbe
emphasized)
Consideryourresources(collectatonetime
orspaceout)
Considerthecomplexityofthedesign(what
istheeasiest?thehardest?)

AdvancedDesign:Betancourtetal.R24projectexaminingtheuse
ofafamilystrengthinterventionwithcommunityinputwith
refugeesinBoston

CommunityBasedParticipatoryResearchApproach

InterventionDesign
Exploratory
SequentialDesign

Question8.Canyoudraw
a diagramofyourdesign?

Convergent Design Wittink et al. (2006)

Source: based on Wittink et al.(2006)

InterventionDesign(Rogers,Day,Randall,,&Bentall ,
2003,studytoimprovethemanagementofantipsychotic
medication)

Flow of the experiment

Experimental Methodology
qual
before
intervention

Procedures:
One-onone semi-structured
interviews
Thematic analysis

Products:
Transcripts
Developed intervention
treatment

QUAN
QUAN
Pre- Intervention
Post- &
Follow-up
Follow-up
Measure
Measures

Procedures:
Three groups: control group, compliance
Intervention group, alliance intervention
group group comparisons
Outcome measures: 1) attitudes toward
medication 2) adherence to treatment
3) avoidance of relapse
DAI measure completed 3 times (pre, post,
& follow up)

Products:
Numerical item scores
Change scores
Test statistics

qual
after
intervention

Overall
results
and
interpretation

Procedures:
One-on-one semistructured interviews
exiting the trial,
participants from two
experimental conditions;
Thematic analysis

Procedures:
Discuss treatment
effectiveness
Discuss themes in
context of
interventions
and outcomes

Products:
Transcripts
Themes and quotes

Products:
Discussion

Question9.Canyouwriteagood
purposestatement(orstudyaim)
thatfitsyourdesign?

Howtowriteamixedmethodsstudyaim:

Thismixedmethodsstudywilladdress[overall content
aim].Aconvergentparallelmixedmethodsdesignwill
beused,anditisatypeofdesigninwhichqualitative
andquantitativedataarecollectedinparallel,analyzed
separately,andthenmerged.Inthisstudy,
[quantitativedata]willbeusedtotestthetheoryof
[thetheory]thatpredictsthat[independent variables]
will[positively,negatively]influencethe[dependent
variables]for[participants]at[thesite].The[typeof
qualitativedata]willexplore[thecentralphenomenon]
for[participants]at[thesite].Thereasonforcollecting
bothquantitativeandqualitativedataisto[themixing
reason].

Intent
Design

Data
Collection
Rationale

46

Question10.Canyouwrite
researchquestionsforyour
mixedmethodsstudy?

Write3typesofquestions:

Quantitativequestionsor
hypotheses
Qualitativequestions
Amixedmethodsquestion

Basicdesignmixedmethodsquestions:
ConvergentDesign

ExplanatoryDesign

ExploratoryDesign

Towhatextentdothe
quantitativeandqualitative
resultsconverge?
Inwhatwaysdothe
qualitativedatahelpto
explainthequantitative
results?
Inwhatwaysdothe
quantitativeresultsgeneralize
thequalitativefindings?

Advanceddesignmixedmethodsquestions:

InterventionDesign

TransformativeSocialJustice
Design

MultistageDesign

Howdothequalitativefindings
provideanenhancedunderstanding
ofthequantitativeresults?

Howdothequalitativefindings
provideanenhancedunderstanding
ofthequantitativeresultsinorder
toexploreinequalities?
Howtothedifferentphasesinthe
projecttoaddresstheoverall
researchgoal?

Nowplacethesestepsinorder
foragoodmixedmethodsproject:

Problem
Theoryand/orphilosophy
Purpose(studyaim)
Rationaleforgatheringboth
quantitativeandqualitativedata
Researchquestions(quantitative,
qualitative,mixed)
Typesofdata/analysis
Definitionofmixedmethods
Typeofmixedmethodsdesign
Diagramofthedesign

StepsinConductingaScholarlyMixed
MethodsStudy
JohnW.Creswell
UniversityofNebraskaLincoln