Chapter 5 - Measuring Variables and Sampling

Research Methods, Design,
and Analysis
Twelfth Edition
CHAPTER 5
Measuring Variables
and Sampling
Research Methods, Design, and Analysis, Twelfth Edition Copyright ©2014 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Variable and Measurement
• Variable
– a condition or characteristic that can take on
different values or categories
– e.g., gender, reaction time
• Measurement
– the assignment of symbols or numbers to
something according to a set of rules
– gender – male/female
– reaction time – minutes or seconds
Scales of Measurement
• Stevens (1946)
– measurement can be categorized by the type of information
that is communicated by the symbols assigned to the
variables of interest
• Nominal scale
– use of symbols to classify or categorize
– e.g., gender, ethnicity, religion, major in college
• Ordinal scale
– rank-order scale of measurement
– equal distances on scale not necessarily equal on dimension
being measured
– e.g., finishing order in a race, letter grades (ABCDF),
SES (low, medium, high)
• Interval scale
– equal distances between adjacent numbers
– e.g., temperature on Fahrenheit or Celsius scale
• Ratio scale
– highest scale of measurement
– same properties of other scales plus absolute zero
point
– e.g., weight, height, number grades, temperature on
Kelvin scale, reaction time, length
Psychometric Properties of Good
Measurement
• Reliability
– refers to the consistency or stability of the scores of
your measurement instrument
• Validity
– refers to the extent to which your measurement
procedure is measuring what you think it is
measuring and whether you have interpreted your
scores correctly
• A measure must be reliable in order to be valid,
but a reliable measure is not necessarily valid
Types of Reliability
• Reliability
– refers to the consistency or stability of the scores of your test,
assessment, instrument, or raters
• Test-retest reliability
– consistency of individual scores over time
– same test administered to individuals two times
– correlate scores to determine reliability
– how long to wait between tests
 typically an increase in time between testings will decrease reliability
• Equivalent-forms reliability
– consistency of scores on two versions of test
– each version of test given to the same group of individuals
– e.g., SAT, GRE, IQ
• Internal consistency reliability
– consistency with which items on a test measure a
single construct
– e.g., learning, extraversion
– involves comparing individual items within a single
test
– coefficient alpha (Cronbach’s alpha) is common
index
 should be +0.70 or higher
 multidimentional tests will generate multiple coefficient
alphas
• Interrater reliability
– degree of agreement between two or more
observers (raters)
– how is interobserver agreement calculated?
– nominal or ordinal scale
 the percentage of times different raters agree
– interval or ratio scale
 correlation coefficient
Validity
• The accuracy of the inferences, interpretations,
or actions made on the basis of any
measurement
• Construct Validity
– involves the measurement of constructs
– e.g., intelligence, happiness, self-efficacy
– do operational definitions accurately represent
construct we are interested in?
– operationalization is a never-ending process
Methods Used to Collect
Evidence of Validity
• Content validity
– validity assessed by experts
 do items appear to measure construct of interest?
(face validity)
 were any important content areas omitted?
 were any unnecessary items included?
• Internal structure
– how well do individual items relate to the overall test
score or other items on the test
– uni- vs. multidimensional constructs
– factor analysis
 statistical procedure used to determine the number of
dimensions present in a set of items
– homogeneity
 item to total correlation (coefficient alpha)
• Relations to Other Variables
– criterion-related validity
 criterion
– the standard or benchmark that you want to correlation with or predict
accurately on the basis of your test scores.
 predictive validity
– using scores obtained at one time to predict the scores on a criterion at a
later time
– e.g.- GRE and graduate school GPA, LSAT and law school GPA, MCAT
and medical school GPA
 concurrent validity
– degree to which scores obtained at one time correctly relate to the scores
on a known criterion obtained at the same time
– e.g.- new depression scale and Beck Depression Inventory
• Relations to other variables
– convergent validity
 extent to which test scores relate to other measures of the same
construct
 e.g., same as predictive and concurrent validity
– discriminant validity
 extent to which your test scores do not relate to other test scores
measuring different constructs
 e.g., happiness and depression, depression and IQ
– known groups validity evidence
 extent to which groups that are known to be different from one
another actually differ on the construct being developed
 e.g., females high on femininity and males high on masculinity
Using Reliability and
Validity Information
• Norming group
– the reference group upon which reported reliability
and validity evidence is based
• Sources of information about tests
– Mental Measurements Yearbook
– Tests in Print
– PsycINFO and PsycARTICLES
Sampling Methods
• Sample
– a set of elements selected from a population
• Population
– the full set of elements or people from which the sample was selected
• Sampling
– process of drawing elements from population to form a sample
• Representative sample
– a sample that resembles the population
• Equal probability method of selection method (EPSEM)
– each individual element has an equal probability of selection into the
sample
Sampling Methods
• Statistic
– a numerical characteristic of sample data
– e.g., sample mean, sample standard deviation
• Parameter
– a numerical characteristic of population data
– e.g., population mean, population standard deviation
• Sampling error
– the difference between the value of the sample statistic and the value of
the population parameter
• Sampling frame
– a list of all the elements in a population
• Response rate
– the percentage of individuals selected to be in the sample who actually
participate in the study
Sampling Techniques
• Biased sample
– a non-representative sample
• Proximal similarity
– generalization to people, places, settings, and
contexts that are similar to those described in
the study
Random Sampling Techniques
• Simple random sampling
– choosing a sample in a manner in which everyone
has an equal chance of being selected (EPSEM)
– sampling “without replacement” is preferred
– random numbers generators simplify the process
 www.randomizer.org
 www.random.org
• Stratified random sampling
– random samples drawn from different groups or
strata within the population
 groups should be mutually exclusive
 strata can be categorical (nominal or ordinal) or
quantitative (interval or ratio)
 proportional stratified sampling
– involves insuring that each subgroup in sample is proportional to
the subgroups in the population
• Stratified random sampling example
(proportional)
– strata – gender (males/females)
– population – presidents of APA, N = 122
 14 female presidents (11%)
 108 male presidents (89%)
– sample – n = 100
 11 female presidents drawn randomly
 89 male presidents drawn randomly
• Cluster random sampling
– involves random selection of groups of individuals
– clusters
 a collective type of unit that includes multiple elements (has more
than one unit in it)
 e.g., neighborhoods, families, schools, classrooms
– one-stage cluster sampling
 randomly select clusters and using all individuals within
 e.g., randomly select 15 psychology classrooms using all individuals
in each classroom
– two-stage cluster
 randomly select clusters AND
 randomly choosing individuals within each chosen cluster
 e.g., randomly select 30 psychology classrooms then randomly select
10 students from each of those classrooms
• Systematic sampling
–involves three steps
1. determine the sampling interval (k)
 population size divided by desired sample size
2. randomly select a number between 1 and k, and
include that person in your sample
3. also include each kth element in your sample
 periodicity
 potential, but uncommon, problem
 problematic situation in systematic sampling that can
occur if there is a cyclical pattern in the sampling frame
• Systematic sampling example
–population N = 100
–sample n = 10
–k = 10
–randomly select a number between 1 and 10
 e.g., 5
–the 5th person in the population will be included in
the sample along with person #15, 25, 35, 45, 55,
65, 75, 85, and 95
Nonrandom Sampling
Techniques
• Generally produce biased, non-representative
samples
• Convenience sampling
– using research participants that are readily available
– e.g., college students
• Quota sampling
– identifying quotas for individual groups and then using
convenience sampling to select participants within each
group
– e.g., gender – 25 males and 25 females
– e.g., year in school – 15 freshman, 15 sophomores, 15
juniors, and 15 seniors
Nonrandom Sampling
Techniques
• Purposive sampling
– involves identifying a group of individuals with specific
characteristics
– e.g., college freshmen who have been diagnosed with
ADHD
• Snowball sampling
– technique in which research participants identify other
potential participants
– particularly useful in identifying participants from a difficult to
find population
– e.g., Spanish speaking ESL students, parents of children
with autism
Random Selection and
Random Assignment
• Random selection
– involves selecting participants for research from the
population to be included in the sample
– purpose is to obtain a representative sample
• Random assignment
– involves how participants are assigned to conditions within
the research
– purpose is to create equivalent groups to allow for
investigation of causality
– e.g., 20 college students sign up to be participants in a
study. Each is randomly assigned to the treatment or control
group of the study
Determining Sample Size
• Five simple rules for determining sample size
1. if less than 100, use entire population
2. larger sample sizes make it easier to detect an
effect or relationship in the population
3. compare to other research studies in area by
doing a literature review
4. use Table 5.3 in book for a rough estimate
5. use a sample size calculator (e.g., G-Power)
• Larger sample sizes are needed if population is
– heterogeneous
 composed of widely different kinds of people
– you want to breakdown the sample into multiple
subcategories
 e.g., look at males and females separately
– if you want to obtain a narrow or more precise confidence
interval
– when you expect a small effect or weak relationship
– when you use less efficient methods of sampling
 e.g., cluster sampling
– for some statistical techniques
– if you expect a low response rate
Sampling in Qualitative Research
• Qualitative research focuses on in-depth study
of one or a few cases
• Several different sampling methods are
available. It is common to mix several different
methods

Chapter 5 - Measuring Variables and Sampling

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Chapter 5 - Measuring Variables and Sampling

Încărcat de

Drepturi de autor:

Formate disponibile

Research Methods, Design,

S-ar putea să vă placă și