Data Analysis and Surveying 101:

Basic research methods and biostatistics as they apply to the

Theresa Jackson Hughes, MPH American College Health Association December 2006

What we will cover today

Research Methods
Sampling Frame and Sampling Generalizability Bias Reliability and Validity Levels of measurement Statistical significance Other key terms Appropriate statistical tests Fun examples from the Spring 2005 dataset!

Biostatistics

Get excited! Its data time!!!

Research Methods

 To do successful research, you don't need to know everything, you just need to know of one thing that isn't known.
Arthur Schawlow

That's the nature of research - you don't know what in hell you're doing.
Harold "Doc" Edgerton

If we knew what it was we were doing, it would not be called research, would it?
Albert Einstein

What exactly is research?

Scientific research is systematic, controlled, empirical, and critical investigation of natural phenomena guided by theory and hypotheses about the presumed relations among such phenomena.
Kerlinger, 1986

Research is an organized and systematic way of finding answers to questions

Important Components of Empirical Research

Problem statement, research questions, purposes, benefits Theory, assumptions, background literature Variables and hypotheses Operational definitions and measurement Research design and methodology Instrumentation, sampling Data analysis Conclusions, interpretations, recommendations

Sampling
What is your population of interest?
To whom do you want to generalize your results?
All students (18 and over) Undergraduates only Greeks Athletes Other

Can you sample the entire population?

Sampling
A sample is a smaller (but hopefully representative) collection of units from a population used to determine truths about that population (Field, 2005) Why sample?
Resources (time, money) and workload Gives results with known accuracy that can be calculated mathematically

The sampling frame is the list from which the potential respondents are drawn
Registrars office Class rosters Must assess sampling frame errors

Types of Samples
Probability (Random) Samples
Simple random sample Systematic random sample Stratified random sample
Proportionate Disproportionate

Cluster sample

Non-Probability Samples
Convenience sample Purposive sample Quota

Sample Size
Depends on expected response rate
Average 85% for paper
FINAL SAMPLE DESIRED / .85 = SAMPLE

Average 25% for web

FINAL SAMPLE DESIRED / .25 = SAMPLE
Size of Campus <600 600-2,999 3,000-9,999 10,000-19,999 20,000-29,000 30,000 Final Desired N All students 600 700 800 900 1,000

Bias and Error

Bias and Error

Systematic Error or Bias: unknown or unacknowledged error created during the design, measurement, sampling, procedure, or choice of problem studied
Error tends to go in one direction
Examples: Selection, Recall, Social desirability

Random
Unrelated to true measures
Example: Momentary fatigue

Reliability and Validity

Reliability
The extent to which a test is repeatable and yields consistent scores Affected by random error/bias

Validity
The extent to which a test measures what it is supposed to measure A subjective judgment made on the basis of experience and empirical indicators Asks "Is the test measuring what you think its measuring? Affected by systematic error/bias

Reliability vs. Validity

In order to be valid, a test must be reliable; but reliability does not guarantee validity.

Levels of Measurement

Levels of Measurement
Nominal
Gender
Male, Female

Interval
Body Mass Index (BMI)

Vaccinations
Yes, No, Unsure

Ordinal
Personal health status
Excellent, Very good, Good, Fair, Poor

Ratio
Number of drinks Number of sexual partners Perception percentages Blood alcohol concentration (BAC)

Last 30 days
Never used, Not in last 30 days, 1-2 days, 3-5 days, 6-9 days, 10-19 days, 20-29 days, All 30 days

Biostatistics

 It is commonly believed that anyone who tabulates numbers is a statistician. This is like believing that anyone who owns a scalpel is a surgeon.
R. Hooke

Torture numbers, and they'll confess to anything.

Gregg Easterbrook

98% of all statistics are made up.

Author Unknown

Types of Statistics
Descriptive statistics
Describe the basic features of data in a study Provide summaries about the sample and measures

Inferential statistics
Investigate questions, models, and hypotheses Infer population characteristics based on sample Make judgments about what we observe

Descriptive Statistics
Mode Median Mean Central Tendency Variation Range Variance Standard Deviation Frequency

Descriptive Statistics Examples

Categorical Variables (Nominal/Ordinal)
Q1 Gen health Frequency 9145 23767 16442 3737 565 132 53788 323 54111 Percent 16.9 43.9 30.4 6.9 1.0 .2 99.4 .6 100.0 Valid Percent 17.0 44.2 30.6 6.9 1.1 .2 100.0 Cumulative Percent 17.0 61.2 91.8 98.7 99.8 100.0

Valid

Missing Total

1 excellent 2 very good 3 good 4 fair 5 poor 6 don't know Total System

Descriptive Statistics Examples

Categorical Variables (Nominal/Ordinal)
Q49 Year in school * Q46 Sex Crosstabulation Q46 Sex 1 female 2 male 1st year undergrad Count 7366 4154 % of Total 14.5% 8.2% 2nd year under Count 6755 3678 % of Total 13.3% 7.2% 3rd year under Count 6195 3333 % of Total 12.2% 6.6% 4th year under Count 5192 2676 % of Total 10.2% 5.3% 5th year or more under Count 1380 985 % of Total 2.7% 1.9% graduate Count 5088 3246 % of Total 10.0% 6.4% adult special Count 203 105 % of Total .4% .2% other Count 266 145 % of Total .5% .3% Count 32445 18322 % of Total 63.9% 36.1% Total 11520 22.7% 10433 20.6% 9528 18.8% 7868 15.5% 2365 4.7% 8334 16.4% 308 .6% 411 .8% 50767 100.0% Q49 Year in school 1 2 3 4 5 6 7 8 Total

Descriptive Statistics Examples

Continuous Variables (Interval/Ratio)

Descriptiv e Statistics Q48 Weight in pounds HT_INCH Height in Inches Q13 How many drinks Q12 Hours alcohol BAC Blood Alcohol Content Valid N (listwise) N 51935 52017 53374 53326 50604 50218 Range 534 56.00 88 65 2.47 Minimum 52 48.00 0 0 .00 Maximum 586 104.00 88 65 2.47 Mean 153.16 67.2035 4.42 2.99 .0731 Std. Deviation 35.791 4.01241 4.401 2.726 .08357 Variance 1281.031 16.099 19.370 7.430 .007

Hypotheses
Null hypotheses
Presumed true until statistical evidence in the form of a hypothesis test indicates otherwise
There is no effect/relationship There is no difference in means

Alternative hypotheses
Tested using inferential statistics
There is an effect/relationship There is a difference in means

Alpha, Beta, Power, Effect Size Alpha probability of

making a Type I error
Reject null when null is true Level of significance, p value

Null is true
Reject null Alpha Type I error 1 Alpha
CORRECT NONREJECTION

Null is false
1 Beta Power
CORRECT REJECTION

Beta probability of making a Type II error

Fail to reject null when null is false

Power probability of correctly rejecting null

1 Beta

Fail to Reject null

Beta Type II error

Effect Size
Measure of the strength of the relationship between two variables

Lets test some hypotheses!!!

Test of the mean of one continuous variable

College students report drinking an average of 5 drinks the last time they partied/socialized
Hypotheses
Ho: = 5 HA: 5

Test: Two-tailed t-test Result: Reject null

One-Sample Statistics N 53374 Mean 4.42 Std. Deviation 4.401 Std. Error Mean .019

How many drinks

One-Sample Test Test Value = 5 95% Confidence Interval of the Difference Lower Upper -.62 -.54

How many drinks

t -30.352

df 53373

Sig. (2-tailed) .000

Mean Difference -.578

Test of a single proportion of one categorical variable

20% of college students report their health is excellent
Hypotheses
Ho: p = 20 HA: p 20 (one-tailed)

Test: Z-test for a single proportion Result: Reject null

Binomial Test Category <= 1 >1 N 9145 44643 53788 Observed Prop. .170 .830 1.000 Test Prop. .2 Asymp. Sig. (1-tailed) .000 a,b

Gen health

Group 1 Group 2 Total

a. Alternative hypothesis states that the proportion of cases in the first group < .2. b. Based on Z Approximation.

Test of a relationship between two continuous variables

There is a relationship between the number of drinks students report drinking the last time they drank and the number of sex partners they have had within the last school year
Hypotheses
Ho: = 0 HA: 0

Test: Pearson Product Moment Correlation Result: Reject null Correlations

How many drinks Pearson Correlation Sig. (2-tailed) N Pearson Correlation Sig. (2-tailed) N How many drinks 1 53374 .238** .000 52576 Partners you had .238** .000 52576 1 52896

Partners you had

**. Correlation is significant at the 0.01 level (2-tailed).

 Men and women report significantly different numbers of sexual partners over the past 12 months
Hypotheses Test: Independent Samples t-test OR One-way ANOVA Result: Reject null
Group Statistics N 32687 18474 Partners you had Sex female male Mean 1.34 1.82 Std. Deviation 2.017 3.627 Std. Error Mean .011 .027

Test of the difference between two means

1 = 2 1 2

Independent Samples Test Levene's Test for Equality of Variances t-test for Equality of Means 95% Confidence Interval of the Difference Lower Upper -.532 -.540 -.434 -.426

F Partners you had Equal variances assumed Equal variances not assumed 867.978

Sig. .000

t -19.360

df 51159

Sig. (2-tailed) .000 .000

Mean Difference -.483 -.483

Std. Error Difference .025 .029

-16.704 25065.988

Test of the difference between two or more means

Mean BAC reported differs across student residences
Hypotheses Test: One-way ANOVA Result: Reject null
Blood Alcohol Content 95% Confidence Interval for Mean Lower Bound Upper Bound .0730 .0752 .1062 .1193 .0598 .0646 .0760 .0785 .0581 .0631 .0545 .0613 .0724 .0738

1 = 2 = 3 = 4 = 5 = 6 i j for at least one pair i, j

Descriptiv es

residence hall frat/sorority house other university housing off campus with parents other Total

N 21285 781 3620 18151 4279 2266 50382

Mean .0741 .1127 .0622 .0773 .0606 .0579 .0731

Std. Deviation .08215 .09278 .07357 .08539 .08490 .08296 .08357

Std. Error .00056 .00332 .00122 .00063 .00130 .00174 .00037

Minimum .00 .00 .00 .00 .00 .00 .00

Maximum 1.27 .75 1.41 2.47 1.17 1.26 2.47

ANOVA Blood Alcohol Content Sum of Squares 3.188 348.695 351.884 df 5 50376 50381 Mean Square .638 .007 F 92.123 Sig. .000

Between Groups Within Groups Total

Test of the difference between two or more means

Multiple Comparisons Dependent Variable: Blood Alcohol Content Games-Howell Mean Difference (I-J) Std. Error -.03865* .00337 .01190* .00135 -.00316* .00085 .01350* .00141 .01623* .00183 .03865* .00337 .05055* .00354 .03548* .00338 .05215* .00356 .05488* .00375 -.01190* .00135 -.05055* .00354 -.01506* .00138 .00160 .00178 .00433 .00213 .00316* .00085 -.03548* .00338 .01506* .00138 .01667* .00144 .01940* .00185 -.01350* .00141 -.05215* .00356 -.00160 .00178 -.01667* .00144 .00273 .00217 -.01623* .00183 -.05488* .00375 -.00433 .00213 -.01940* .00185 -.00273 .00217 (I) Currently live residence hall (J) Currently live frat/sorority house other university housing off campus with parents other frat/sorority house residence hall other university housing off campus with parents other other university housing residence hall frat/sorority house off campus with parents other off campus residence hall frat/sorority house other university housing with parents other with parents residence hall frat/sorority house other university housing off campus other other residence hall frat/sorority house other university housing off campus with parents Sig. .000 .000 .003 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .947 .323 .003 .000 .000 .000 .000 .000 .000 .947 .000 .809 .000 .000 .323 .000 .809 95% Confidence Interval Lower Bound Upper Bound -.0483 -.0290 .0081 .0157 -.0056 -.0007 .0095 .0175 .0110 .0215 .0290 .0483 .0404 .0606 .0258 .0451 .0420 .0623 .0442 .0656 -.0157 -.0081 -.0606 -.0404 -.0190 -.0111 -.0035 .0067 -.0017 .0104 .0007 .0056 -.0451 -.0258 .0111 .0190 .0125 .0208 .0141 .0247 -.0175 -.0095 -.0623 -.0420 -.0067 .0035 -.0208 -.0125 -.0035 .0089 -.0215 -.0110 -.0656 -.0442 -.0104 .0017 -.0247 -.0141 -.0089 .0035

*. The mean difference is significant at the .05 level.

Test for a relationship between two categorical variables

Is there an association between being a member of a fraternity/sorority and ever being diagnosed with depression?
Hypotheses
Ho: There is no association between being a member of a fraternity/sorority and ever being diagnosed with depression. HA: There is an association between being a member of a fraternity/sorority and ever being diagnosed with depression.

Test: Chi-square test for independence Result: Fail to reject null

Test for relationship between two categorical variables

Ev er - Depression * Frat or sorority? Crosstabulation Frat or sorority? yes no 681 7692 715.6 7657.4 3744 39657 3709.4 39691.6 4425 47349 4425.0 47349.0 Total 8373 8373.0 43401 43401.0 51774 51774.0

Ever - Depression

yes no

Total

Count Expected Count Count Expected Count Count Expected Count

Chi-Square Tests Value 2.185 b 2.122 2.211 df 1 1 1 Asymp. Sig. (2-sided) .139 .145 .137 Exact Sig. (2-sided) Exact Sig. (1-sided)

Pearson Chi-Square a Continuity Correction Likelihood Ratio Fisher's Exact Test Linear-by-Linear Association N of Valid Cases

.141 2.185 51774 1 .139

.073

a. Computed only for a 2x2 table b. 0 cells (.0%) have expected count less than 5. The minimum expected count is 715. 62.

 An significant association does not indicate causation Statistical significance is not always the same as practical significance Multiple factors contribute to whether your results are significant It gets easier and easier as you practice!

Important Points to Remember

Questions???