Sunteți pe pagina 1din 25

MATH& 146

Lesson 32

Practical Significance

Example 1

Suppose a random sample of size 24 had a sample mean of 33.8 and a sample standard deviation of 9.8.

a) Test the hypotheses

H

0

30,

H

:

:

A

30.

Use a significance level of 0.05.

b) Find the 95% confidence interval to estimate μ. Where is the null value of 30 on this interval?

Example 2

Suppose a random sample of size 30 had a sample mean of 32.8 and a sample standard deviation of 10.57.

a) Test the hypotheses

H

0

40,

H

:

:

A

40.

Use a significance level of 0.01.

b) Find the 99% confidence interval to estimate μ. Where is the null value of 40 on this interval?

Example 3

Suppose a random sample of size 10 had a sample mean of 19.9 and a sample standard deviation of 6.85.

a) Test the hypotheses

H

0

15,

H

:

:

A

15.

Use a significance level of 0.05.

b) Find the 95% confidence interval to estimate μ. Where is the null value of 15 on this interval?

Confidence Intervals and Tests

There is a close relationship between confidence intervals and significance tests.

Specifically, if a point estimate is significantly different from the null value at the 0.05 level, then the 95% confidence interval will not contain that null value.

Confidence Intervals and Tests

Values inside the confidence interval are plausible values for the parameter, whereas values outside the interval can be considered implausible.

Null values inside interval mean "fail to reject".

Null values inside interval mean "fail to reject". Null values outside interval mean "reject".

Null values outside interval mean "reject".

Null values outside interval mean "reject".

(------------------------------|------------------------------)

Point

Estimate

Confidence Intervals and Tests

Looking at non-significant effects in terms of confidence intervals makes clear why the null hypothesis should not be accepted when it is not

rejected: Every value in the confidence interval is

a plausible value of the parameter.

If a null value is in the interval, then it is plausible and cannot be rejected. However, there is an

infinite number of other values in the interval

(assuming continuous measurement), and none of them can be rejected either.

Example 4

The null hypothesis for a particular experiment is that the mean test score is 20. If the 95% confidence interval is (18, 24), can you reject the null hypothesis?

No. You cannot reject the null hypothesis because

the confidence interval shows that 20 is a plausible

value of the population parameter.

Example 5

Which of these 95% confidence intervals represent samples that are significantly different from zero? Select all that apply.

a) (4.6, 1.8)

b) (0.2, 8.1)

c) (5.1, 6.7)

d) (3.0, 10.9)

Example 6

True or false?

If a 95% confidence interval contains 0, so will the 99% confidence interval.

True. The 99% confidence interval contains all of the values that the 95% confidence interval has,

but it extends farther at both ends and has other

values, too. If something is not significant at the .05 level, it is also non-significant at the .01 level.

Example 7

A researcher hypothesizes that the lowering in cholesterol associated with weight loss is really due to exercise. To test this, the researcher

carefully controls for exercise while comparing the

cholesterol levels of a group of subjects who lose

weight by dieting with a control group that does not diet. The difference between groups in cholesterol is not significant. Can the researcher claim that

weight loss has no effect?

Problems with Significance

Failing to reject the null hypothesis can mean either that (1) the null hypothesis is true, or (2) the alternative was actually true but there just wasn't enough evidence (a Type 2 Error).

However, rejecting the null hypothesis can also be problematic.

Example 8

Each graph below shows the difference of two proportions. Which one shows a statistically significant difference?

8 Each graph below shows the difference of two proportions. Which one shows a statistically significant
8 Each graph below shows the difference of two proportions. Which one shows a statistically significant

Example 9

Now using the p-values, which one shows a statistically significant difference? What is going on?

a statistically significant difference? What is going on? p - v a l u e =

p-value = 0.4902

significant difference? What is going on? p - v a l u e = 0 .

p-value = 4.79 E 6

Example 10

Now consider the sample sizes. How does that explain the p-values?

 

A

B

Total

 

A

B

Total

Success

3

4

7

Success

40,000

39,000

79,000

Failure

2

1

3

Failure

60,000

61,000

121,000

Total

5

5

10

Total

100,000

100,000

200,000

p-value = 0.4902

p-value = 4.79 E 6

Problems with Significance

One problem with hypothesis tests is that samples that are too large will tend to reject the null hypothesis regardless of any effect.

The solution is that when you do reject the null hypothesis, you should also consider the effect size.

Effect Size

In most of the hypothesis-testing situations, we are interested in comparing a population mean or proportion to a specific null value. In many research situations, we would like to know

something about the magnitude of the comparison.

The test statistic and p-value for a test are not useful for this purpose because they depend on

the size of the sample.

Effect Size

Statistical Significance: Measures the likelihood you could have gotten your results by random chance. P-values and confidence intervals are considered.

Practical Significance: Measures the likelihood

that the truth differs by chance. Effect size is considered, removing sample size from

calculations.

Cohen's d

A common effect-size measure is

Cohen's d

point estimate

null value

standard deviation

d  point estimate  null value standard deviation Keep your final answer positive. Compare this

Keep your final answer positive.

Compare this to the test statistic:

Test Statistic

point estimate

null value

standard error

Cohen's d

The following table is somewhat arbitrary and should only be used as a guideline of the effect size.

Effect Size

Magnitude

Interpretation

Small

0.0 0.1

Not obvious without statistics

Modest

0.1

0.3

Obvious only to very careful observers

Moderate

0.3

0.5

Obvious to careful observers

Large

 

> 0.5

Obvious to most observers

Example 11

Compare the significance and effect size for each difference.

the significance and effect size for each difference. p - v a l u e =

p-value = 0.4902 d = 0.4364

for each difference. p - v a l u e = 0 . 4 9 0

p-value = 4.79 E 6 d = 0.0205

Hypotheses Testing Steps

1) State the null and alternate hypotheses (in symbols)

2) Choose the significance level (default is α = .05) 3) Choose the test and check the assumptions

4) Calculate the test statistic

5) Calculate the p-value 6) Compare the p-value to alpha

7) Write the decision (reject or fail to reject null)

8) Write a meaningful conclusion about the alternate 9) If null is rejected, then check the effect size

Example 12

Calculate and interpret the test statistic and effect size for each test.

a)

H 0 : μ = 20 vs. H a : μ ≠ 20; SE = 0.25

x

= 19.8, SD = 1.5,

b)

H 0 : p = 0.1 vs. H a : p ≠ 0.1; SE = 0.0595

pˆ

= 0.15, SD = 0.357,

Example 13

In a recent year, of the 109,857 arrests for Federal offenses, 29.1% were for drug offenses (based on data from the U.S. Department of Justice). Test the claim that the drug offense rate is equal to

30%. How can the result be explained, given that

29.1% appears to be so close to 30%?

Use

pˆ

= 0.291, SD = 0.454, and SE = 0.00137.

Example 14

USA Today ran a report about a University of North Carolina poll of 1248 adults from the Southern United States. It was reported that 8% of those surveyed

believe that Elvis Presley still lives. The article began

with the claim that "almost 1 out of 10" Southerners

still thinks Elvis is alive. Test the claim that the true percentage is less than 10%. Based on the result, determine whether the 8% sample result justifies the

phrase "almost 1 out of 10."

Use

pˆ

= 0.08, SD = 0.271, and SE = 0.00768.