Chapter 15 Independent Groups t-Test

Chapter 15: Independent Groups t-Test

15.1 Uses and Requirements From the preceding chapters, recall that both the z-test and the one-sample t-Test are used to compare a sample mean to a known or assumed population mean, respectively. In both cases there was no independent variable being manipulated; rather, subjects in a sample were selected and that sample was compared to the population mean from which it came. The limitation of the z-Test and one-sample t-Test is they cannot be used to compare performance between levels of an independent variable. To do this you must use a bivariate test. In this chapter I will cover examples where an independent variable is manipulated between groups of individuals. The type of statistical analysis used in such cases is an independent-groups t-Test. As you will see, the procedures, assumptions, and test are not all that much different here than in the one sample t-Test. 15.2 Mean Differences and Variance between Independent Populations The independent groups t-Test is used to determine have been drawn from two different populations, independent groups t-Test, I introduce a few basic each sample is assumed to have come from and populations. whether two sample means, which are assumed to are statistically different. Before getting into the assumptions about the population distribution that the sampling distributions of the means of those

First, in an independent groups design, there are different samples of subjects. It is assumed that the subjects in each group are identical except for the level of the independent variable that each group is exposed. Thus, subjects in each sample are assumed to come from a different population; that is, subjects in Sample A are assumed to come from Population A, and subjects in Sample B are assumed to come from Population B. These populations are assumed to differ only by their level along some independent variable. In short, it is assumed that there are two independent samples, each coming from their own population, which differ by the level of an independent variable. Thus, any difference found between the sample means should also exist between the population means, and any difference between the populations means must be due to the difference in the independent variable. You can visualize this in the figure below, which presents two population distributions. Subjects in each population are assumed to be equal, except for the level of some independent variable. The distance between the peaks (means) of the populations is assumed to be due to any difference in the independent variable. Thus, the amount the distributions overlap reflects how much of an effect the independent variable has on some measure of performance: The more overlap, the less of an effect the independent variable has.

Chapter 15 Independent Groups t-Test Recall, that if you were to select all possible samples of a specific size (n) from a population, and calculate the mean of each of sample, you create a sampling distribution of the mean. This sampling distribution of the mean approximates a normal distribution as n approaches infinity, as per the central limit theorem. The same is true if you have two populations and you select every possible sample of n scores from Population A and every possible sample of n scores from Population B: You end up with a sampling distribution of means for Population A and a sampling distributions of means for population B. Each of these sampling distributions has a mean equal to of its population, and has a standard deviation equal to n (standard error of the mean). If we assume that the sample mean is the best approximation to , then any difference between sample means should reflect the true difference between population means. Thus, any difference in the sample means can be through of as the effect of the independent variable. (Whew, that was a lot to get through! But reread if necessary.) Recall that inferential tests are ratios of some measure of effect to some measure of error. The difference between sample means can be thought of as the effect of the independent variable. So what is the error? Maybe we could use the standard error? Nice guess...but, the standard error of the mean for each population represents only one population. Because we have two populations we need to account for the error variance in both populations, or the variance in the difference between the populations. Consider this: If you selected every sample of size (n) from a population, you end up with a sampling distribution of means, right? So, if you did this for both populations, but calculated the difference between every pair of sample means, you would end up with a sampling distribution of mean differences between independent groups. That is, if you selected every possible sample of n scores from Population A and every possible sample of n scores from Population B, then took the difference between every possible pair of sample means, you end up with a single distribution. This single distribution would be a distribution of mean differences between samples of size n. Say that you select a sample from Population A and find it has a sample mean of 10, and you select a sample from Population B and find it has a sample mean of 11. The mean difference is 10 11 = -1 = 1. If you continue to do this for every possible pair of samples drawn from Population A and Population B, you will find that some differences are greater than zero and some differences are less than zero, but overall the positive differences would cancel out the negative differences. Also, after calculating the differences between sample means, you would find that the resulting sampling distributing of mean differences has a mean difference equal to the difference between the population means (A - B), which is normally assumed to be zero under the null hypothesis. The standard deviation of this sampling distributing of mean differences is equal to the standard error of the mean difference between independent groups:

X1 X 2 =

12 2 + 2 N1 N 2

Each subscript (1 and 2) refers to a different population. The standard error of the difference basically combines the variance from each independent population into one measure. Conceptually, the standard error of the difference is the average difference between sample means of a given size (n) from each population. Note that the standard error of the difference is actually just the summed standard error of the mean for each distribution:

X1X2 =

12 2 2 + N1 N2

or

X1X2

12 2 = + 2 N1 N2

Chapter 15 Independent Groups t-Test However, this formula is only useful when several conditions are met: 1. The population variances are known for both populations, which is almost never the case. 2. The variances in each population are equal, which even if known, is almost never the case.Nonetheless, in most circumstances, you assume that the variances are equal, as per the homogeneity of variance assumption. Because we almost never know the true population variances, you must estimate the standard error of the mean difference between independent groups. 15.3 Pooled Variance and the Estimated Standard Error of the Difference Calculating the estimated standard error of the difference is a two-step process. First, you calculate the pooled variance estimate, which is the combined average estimated variance for both populations based on samples of size n1 and n2:

2 s Pooled =

(n

1) 12 + (n 2 1) 2 s s2 n1 + n 2 2

The subscripts 1 and 2, reference different samples. It does not matter which sample is labeled '1' and which sample is labeled '2', so long as you are consistent. The pooled variance estimate is the average estimated variability between the two independent samples. Because the product of an estimated population variance from a sample and the degrees of freedom of that sample is equal to the sum of squares for a sample, the pooled variance estimate can also be calculated as follows:

2 s Pooled =

SS1 + SS2 n1 + n 2 2

The pooled variance estimate is then used in calculating the estimated standard error of the difference between independent means, or what I often refer to as Bob:

X X s 1 2

2 s Pooled 2 s = + Pooled n1 n2

This is the error term for the independent group t-Test. It represents the estimated average difference between sample means of size n1 and n2 that were selected from independent populations. 15.4 Hypotheses in the Independent Groups Design Recall from the preceding section that the mean of the sampling distribution of the mean differences between independent samples will have a mean equal to the difference between the population means (A - B). Usually, it is assumed that this difference between the population means is equal to zero, which would indicate there is no effect of some independent variable (i.e., A - B = 0) . Any deviation from zero (the population difference) between the two sample means is assumed to be due to the independent variable. Nonetheless, under the null hypothesis the difference between means from two independent groups is generally expected to be zero:

H0: 1 - 2 = 0

or

H0: 1 = 2

Chapter 15 Independent Groups t-Test In short, if under the null hypothesis the two population means are expected to be equal, then the alternative hypothesis is:

H1: 1 - 2 0

or

H1: 1 2

Thus, under the alternative hypothesis the difference between the two population means is not expected to be zero. The null and alternate hypotheses above reflect non-directional (two-tailed) hypotheses; that is, the alternate hypothesis states that there would be some difference between the means, but specifically whether the mean of Population 1 will be greater/less than the mean of Population 2. It is also possible to generate directional (one-tailed) hypotheses for an independent group t-Test. For example, say I predict the mean of Population 1 will be greater than the mean of Population 2. The null and alternative hypotheses are:

H0: 1 - 2 = 0 H1: 1 - 2 > 0

or or

H0: 1 = 2 H1: 1 > 2

The alternate hypothesis is predicting the mean of Population 1 will be greater than the mean of Population 2; that is, the difference between the means of Populations 1 and 2 will be greater than zero. 15.5 Degrees of Freedom in the Independent Groups t-Test Why did I make a section dedicated to degrees of freedom? Because, to put it bluntly, this is the one thing I specifically cover in every statistics class and ~50% of students still manage to get it wrong. PAY ATTENTION MAN! Recall that degrees of freedom are equal to n 1 in a sample. In an independent groups design you have two samples. Each group has their own n 1 degrees of freedom. For example, say that sample A has n = 10 people and sample B has n = 12 people. In sample A, df = 10 1 = 9 and in sample B, df = 12 1 = 11. When dealing with an independent groups t-Test, you need to account for the total degrees of freedom (dfT). Most students assume the total degrees of freedom are equal to the total number of subjects across both samples minus one. Thus, most students guess that the degrees of freedom in this example would be df = nT 1 = 22 1 = 21, because there are nT = 22 people. But, in this case, you have only accounted for 21 of the 20 total degrees of freedom between the samples! Why are there 20 degrees of freedom and not 21? Remember, degrees of freedom are equal to n 1 in a sample. Because there are two samples, we need to account for the degrees of freedom in each. Thus, there are 9 degrees of freedom in sample A and 11 degrees of freedom in sample B, so the total degrees of freedom is df = 9 + 11 = 20. An easier way to calculate the degrees of freedom in an independent groups design is df = nT 2 where nT is the total number of subjects tested across both groups (22). Hence, dfT = 22 2 = 20. 15.6 Example of Independent Groups t-Test Many colleges and universities require students to take a Freshman Seminar course, where students are acclimated to college life and taught study skills and time-management skills. Lets say at Faber College, freshmen are required to take such a course, and this course has always covered basic study skills. One year, a psychologist who conducts research in learning develops a new study technique where students

Chapter 15 Independent Groups t-Test acquire study skills working in groups, rather than studying alone. This new system should increase GPA relative to covering only basic study skills. The researcher randomly selects n = 10 freshmen and puts them into a traditional seminar course (Basic Study,) and randomly selects n = 10 different freshmen and puts them into the new seminar course (Group Study). All students complete this course and at the end of their freshman year the GPAs from all 20 students are collected and the mean GPA is compared between groups. The researcher predicts that the mean GPA in the Group Study condition will be greater than the mean GPA in the Basic Study condition (directional hypothesis); thus, the hypotheses are:

H0: Group = Basic H1: Group > Basic

The data (GPAs) for both groups at the end of the semester are presented in the table below. You can see there are n = 10 different students in each condition and each GPA is measured to the nearest thousandth.

Basic Study Student Bob Rob Tom Judy Mary Pete Hans Pat Floyd Marge GPA 2.000 2.300 2.300 2.700 3.000 3.700 2.700 3.000 3.300 2.000

Group Study Student Stew Roger Phil Jen Christine Melissa George Gina Tim Tony GPA 3.300 3.300 3.300 4.000 3.700 2.000 2.000 2.300 2.700 2.700

Before calculating any means and performing the t-Test, we first need to determine our critical t-Value. To determine t you need the alpha level (), whether you have a directional or non-directional hypothesis, and the total degrees of freedom Above, I stated that the researcher was predicting that the Group Study Condition should have a mean that is greater than the mean in the Basic Study Condition; thus, the researcher has a directional (onetailed) hypothesis. Say that the psychologist selects an alpha-level of = .01 The total degrees of freedom are equal to the total number of subjects in this study minus two; thus dfT = 20 2 = 18. From the t-Tables, the critical t-Value is t = +2.552. The steps for conducting the independent groups t-Test, which are going to be similar to those for the correlated samples t-test in the next chapter, are as follows: 1. 2. 3. 4. 5. Determine the mean and sum of squares for each sample Using the sums of squares, calculate the estimated variance Estimate the standard error of the mean difference between independent groups Conduct the independent groups t-Test Determine the significance of the t-Test and make decisions about hypotheses.

Chapter 15 Independent Groups t-Test Steps 1 3 have been alluded to earlier and the formulas displayed in previous sections, so they will not be elaborated on here. First, we need to calculate the means and sums of squares of each group. This is done in the table below, using the data from above: Group Study Student Stew Roger Phil Jen Christine Melissa George Gina Tim Tony GPA (X1) 3.300 3.300 3.300 4.000 3.700 2.000 2.000 2.300 2.700 2.700 X1 = 29.3 Basic Study

(X

X1 )

(X

X1 )

Student Bob Rob Tom Judy Mary Pete Hans Pat Floyd Marge

GPA (X2) 2.000 2.300 2.300 2.700 3.0000 3.700 2.700 3.000 3.300 2.000 X2 = 20

(X

X2 )

(X

X2 )

0.37 0.37 0.37 1.07 0.77 -0.93 -0.93 -0.63 -0.23 -0.23

0.137 0.137 0.137 1.145 0.593 0.865 0.865 0.397 0.053 0.053 SS1 = 4.382

-0.700 -0.400 -0.400 0 0.300 1.000 0 0.300 0.600 -0.700

0.490 0.160 0.160 0 0.090 0.000 0 0.090 0.360 0.490 SS2 = 2.84

X1 = 2.93

X 2 = 2 .7

Using the sums of squares, we first calculate the pooled variance estimate:

2 s Pooled = 2 s Pooled = 2 s Pooled 2 s Pooled

SS1 + SS2 n1 + n 2 2

4.382 + 2.84 10 + 10 2 7.222 = 18 = 0.401

Now that we have the pooled variance estimate we can calculate the estimated standard error of the difference between independent groups (Bob):

X X = s 1 2 X X = s 1 2 X X = s 1 2

0 . 401 0 . 401 + 10 10 0 . 040 + 0 . 040 0 .08 0

X X = 0 . 283 s 1 2
This value (0.283) is the estimated standard error of the difference for the independent groups t-Test; that is, it is the estimated average mean difference between population the means for the Basic Study condition and Group Study condition.

Chapter 15 Independent Groups t-Test The next step is to perform the independent group t-Test and calculate our obtained t-Value:

t=

(X

X 2 ) ( 1 2 ) X X s 1 2

The numerator includes the actual difference between the two sample means and the hypothesized difference between the two population means. The difference between the two population means is usually assumed to be zero under the null hypothesis. In such cases, the (1 - 2 ) will be equal to zero and can be dropped form the t-test:

t=

X1 X 2 X X s1 2

The next step is substituting in the sample means and performing the t-Test. This is very important: If you have a non-directional (two-tailed) hypothesis test, it does not matter which sample mean that you subtract from the other. This is because, with a non-directional hypothesis, you are predicting some difference, and that difference could be positive or negative, it does not matter. In contrast, when you have a directional (one-tailed) hypothesis, it does matter which sample mean that you subtract from the other. Follow these rules if you have a directional (one-tailed) hypothesis: (1) If you predict one sample mean to be greater than the other sample mean, then you are predicting a positive difference between sample means and a positive t-Value (test statistic). The mean that you are predicting to be greater than the other should be the sample mean with the subscript '1'; that is, the mean that comes first in the numerator. (2) If you predict one sample mean to be less than the other sample mean, then you are predicting a negative difference between sample means and a negative t-Value (test statistic). The mean that you are predicting to be smaller than the other should be the sample mean with the subscript '1'; that is, the mean that comes first in the numerator. Please note that for both of these rules, it does not matter whether this is what you find in the means; the placement of the sample means into the t-Test is based on what you predict, not what you observe. Substituting in our sample means and the estimated standard error of the mean difference calculated earlier, we have:

t=

X1 X 2 2.93 2.7 0.23 = = = 0.813 X X s1 2 0.283 0.283

This obtained t-Value (t = 0.813) is our test-statistic (obtained t-value). We compare this value to our critical t-Value, which was t = 2.552. Because the absolute value of our test statistic value is smaller than the absolute value of our critical value, we conclude there is not enough evidence to claim that there is a statistically significant difference in GPA between the Basic Study condition and the Group Study condition. That is, the mean difference is not statistically significant. Therefore we retain the null hypothesis and make no decision regarding the alternate hypothesis. In laymans terms, we conclude that exposing freshmen students to a new for of Group Study technique resulted in a small non-significant increase in end of year GPAs compared to students taking a Basic Study skills course.

Chapter 15 Independent Groups t-Test 15.6 Reporting in the literature Several parameters need to be reported with the independent groups t-Test: (a) either both sample means or the mean difference between the sample means, (b) the obtained t-Vlue, (c) the estimated standard error of the mean difference between independent samples, (d) the total degrees of freedom, and (e) the alpha-level. Below, I present a generic example of how the results from Section 15.5 (Note, that in APA format, an Italicized M is use as a symbol for the sample mean.): Results Twenty Faber College students were randomly sampled and randomly assigned into a Basic Study condition or into a Group Study condition, with n = 10 students per independent group. At the end of these students freshman year, the GPA of each student was recorded and the mean GPA in the Basic Study condition was compared to the mean GPA in the Group Study condition. An independent group tTest revealed a non-significant difference between the mean of the Basic Study condition (M = 2.70) and the Group Study condition (M = 2.93), t(18) = 0.813, SE = 0.283, p < .01 (one-tailed). Thus, students learning to study as a group had a small and non-significant increase on GPA. 15.6 Confidence Intervals Around the Mean Difference Confidence intervals can be calculated the difference between the sample means. In an independent groups t-Test most researchers calculate confidence interval around each sample mean. To do this, you estimate the standard error of the mean for each sample and using the critical t-Value based on the degrees of freedom for each sample multiply that estimated standard error by t to get the confidence interval around each sample mean. By doing this you can see if each sample mean lies within the confidence interval of the other sample mean. Others like to calculate confidence interval around the mean difference between the sample means. To do this, you use the standard error of the difference and t from the independent groups t-Test. Thus, for the example from Section 15.5, we have (note, because = .01, this would be the 99% confidence interval around the mean difference) :

CI = (X1 X 2 ) X1 X2 t s CI = (2.93 2.7 ) (0.283)(2.552) CI = 0.23 0.722

Recall that the null hypothesis predicts that the difference between means will be equal to zero. If zero falls within this confidence interval we can assume that the mean difference is not significantly different from the expected mean difference of zero, which is the case in the example here. 15.7 Effect Size: Eta Squared One question is how much strong an effect the independent variable had on the dependent variable. That is, from the example above, is a mean difference of 0.230 a large or a small? To know how much of an effect the independent variable had on the dependent variable, one must calculate a measure of effect 2 size. One measure of effect size measure called eta-squared ( ), which is the proportion of variance in

Chapter 15 Independent Groups t-Test the dependent variable that is attributable to the effect of the independent variable. The eta-squared effect size is the ratio of total treatment variance (the effect of the independent variable) to total variance; thus:

2 =

SS Effect SSTotal

The total sum of squares (SSTotal) is the total variation across all of the scores in a set of data. That is, combined over both levels of an independent variable, what is the total variability among all of the scores? To find the total sum of squares, you need to calculate something called the grand mean,, which is is the sum of all scores in a set of data divided by the total number of scores. In the example from Section 15.5 the sum of all the scores (GPAs from both the Basic Study and Group Study conditions) was X = 56.300, and the total number of subjects was n = 20. The grand mean is G = 56.3/20 = 2.815. Note, this is the same value you would get by adding the mean of each condition and dividing by two. The sum of squares total is found by subtracting the grand mean from each score in the data, squaring the differences, and adding the squared differences. This is done in the table below: Calculating SSTotal 2 (X - G) (X - G) 0.485 0.235 0.485 0.235 0.485 0.235 1.185 1.404 0.885 0.783 -0.815 0.664 -0.815 0.664 -0.515 0.265 -0.115 0.013 -0.115 0.013 -0.815 0.664 -0.515 0.265 -0.515 0.265 -0.115 0.013 0.185 0.034 0.885 0.783 -0.115 0.013 0.185 0.034 0.485 0.235 -0.815 0.664 SSTotal = 7.486 Calculating SSEffect 2 (M - G) (M - G) 0.115 0.013 0.115 0.013 0.115 0.013 0.115 0.013 0.115 0.013 0.115 0.013 0.115 0.013 0.115 0.013 0.115 0.013 0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 -0.115 0.013 SSEffect = 0.264

Student Group Stew Group Study Roger Group Study Phil Group Study Jen Group Study Christine Group Study Melissa Group Study George Group Study Gina Group Study Tim Group Study Tony Group Study Bob Basic Study Rob Basic Study Tom Basic Study Judy Basic Study Mary Basic Study Pete Basic Study Hans Basic Study Pat Basic Study Floyd Basic Study Marge Basic Study

GPA (X) 3.300 3.300 3.300 4.000 3.700 2.000 2.000 2.300 2.700 2.700 2.000 2.300 2.300 2.700 3.000 3.700 2.700 3.000 3.300 2.000

M 2.93 2.93 2.93 2.93 2.93 2.93 2.93 2.93 2.93 2.93 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70

The sum of squares effect (SSEffect) is calculated by subtracting the grand mean from the sample mean that is associated with each individual. For example, in our example from section 15.5, the student Stew was in the Group Study condition, and that condition had a mean of 2.93. The grand mean (2.815) is subtracted from the mean of Stews group (2.93). This results in a treatment effect of 0.115 for Stew. This is done for each individual in Stews group, and the same is done for each individual in the Basic Study condition, with the grand mean being subtracted from the mean for that other group (2.7). Thus, Bob and all of the others in the Basic Study condition have a treatment effect of 2.700 - 2.815 = -0.115. This is shown in the table on the next page. In the table, each student is listed in the leftmost column, and I have then listed which study group condition each student was in. Each students GPA is then listed, followed by the mean (M) of that 2 students group. The (X - G) and the (X - G) columns show the calculation of the sum of squares total.

Chapter 15 Independent Groups t-Test Finally, the (M - G) and the (M - G) columns show the calculation of the sum of squares effect. Using the SSTotal and SSEffect values, the eta-squared effect size is:
2

2 =

SSEffect 0.264 = = 0.035 SSTotal 7.486

Thus, about 3.5% of the variance in the dependent variable (GPA) can be accounted for by the effect of the independent variable (Study Condition). The proportion of variance that cannot be explained is 100% - 3.5% = 96.5%. Thus, about 96.5% of the variance cannot be explained by the effect of the independent variable. Of course, this process would take some time, and there is a simpler method for calculating eta-squared, which makes use of total degrees of freedom and your obtained t-value:

t2 = 2 t + df
2

Using the total degrees of freedom and obtained t-value from earlier, eta-squared is:

0.8182 0.661 0.661 = = = = 0.035 0.8182 + 18 0.661 + 18 18.661

2

15.8 Effect Size: Cohens d Another popular index of effect size is Cohens d. There are several methods for calculating Cohens d, but the most appropriate is to divide the difference in the sample means by the estimated pooled standard deviation, which is just the positive square root of the pooled variance estimate. What is nice about Cohens d is that it provides a standardized measure that can be used to compare across different studies. Specifically, Cohens d is a measure of the standardized difference between sample means; that is, it is a distance between two sample means in standard deviations. From the example in Section 15.5, the estimated pooled standard deviation is:

Pooled = 2 s s Pooled = 0.401 = 0.633

Cohens d is:

d=

X1 X 2 2.93 2.7 0.23 = = = 0.363 Pooled s 0.633 0.633

Thus, the two sample means are separated by only 0.363 standard deviations.

Chapter 15 Independent Groups t-Test Cohen also provides useful labels to describe how big or how strong of a relationship the effect size indicates. The table below reports the minimum Cohens d and eta-squared values that correspond to weak, moderate, and strong effect sizes (also called small, medium, and large effect sizes):

Effect Size Small (Weak) Medium (Moderate) Large (Strong)

Cohens d .20 .50 .80

eta-squared ( ) .01 .06 .14

Based on our eta-squared value ( = 0.035) and or Cohens d value (d = 0.363), the strength of the effect of the independent variable in the example from section 15.5 would represent a weak or small effect of taking ginko-baloba versus a placebo on GPAs. 15.9 Statistical Power Recall, from earlier chapters, statistical power (1 - ) is the probability of correctly rejecting a false null hypothesis. You want this probability to be high, so that a statistically significant result most likely reflects a true significant difference; or to show that a non-significant difference is most likely not significant. Thus, you generally want the statistical power of an inferential test to be .80 or greater (note that power has a limit of 1.00, because it is a probability). There are a number of methods that are used to determine the power of an inferential test, but here, I will show you how to estimate power from a set of power tables. Most researcher actually start out by selecting a level of power that they want for their inferential test; much like choosing an alpha level. Hence, power is not always calculated beforehand, but it can be. The reason that researchers choose an a priori power level, is so that they can determine the sample size needed to achieve that level of power. That is, say that you want to achieve a power of .80 in an independent groups t-Test. Based on this desired level of power, and on several other parameters, you can perform a power analysis to determine how many subjects are needed to achieve this level of power. This is because, sample size is directly related to power, and sample size is something that a researcher has direct control over. Below, I list the parameters that are associated with power, and their relationship to power: 1. Effect size and power: The larger the effect size, the more power in an inferential test. 2. Alpha level and power: The smaller the alpha level, the less power in an inferential test. 3. Directionality of alternate hypothesis and power: Non-directional alternate hypotheses are less powerful than directional hypotheses. 4. Sample size and power: Typically, the larger the sample size, the more power there is in an inferential test. Using the example from Section 15.5, we can determine approximately how much power the t-Test had for correctly rejecting a false null hypothesis. Recall, that the number of subjects per group was n = 10, the alpha level was = .01, we had a directional (one-tailed) alternate hypothesis, and the effect size was 2 = .035 for eta-squared and d = .363 for Cohens d. The power tables are in in Appendix E. First, in the Power Tables, locate the table for the type of inferential test used (independent groups tTest), the alpha level ysed ( = .01), and whether the alternate hypothesis was directional or nondirectional (directional). This will be Table 1.d in the Power Tables, whihc I have reproduced below:

Chapter 15 Independent Groups t-Test

= .01 (one tail) / = .02 (two tails) effect size (d) 0.05 0.05 0.10 0.15 0.20 0.25 0.30 0.35 Power (1 - ) 0.40 0.45 0.50 0.55 746 1749 2665 3530 0.10 0.15 188 439 668 884 85 197 299 395 488 580 673 767 864 965 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 49 112 169 223 276 327 379 432 487 544 604 668 738 815 903 32 73 109 144 177 211 244 278 313 349 388 429 473 523 579 645 726 836 23 51 77 101 124 18 38 57 75 92 14 30 44 58 71 84 97 12 24 36 46 57 67 77 88 98 110 122 10 20 29 38 46 55 63 72 80 89 99 9 17 25 32 39 46 53 60 67 74 82 91 8 15 21 27 33 39 45 51 57 63 70 77 84 93 103 114 7 13 19 24 29 34 38 43 49 54 60 66 72 80 88 98 7 12 16 21 25 29 34 38 42 47 52 57 63 69 76 85 95 6 11 15 19 22 26 30 33 37 41 46 50 55 61 67 74 83 95 6 10 13 17 20 23 26 30 33 37 40 44 49 54 59 66 73 84 5 9 12 15 18 21 24 27 30 33 36 40 44 48 53 58 65 75 90 5 8 11 14 16 19 21 24 27 30 33 36 39 43 47 52 59 67 81 5 8 10 13 15 17 20 22 24 27 29 32 35 39 43 47 53 60 73 5 7 10 12 14 16 18 20 22 25 27 29 32 35 39 43 48 55 66

4369 1094 5198 1302 6031 1510 6878 1722 7752 1940 8662 2167

147 109 170 126

194 143 110 218 161 124 243 179 138 270 199 153 298 220 169 330 243 187 364 268 206 403 297 228 449 330 254 505 372 285 581 428 328 704 518 397

9622 2408 1072

0.60 10650 2665 1186 0.65 11768 2944 1310 0.70 13006 3253 1447 0.75 14411 3605 1604

134 109

148 120 100 163 133 110 181 147 122 201 163 135 226 184 152 260 211 175 314 255 211

0.80 16060 4017 1787 1006 0.85 18096 4526 2013 1134 0.90 20830 5209 2317 1304

128 110

147 126 109 178 152 131

0.95 25235 6311 2806 1580 1012

115 101

You will notice that the power table is made up of columns of Cohens d values in increments of 0.05, and rows of Power (1 - ) values, in increments of 0.05. The values in the tables are sample sizes. To determine approximately how much power we had in the independent group t-test from Section 15.5: First, locate the closets Cohens d value in the columns that is not greater than the Cohens d measure of the t-test (d = 0.363). The closest Cohens d value that is not greater than d = 0.363 is 0.35 (column is hilighted in yellow). Second, scroll down that column until you come to the number of subjects in the study, which in the example from Section 15.5 was n = 20. Youll notice that n = 20 does not appear in that column. Just use the closest value that is less than n = 20, which will ne n = 18 (the row is highlighted in orange). Finally, look to the far left column where the Power values are lcoated. The value in that row for n = 18 is the approximate power associated with this t-test (Power = 0.05). We conclude that the Power to correctly reject a false null hypothesis is less than 0.05, which is EXTREMELY LOW! This is relatively low power. Given that we have such low power, we cannot really conclude that the null hypothesis is correct; we need more power to conclude that the mean difference observed in Section 15.5 is truly non-significant. The power tables can also be used to determine how many subjects are needed for a study, based on a desired level of statistical power and a predicted effect size. For example, say that I run a study that will compared two levels of an independent variable, A1 and A2. Based on previous research, I expect an effect size of approximately 0.50. I want to be able to correctly reject the null hypothesis was a Power equal to 0.80. To determine the number of subjects needed for this study, I go to the row associated with d = 0.50 and scroll down until I come to the row associated with Power = 0.80. The sample size at that intersection of Power = .80 and d = .50, which I have hi-lighted in green, is n = 163. This is the number of subejcts I would need to include in my study. These n = 163 subejcts would be roughly equally distributed between my groups, A1 and A2, to achieve this much Power with that effect size (about 82 per group). Thus, these tables have two uses: determining the Power of an inferential test, and determining the sample size needed to achieve a certain level of Power.

Chapter 15 Independent Groups t-Test

Homework Questions
1. When would the independent groups t-test be used to analyze a bivariate relationship? 2. What will the mean of a sampling distribution of the difference between two independent means be equal to? 3. Determine the critical t-value(s) for an independent groups t-test for an alpha level of .05 and .01 under each of the following conditions: a. H0: 1 = 2, H1: 1 2 n1 = 8, n2 = 8 b. H0: 1 = 2, H1: 1 > 2 n1 = 8, n2 = 8 c. H0: 1 = 2, H1: 1 2 n1 = 20, n2 = 15 d. H0: 1 = 2, H1: 1 > 2 n1 = 20, n2 = 15 e. H0: 1 = 2, H1: 1 2 n1 = 8, n2 = 17 f. H0: 1 = 2, H1: 1 < 2 n1 = 8, n2 = 17

4. Given the following information, compute the estimated pooled variance and the estimated standard error of the mean difference between independent groups for n1 = 10, n2 = 13, s 2 = 5.575 , s 2 = 4.235 . 5. Given the following information, compute the estimated pooled variance and the estimated standard error of the mean difference between independent groups for n1 = 50, n2 = 50, SS1 = 40000, SS2 = 60000. Use the following to answer Exercises 6 - 8: The following information comes from two samples that were randomly selected from their respective populations: Sample A nA = 100 Sample B nB = 100

X A = 120
SSA = 1025

X B = 130
SSA = 2050

6. Compute the estimated pooled variance and the estimated standard error of the mean difference between independent groups 7. Calculate the obtained t-Value between the means (assume a non-directional alternate hypothesis). 8. Using an alpha level of = .01, and assuming a non-directional alternate hypothesis, is the mean difference statistically significant?

Chapter 15 Independent Groups t-Test Use the following to answer Exercises 9 - 11: The following information comes from two samples that were randomly selected from their respective populations: Sample A nA = 5 Sample B nB = 8

X A = 15
SSA = 125

X B = 12
SSA = 135

9. Compute the estimated pooled variance and the estimated standard error of the mean difference between independent groups

10. Calculate the obtained t-Value between the means (assume a non-directional alternate hypothesis). 11. Using an alpha level of = .05, and assuming a non-directional alternate hypothesis, is the mean difference statistically significant? Use the following to answer Exercises 12 - 19: A professor teaches two sections of the same course and wants to know whether posting audio recordings of his lectures to his website has any effect on student grades. The professor posts audio recordings for one class, and for the other class he posts a written summary of his lecture. Below are the data for both conditions: Audio Recordings n = 25 No Audio Recordings n = 25

X = 3.35
SS = 45

X = 2.55
SSA = 45

12. Expressed in terms of , what are the null and alternate hypotheses? 13. What is the critical t-Value (t) for = .05? 14. What is the estimated pooled variance and estimated standard error of the difference between independent groups? 15. What is the obtained t-Value for the difference between the two sample means? 16. Based on the obtained t-Value, what conclusions should you draw about the null and the alternate hypotheses? 17. Calculate the effect size measure eta-squared ( ). Does this represent a small, medium, or large effect? 18. Calculate the estimated pooled standard deviation and then calculate Cohen's d. What does the value of Cohen's d tell you about the separation of the two means? 19. Using the Power tables, approximately how much statistical power did this t-Test have to correctly reject the null hypothesis?
2

Chapter 15 Independent Groups t-Test Use the following information to complete Exercises 20 - 27: A researcher studied the influence of marijuana on reaction times by having five subjects eat a marijuana laced brownie until a certain level of stoned-ness was achieved. Another group ate non-marijuana laced brownies (placebo). All subjects then completed a speeded task. The reaction times (in seconds) are presented in the table:

Alcohol 3.00 3.50 3.00 2.00 2.50

Placebo 1.00 0.50 1.00 1.00 1.50

20. What is the critical t-Value (t) for = .05? 21. Calculate the means for each group and the sum of squares for each group. 22. What is the estimated pooled variance and estimated standard error of the difference between independent groups? 23. What is the obtained t-Value for the difference between the two sample means? 24. Based on the obtained t-Value, what conclusions should you draw about the null and the alternate hypotheses? 25. Calculate the effect size measure eta-squared ( ). Does this represent a small, medium, or large effect? 26. Calculate the estimated pooled standard deviation and then calculate Cohen's d. What does the value of Cohen's d tell you about the separation of the two means? 27. Using the Power tables, approximately how much statistical power did this t-Test have to correctly reject the null hypothesis? 28. For each of the following situations, find the total number of subjects that would be needed to achieve the desired level of Power. a. d = .50; = .05 (non-directional); Power = .80 b. d = .10; = .05 (directional); Power = .95 c. d = .80; = .01 (directional); Power = .85 d. d = .25; = .01 (non-directional); Power = .90 29. For each of the following situations, find the amount of Power, based on the parameters given. a. d = .25; = .05 (non-directional); n = 75 b. d = .55; = .01 (non-directional); n = 180 c. d = .75; = .05 (directional); n = 30 d. d = .45; = .01 (directional); n = 190
2