The FOCAL SKILLS Advantage

Ashley Hastings Shenandoah University July 2003 Copyright 2003 by Ashley Hastings. You may freely download, print, copy, and distribute this material, but do not alter, add, or remove any content, including this copyright notice.

A certain amount of research has been done to measure the effectiveness of FOCAL SKILLS in comparison with ordinary approaches to intensive ESL. While the body of research is not very large, it points strongly to the conclusion that students in FS programs acquire English proficiency at a significantly faster rate than their counterparts in other programs. I call this the "FOCAL SKILLS advantage." In this paper, I will briefly review and discuss several research studies (some published, some not) that attempt to quantify the FS advantage in terms of three distinct measures: (1) gain scores on the FS placement tests; (2) gain scores on the TOEFL; (3) time required to complete an ESL program. All of the data discussed here are from university-level intensive English programs (IEPs).

1. FOCAL SKILLS students make superior progress as measured by the FS tests.

In a study of short-term gains, Hastings (1995) examined the gains made by FS Listening, Reading, and Writing students (University of Wisconsin-Milwaukee) on their respective skill tests over a 4-week period. The same tests were administered to students in a non-FS program (pre- and post-test), and the gains were compared across programs. The FS students gained about three times as many points per week as their nonFS counterparts. This is not too surprising, given the fact that the FS students spent most of their time focusing on the one skill, while the control students had their time more evenly distributed among the various skills. However, the FS students also made reasonable progress in the skills that they were not currently focusing on. For example, Listening students gained as much per week on the reading and writing assessments as comparable students in the control group. In fact, the evidence suggests that the intensive focus on a single skill does not cost FS students anything in terms of their progress in the other skills. In a separate study with a different comparison program, Yu (1998) obtained similar results. Apparently, then, by focusing on a single skill, FS accelerates progress in that skill by a factor of three, while achieving normal progress in the other skills. The use of the FS tests in these studies raises various questions which need to be answered: Did the FS students study material directly related to the content of the tests? Were the FS students more No. The FS teachers used authentic materials which had no particular connection with the content of the tests, beyond the fact that they were in English. Furthermore, the tests were neither reviewed nor previewed; the test contents were never disclosed, explained, or discussed. Students received no feedback about the tests except their scores. Some of the control students may have had little enthusiasm for the tests. This could have resulted in lower

motivated than the control students to do well on the FS tests?

gain scores that underrepresented their true progress to some extent. However, it is worth noting that even with strong motivation to do well on all the tests, the FS students had impressive gains only in the skills they were focusing on. In the other skills, they performed more or less the same as the control students. This suggests that the main reason for the FS students' superior gains was the effectiveness of the focused instruction they were receiving. A certain amount of this may go on. However, there are three versions of the FS tests, and they are used in rotation. Providing other students with the right answers for the right version at the right time would be quite a task. Furthermore, when these tests have been used for the first time in new FS programs (and the students could have had no prior access to them), the gain scores have been similar to those reported here. This suggests that the FS students really deserved their higher scores. Both of the non-FS IEPs used for comparison purposes in the Hastings 1995 and the Yu 1998 studies were considered to be leading programs in their states. Both were large, financially successful, accredited, and directed by recognized leaders in the field.

Could the FS students have received help ("cheat sheets") from other students?

Were the comparison (nonFS) IEPs really good ones?

2. FOCAL SKILLS students make above average progress as measured by the TOEFL.
It is commonly stated that the average IEP student can expect to gain about one TOEFL point for every 10 hours spent in class. Since the typical IEP offers about 20 hours of instruction per week, this would lead us to expect an average TOEFL gain of about two points per week, or 30 to 32 points per semester. (Note: all the TOEFL scores used here are from the paper TOEFL, which used to be the standard format and is still used extensively in the Institutional TOEFL testing program.) The estimate of two points per week is supported by a large-scale study of institutional TOEFL scores (Wilson 1987). Averaging data from 954 test-takers, Wilson found that the average gain was 36.1 points over an average interval of 4.2 months. Counting a month as 4.3 weeks, this comes to exactly two points per week. Hastings (1995) reported on one semester-long comparison of TOEFL gains involving FS students and a control group. The FS students (University of Wisconsin-Milwaukee) gained an average of 3.0 points per week, while the control students gained 2.4 points per week. Since doing the research reported in Hastings (1995), I have obtained TOEFL gain score data from another FS program (University of Dallas). These students' average gain was 2.7 points per week. It is remarkable, though, that of the 76 students in this group, over half (45) were Japanese. Wilson (1987) breaks his data down by nationality, and it turns out that as a group, Japanese students have a relatively low average TOEFL gain score; using Wilson's data, I compute it as 1.6 points per week. The Japanese students in the FS data had an average TOEFL gain of 1.9 points per week, which is better than Wilson's sample. When we remove the Japanese students from the TOEFL data, the remaining FS students had an impressive 3.6 point average weekly TOEFL gain. This is higher than any of the national groups reported by Wilson. Additional data from Shenandoah University indicates that during the years when SU had a FOCAL SKILLS IEP, its students' TOEFL gains were somewhat better than the averages computed from Wilson's (1987) data.

However, the rate of progress was not as great as that observed in the other FS programs. One reason for this may have been the fact that most of the SU ESL students were already taking university courses in their major fields of study, and could not always spare much time or energy for their ESL studies. ESL attendance was rather low in many cases, and the data indicate that those students with especially poor ESL attendance had very poor TOEFL results. I have been told of a recent study at the University of Wisconsin-Milwaukee that found only average or slightly above-average TOEFL gains for the students in the FS program there. However, this study apparently included only students who had already completed at least one semester in the program. This research strategy is not suitable for measuring the effectiveness of a program, for at least two reasons. First, many of the faster learners complete the program in one semester and therefore do not contribute any data; the sample being measured is therefore biased in favor of the slower learners. Second, the main strength of the FS program design is its ability to rapidly boost students' English proficiency to the intermediate level. After an entire semester, many of the students have already experienced a large part of this boost, and their subsequent growth in English proficiency is almost certainly going to take place at a slower rate. By starting the study at the end of the first semester, the research missed a major part of the students' progress. It is always difficult to evaluate TOEFL data, due to the fact that the multiple-choice format poses a cognitive challenge that is quite distinct in nature from the challenge of dealing with a second language. Furthermore, the TOEFL preparation industry, with its plethora of courses and materials, introduces a wild card into TOEFL research: we can never be sure how much of a student's performance is due to the effects of English study and how much is due to test-taking strategies. Still, because of the impact the TOEFL has on the lives of many ESL students, it is important for any program evaluation to take the TOEFL into account. We could not be comfortable with FOCAL SKILLS if we had reason to believe that it did not give its students at least as good a chance at improving their TOEFL scores as any other approach. While the data we have reviewed presents a somewhat uneven picture, it still seems evident that FS students, under normal circumstances, can expect to enjoy above-average gains on the TOEFL.

3. FOCAL SKILLS students complete their programs faster than other students.
The IEPs that were used as control programs in the FS test studies (described in the first section above) had six levels and operated on an eight-week cycle. Students who were able to meet the promotion requirements were moved up to the next level after one cycle. However, not every student was normally able to pass after only eight weeks. Statistics from the programs in question indicated that about 20% to 25% of the students needed an additional eight-week term to pass a level. Doing the math, we find that the average time per level was about 10 weeks. Data from FOCAL SKILLS programs indicate that students may take four, eight, twelve, or sixteen weeks to complete a module. (Of course, a few take even longer or drop out before passing, but this is also true of the control programs.) The best estimate that I have been able to make is that the average time per module is about 10 weeks. If the average time required to complete either a standard IEP level or a FS module is 10 weeks, then we are in a position to calculate how fast the students advance through their programs. One level is one-sixth, or 17%, of a six-level program. One module is 25% of a FS program. Therefore, it appears that the FS students' average rate of progress through their program is substantially faster. To put it another way, it would take the average student 40 weeks (two semesters plus one summer) to move all the way through a FS program, but it would take the average student 60 weeks (about 4 semesters) to complete an entire standard six-level IEP. (Of course, most students in IEPs do not actually start at the beginning and go all the way through; many receive initial placement in a more advance level or module, and quite a few leave before completing the program. Applying the figures given here, we could estimate that it takes an average of 20 weeks to complete half of a FS program, or 30 weeks to complete half of a standard program, etc.) At this point, though, we must ask whether the FS students lose anything by moving through the program so rapidly. Are they comparable to students in other programs when they finish? Some of the data reported in Hastings (1995) addressed this question.

One of the control programs used a composition as part of its regular assessment process at the end of every semester. The same composition topic was assigned to the students in the most advanced FS module (Immersion) at the end of a semester, and the scores were compared with those of the students in the highest level of the control program. The control group had a slightly higher average score, but the difference was small and not statistically significant. In another study carried out at the same time, the same Immersion students were given a Sentence Repetition Test (SRT), which has been shown to be highly correlated with speaking fluency (Radloff 1991). The same SRT was given to the students in the highest level of the other control program. The FS students had slightly higher scores, but again, the difference was small and of no statistical significance. These comparisons add weight to the claim that FS students acquire English faster than comparable students in other kinds of programs. Neither the composition test nor the SRT is normally used in FS programs, so there is no possibility of bias in favor of the FS students. While the FS students did not achieve higher scores than their counterparts, the point is that they were able to reach the same level of proficiency in less time.

4. So what is the FOCAL SKILLS advantage, and where does it come from? The
evidence we have reviewed says that the FS advantage is time. Students ultimately achieve the same level of ability in any kind of program, given enough time, but on the average, FS students require less time. Exactly how much less is difficult to say, but most of the figures given above represent substantial economies of time for FS students. For example, if a student needs to add 45 points to his TOEFL score in order to be admitted to his university program, the data we looked at in section 2 suggest that one semester in a FS program might well be enough (3 points a week for 15 weeks equals 45 points), but he might need to enroll in ESL for an additional semester in many other programs (2 points a week for 15 weeks gives only 30 points). This could be a very important difference in terms of financial resources and career options. Where does the FS advantage come from? In my opinion, this is easy to understand. In FS, students save time by not wasting it. They do not waste time trying to learn to write before they can read, or trying to learn to read before they can understand a discussion of what they are reading. They do not waste time studying artificial materials that present a cramped, impoverished picture of the language, or learning grammar rules of doubtful utility and questionable accuracy. Nor do they waste time worrying about daily quizzes and weekly tests. The FS principles of progressive functional skill integration, authentic materials, comprehensible input, genuine communication, and low anxiety create an exceptionally efficient environment for language acquisition. The FOCAL SKILLS advantage is predicted by theory, suggested by common sense, and demonstrated by the evidence.

References
Hastings, A. J. (1995). The FOCAL SKILLS Approach: An Assessment. In Eckman et al., Second Language Acquisition: Theory and Pedagogy. Hillsdale, NJ: Lawrence Erlbaum. pp. 29-43. Radloff, C. F. (1991). Sentence Repetition Testing for Studies of Community Bilingualism. Arlington, TX: Summer Institute of Linguistics and University of Texas, Arlington. Wilson (1987). Patterns of Test Taking and Score Change for Examinees Who Repeat the Test of English as a Foreign Language. TOEFL Research Report RR-22. Princeton, NJ: Educational Testing Service. Yu, Bai. (1998). A Comparison of English Proficiency Gains in One FOCAL SKILLS and Two Traditional ESL Programs. Shenandoah University Master's thesis.