An Approach to ESP Materials Evaluation

Paul McGuinness, Manchester University Med ELT MD 350

Abstract
This paper sets out the approach taken to evaluate published materials for use on the
corporate English for General Business Purposes (EGBP) language training course described
in an earlier paper (McGuinness, 1999a). A summary of the teaching situation and the needs
and priorities identified will be given, the approach to the selection of suitable materials
explained and the evaluation criteria described. A comparative evaluation of the texts will be
conducted based on the data gathered and a best buy for the particular training situation will
be suggested.

Introduction
A training course for a group of middle management employees at a Thai company was
requested from a private language school. A needs analysis approach to the provision of the
course was adopted with the results indicating that the company required a short, one-off
course with broad business English needs at a high elementary/low pre-intermediate level.
Although certain skills and situations had been identified as key areas by the Participants, a
specific focus on these objectives was not requested. Given this and considering the amount
of money and time available, it was decided to treat the course as English for General
Business Purposes (EGBP) and to use commercially available EGBP materials that included
some work on the areas suggested by the needs analysis. This necessitated a summative
evaluation (Rea-Dickins and Germaine, 1992: 26) be conducted in order to evaluate and
select appropriate materials from the wide range available. The evaluation was based on the
three stage approach described by Leckey (1987) with modifications in the focus to make it
suitable to the specific situation outlined above.

In the first stage, a survey of EGBP materials available on the market was conducted
and those that matched broad practical needs of the course such as fulfilling the role
expected of them, price and availability were selected for further examination.

In the second stage, a more detailed examination of the practical and pedagogic
features of the shortlisted material was conducted using criteria drawn from both the
needs analysis and the relevant literature.

Finally a comparative analysis based on the findings was used to identify the material
that best matched the needs established by the needs analysis.

While conducting the evaluation it was understood that it would be difficult if not impossible
to find published material that was perfectly suitable (Robinson, 1991: 58) and that practical
considerations may have to outweigh pedagogic concerns (Robinson, 1991: 58). However, by
evaluating the materials on both practical and pedagogic levels considering all the variables
involved as McDonough (1998: 164) has suggested it was hoped that material which best
suited the environment overall would be identified.

I will now outline the approach taken in more detail.

First stage
Rea-Dickins and Germaine suggest that the first objective in an evaluation process is to
define what we mean by materials so that it is clear what is being examined (1992: 28). For
2

this situation materials refers to published (commercially available) course packages which
in addition to a students book can include supplementary resources such as a teachers book,
workbook, audio cassette(s) and possibly video cassette.

Next, we should define the role the materials are expected to fulfil so that we can assess their
suitability for evaluation (Ellis and Johnson, 1994: 125). The materials will be the main
resource (a primary text (Leckey 1987: 114)) for both the teacher and the participant and so
should provide as complete a package as possible so that the need for modification or
supplementation is minimised. They should provide a ready made, coherent EGBP syllabus
that includes work on the target language or skills identified from the analysis of the
participants target needs, and should be at a high elementary/low pre-intermediate level.

These factors formed the initial basic criteria for selection and the following material was
identified from publishers lists and bookshops as suitable candidates for further evaluation:

Business Basics (1995) Elementary to pre-intermediate.

Business Challenges (1995)- False beginners
Business Objectives (1996) - Lower intermediate.
Getting Ahead (1987) Pre-intermediate
International Express (1996) Pre- intermediate

The next stage of the evaluation was to conduct a closer practical and pedagogic survey of
each of the texts.

Stage Two
To facilitate the examination of the shortlisted courses the instrument in Appendix I was
created. A brief explanation of the rationale underlying the criteria contained in the
instrument and how it relates to the needs analysis is given below.

Practical Criteria
The practical needs were defined by the ecosystem of the course, which included the
constraint of time, the wishes of the Sponsor and considerations of the needs of Participants
and teacher.

In relation to time, the course material was examined to assess whether it could fit into a 30hour time frame in a coherent and satisfying way. Similarly the internal structure of the book
was considered because the materials had to be used in lessons of only one hours duration.
With reference to the wishes of the Sponsor, he had requested a course with a clear outline, so
the material should have a map of the syllabus, which clearly shows the learning objectives of
each part. An end of course progress test had also been requested so material with built in
progress tests, review sections or workbooks that could be used for review purposes were
needed. For the teacher and Participants the materials should have an attractive presentation,
be up to date and have relevant supplementary materials.

Pedagogic Criteria
The pedagogic aspects of the materials were examined in relation to criteria drawn from
relevant literature. The first four criteria contained in the instrument; Audience, Aims,
4

Content and Methodology, are based on Hutchinson and Waters materials evaluation (1987:
99). The Motivational factors are from Cunningsworth (1984:79).

Audience
The kind of learner the material is aimed at should be similar to the kind of learner we are
going to use the text with. In this case the Participants are mature, middle class, middle mana
gement employees with a number of years of job-experience.

Aims
The claimed aims and the aims as discerned in the material should match the aims of the
Participants as identified from the aims analysis improvement in grammar, vocabulary,
conversational ability and writing skills.

Content
When looking at the content of the materials several key features that will affect the
suitability and success of the materials were examined. These included the approach taken,
the skills covered, the nature of the texts and the organisation and sequencing of the content.
These aspects should match the Participants personal details, educational background, and
learning styles preferences as closely as possible.

Methodology
This part of the survey included an examination of the theories of learning that the course is
based on (behaviourist, cognitive, affective or a combination), the kinds of exercises provided
and the teaching/learning techniques that can be used with the material. It is important in the
evaluation to consider whether or not the exercises are appropriate to the preferred learning
styles, strategies and activities of the Participants as discerned from the Learning Styles
Assessment, Participant Profile and Target Needs Analysis.

Motivational factors
Learner motivation is very important in the learning process (Hutchinson and Waters, 1987:
57) and on a Sponsored corporate course it can be even more of a concern . There can never
be a guarantee that extrinsic motivation will be high. Although it may appear that Participants
will be extrinsically motivated because they need English for their jobs very often this is not
the case (Hutchinson and Waters, 1987: 48). Participants might feel they have been forced
into the learning and might even resent being in class. They may on the other hand welcome
the chance to be in a class but have different target objectives that they see as a priority (eg
English for personal use outside work). In such cases intrinsic motivation (generated by the
materials or teacher) becomes crucial to the success of a course. Even when extrinsic
motivation is high intrinsic factors remain a priority (Hutchinson and Waters, 1982: 122) and
a course will be more successful if both kinds of motivation can be tapped in the learners.

In this situation materials with relevance to the general situation and target situation language
would be best with enough variety and pace to motivate tired, perhaps resentful learners in

only an hour. They should avoid being culture-specific and be interactive and comprise some
element of personalisation

Stage Three
Once the examination of the materials was completed the materials were evaluated in relation
to the needs established in the Needs analysis. A process of elimination could then begin as
their relative strengths and weaknesses became apparent.

The first book to be rejected was International Express. Although it is attractively presented
and contains many of the features required the Euro-centric content, general interest topics an
d General English Coursebook layout are unsuitable for a Sponsored business English cour
se. It could be useful for a language school to use for open general business classes as a transi
tional text for learners moving from EGP to EBP. Learners in such classes may well feel more
secure with a course that presents business through familiar topics (general/human interest) in
a familiar format (General English course book format). In a Sponsored course however the f
ocus needs to be more Business oriented, if not for the learners then at least for the Sponsor.

From the remaining four titles Business Basics was next to be eliminated because although it
is very business oriented, it does not have progress tests and the workbook can not be used fo
r the purpose of summative evaluation either. It also lacks the pace of the other materials and
would appear to be more suitable for a long-term continuous course.

Business Challenges although containing progress tests, short snappy units with internal
variety and pace and relevant content suffers from its tendency to concentrate on conversation
and ignore texts. It is largely Euro-centric and uses companies and brands that are unknown
here in Thailand. Its design also detracts from its competitive edge. It looks somewhat drab
with its olive green cover and dull illustrations and the pages can seem cluttered because of
the two-column format.

The two titles with most promise were Business Objectives and Getting Ahead. In practical
terms they are both very similar although Getting Ahead does suffer because of its age. This
is most noticeable in that it has few colour photographs and relies heavily on cartoons and
illustrations, which although in colour are not found as much in more recent publications.
Despite this it is attractive; the format is clear, the units well-labeled and easy to move
around. Another minor flaw however is that it lacks a detailed outline of the content of each
unit, something which again is increasingly the norm and something that was requested by
the Sponsor.

Business Objectives on the other hand appears a lot more attractive simply because it has a
glossy cover and contains more colour photographs than cartoons and illustrations. In additi
on to this it has a detailed outline which gives information about the content of each unit
under the headings: topics, language, vocabulary/pronunciation and skills work. Internally it
has a clear format and it reminds Participants about the aims and objectives of each unit at the
beginning of each unit.

In terms of audience both books have been designed for job-experienced and pre-experience
learners and neither suffers from any major incongruities because of this. Suggestions are

made in both texts and their respective teachers books on how to cater to the differing needs
of either set of learners. The Participants should find the material suitably appropriate.

With regards to meeting the time constraints of the course however, Getting Ahead is slightly
better suited to the thirty, one-hour sessions format than is Business Objectives. The units in
Getting Ahead are shorter (6 pages to as compared to the 10 page units in Business
Objectives) and there is constant presentation and practice of language items using a variety
of skills which means constant variety in a one hour session. The long units in Business
Objectives spend more time on individual aspects of a lesson (presentation, practice and
production) and so could appear slower in one-hour classes and would require a teacher to
work harder to vary the format. They would be better suited to two-hour sessions.

Getting Ahead is also better suited to a thirty hour format because the book provides material
for between 50 and 70 hours of work so it is possible to comfortably conduct two coherent
thirty-hour courses. This can be motivating for Participants because they will get more of a
sense of progress in terms of both amount of material and topics covered than if they used
Business Objectives, which would have to be divided into three thirty-hour sections.

Both titles have progress tests, but the Business Objectives progress tests are in the Teachers
book whereas in Getting Ahead they are contained in the students book. For our situation
where progress tests have been identified as central to the course a high level of control is
beneficial and therefore this is a distinguishing feature between the texts. In these terms
Business Objectives would be much better suited to our course because of the security and
control the teacher has over the test material.

In terms of content both books are equally good, the topics are relevant and there is adequate
coverage of the target language areas needed by the Participants. The principle organising
approach of both books is similar - Getting Ahead is situational/functional and Business
Objectives is skills/situations and both books also include other approaches. This is fairly
common in materials (Hutchinson and Waters, 1987: 89) and has been done well. Both books
provide a sensible teaching program (Swan, 1985; quoted in Robinson, 1991: 41; and
Hutchinson and Waters, 1987: 89).
They are both also to be commended in that they mainly avoid culturally specific situations the content is international and deals with global brands only. Getting Ahead goes further
however because its topics are handled very broadly and are often non-culture specific which
means they can be personalised much more easily. The standard food/eating out/socialising
unit for example is not about Western restaurant habits as in so many other books (including
unit 6 in Business Objectives, which is very Euro-centric). In both courses however the
audio-tape could pose problems for Thais because it uses a lot of non-native speakers who are
predominantly European.

With regards to methodology Getting Ahead is more Participant and teacher friendly. As
mentioned earlier there is plenty of variety and pace, it has a good balance of skills with
cognitive problem solving exercises and personalised communicative activities. For the
teacher it does not require much, if any supplementation. Business Objectives on the other
hand requires the teacher to work a bit harder because things do not seem to follow on as well
as they could. It would seem to be necessary to have to jump around the book and possibly to
supplement from other sources. The language work, practice and production phases come in
chunks which can cause the units to drag a little and sometimes what is presented in the
grammar and what is practiced doesnt always seem to follow logically. However an

10

important point is that neither text relies too heavily on cognitive methodology or discovery
learning. This makes them suitable for use with the Participants whose learning styles
analysis results indicated a preference for a more teacher-centred classroom which is in
accordance with their educational background (Brindley, 1989: 76) and their level of English
(see Del Mar et al, 1982).

Motivationally possibly Getting Ahead could be said to be easier to teach up or down which
is important given that the class in question is mixed ability. In addition the Target language
areas requested occur frequently in the first half of the book and the pace and of the units (10
units can be covered in the course) should also help with motivation. With Business
Objectives the target areas are not as prominent and less of the material can be covered in a
thirty hour course. More work would be required of the teacher to keep the material
motivating and lively.

Result of Analysis
Getting Ahead and Business Objectives would both be ideal candidates for a course book for
this specific situation. Both have different strengths and weaknesses and ultimately the choice
could depend on who the teacher is going to be (because Business Objectives demands
slightly more work from the teacher), or on whether the focus is going to be on security
(because the progress tests are in the Getting Ahead Students book).

For the sake of this paper it will be assumed that a standard ESP teacher will be employed
to deliver the material. Despite the fact that a test was requested there is not a lot of emphasis

11

placed on security in this situation because the tests were regarded by the Sponsor as affective
rather than summative tools (to give the Participants a sense of achievement and progress).

With regards to this therefore I would select Getting Ahead. The teacher would like it because
it is easier to teach and can be tailored to a mixed ability class with different Learning
Strategies profiles quite easily. The Participants should be happy because it has variety and
pace which should keep them motivated even in short one-hour lessons that are conducted
outside office hours. It also deals extensively with the target areas identified in the needs
analysis. The Sponsor should be satisfied because it does contain a course outline (which
could be more fully outlined by the Language School), It fits nicely into the hours available
matches the target needs (business) and is a reasonable price. If everything runs smoothly in
the course (monitored by on-line needs analysis) then the Language School and Academic
Director should be satisfied too because a successful course has been provided.

Conclusion
This paper has described an approach for the selection of published material for an ESP
course. The approach was based on the belief that materials should be appropriate to the need
s of all the parties involved in the course, Sponsor, Provider, Participant, Teacher, Language s
chool and Training Manager. This approach was used in order to minimise potential clashes a
nd ensure that all parties felt some satisfaction in being a part of the organisation, and running
of the course. Hopefully every party would feel that the course had been a success (judged su
bjectively of course success would be defined differently by each party) and further courses
might be a possibility. The evaluation criteria were based on information gathered through a n
eeds analysis approach the objective of which was to gather information that would facilitate

12

the provision of a course that satisfies all parties involved as fully as possible in accordance w
ith the particular features of the ecosystem. Getting Ahead was selected as suitable because it
met the needs of all parties and the ecosystem to the greatest extent.

13

Bibliography
Brindley G (1989) "The role of needs analysis in adult ESL programme design", in Johnson R
(Ed) (1989).
Cunningsworth A (1984) Evaluating and Selecting EFL Teaching Materials, London:
Heinemann.
Del Mar M, M Viano and V Orquin (1982) Identifying our Students Strategies for Learning
English as a Foreign Language, Modern English Teacher, 9/4.
Ellis M and C Johnson (1994) Teaching Business English, Oxford: Oxford University Press.
Hutchinson T and A Waters (1987) English for Specific Purposes, Cambridge: Cambridge
University Press.
Johnson R (Ed) (1989) The Second Language Curriculum, Cambridge University Press.
Leckey J (1987) Curriculum Cobbling: or How Companies can Take Over and Effectively
Use Commercial Materials, in Sheldon L (Ed) (1987)
McDonough J (1998) Survey Review: recent material for the teaching of ESP, English
Language Teaching Journal, 52/2
McGuinness P (1999a) ESP Course Provision: A Needs Analysis Approach, unpublished
MEd assignment
Rea-Dickins P and K Germaine (1992) Evaluation, Oxford: Oxford University Press
Robinson P (1991) ESP Today: A Practitioners Guide, Hertfordshire: Prentice Hall
International
Sheldon L (Ed) (1987) ELT Textbooks and Materials: Problems in Evaluation and
Development, ELT Documents:126, Oxford: Modern English Publications

Surveyed Material
Business Basics (1995) Grant D and McClarty R, Oxford: Oxford University Press.
Business Challenges (1995)
Business Objectives (1996) Hollett V, Oxford: Oxford University Press.
Getting Ahead (1987) Jones-Macziola S and G White, Cambridge: Cambridge University
Press.
International Express (1996) Taylor L, Oxford: Oxford University Press.

14