Documente Academic
Documente Profesional
Documente Cultură
http://rural.edu.ro
e-mail: office@ump.kappa.ro
ISBN 00 000-0-00000-0;
ISBN 00 000-000-0-00000-0.
Tu i poi ajuta!
Tu i poi ajuta!
2007
Dumitru DOROB
THE METHODOLOGY
OF EVALUATION AND TESTING
Dumitru DOROB
2007
Dumitru DOROB
2007
2007
ISBN 978-973-0-04814-8
Table of contents
TABLE OF CONTENTS
Introduction....................................................................................................................... vi
Unit 1
Introduction to Language Testing.................................................................................... 1
1.1 Unit Objectives ............................................................................................................. 1
1.2 Assessment. Testing. Evaluation .................................................................................. 2
1.3 Setting Testing Parameters ........................................................................................... 4
1.4 Participants in Testing .................................................................................................. 5
1.4.1 The Tester .................................................................................................................. 6
1.4.2 The Test Takers/ The Testees.................................................................................... 6
1.4.3 The Test User ............................................................................................................ 6
1.5 The Beneficiaries of Testing ......................................................................................... 6
1.6 The Overall Impact of Testing in Students Motivation .................................................. 7
1.7 Summary ....................................................................................................................... 9
1.8 Key Concepts ................................................................................................................ 9
1.9 Checklist ........................................................................................................................ 9
1.10 Answers to SAQs ...................................................................................................... 10
1.11 Further Readings ....................................................................................................... 10
Unit 2
Conditions of a Good Test.............................................................................................. 11
2.1 Unit Objectives ........................................................................................................... 11
2.2 Principles of Good Practice for Assessing Student Learning ...................................... 13
2.3 Validity ........................................................................................................................ 14
2.3.1 Content Relevance .................................................................................................. 16
2.3.2 Content Coverage ................................................................................................... 16
2.3.3 Face Validity ............................................................................................................ 16
2.3.4 Content Validity ....................................................................................................... 16
2.3.5 Predictive Validity .................................................................................................... 17
2.3.6 Construct Validity ..................................................................................................... 17
2.3.7 Curricular Validity .................................................................................................... 19
2.3.8 Criterion Related Validity ......................................................................................... 19
2.3.9 Concurrent Validity .................................................................................................. 20
2.4 Reliability .................................................................................................................... 21
2.4.1 Measuring Reliability ................................................................................................ 21
2.4.1.1 Test-Retest Method .............................................................................................. 21
2.4.1.2 Parallel Forms of the Test to the Same Group ..................................................... 22
2.4.1.3 The Split-Half Method ........................................................................................... 22
2.4.1.4 Factors that Affect Language Scores .................................................................... 23
2.4.1.5 Test Length............................................................................................................ 26
2.5 Discrimination ............................................................................................................. 27
2.6 Feasibility .................................................................................................................... 28
2.7 Washback ................................................................................................................... 29
2.7.1 Negative Washback ................................................................................................. 30
2.7.2 Positive Washback .................................................................................................. 30
2.8 Summary ..................................................................................................................... 31
2.9 Key Concepts .............................................................................................................. 31
Proiectul pentru nvmnt Rural
Table of contents
Table of contents
iii
Table of contents
Table of contents
Introduction
INTRODUCTION
1. What this course is about
This course is an introduction to the methodology of evaluation
and testing in teaching and learning English as a Foreign Language.
It is obvious to all educators that the issues of grading and reporting
on student learning continue to challenge teachers. However more is
known at the beginning of the 21st century than ever before about
the complexities involved and how certain practices can influence
teaching and learning. This introduction tries to identify grading and
reporting practices that can beneficially influence teaching and
learning. Developing teachers awareness is another area of
interest. The practical side of the course is obvious: to encourage
the design and use of effective techniques in English language
testing. Summarily, the course addresses and answers a number of
questions about testing, helping you to develop a scientific
perspective before you begin using and devising tests.
2. Course objectives
One of the major goals is to assist you in recognizing that the
purposes of measurement and evaluation are good not bad.
Measurement, evaluation and testing are essential to sound educational
decision making. After reading this course you will be able to:
recognize that evaluation and testing are essential to sound
educational decision making;
understand the components of a model of decision making;
recognize the way evaluation and testing can assist in
instructional, guidance, administrative, and research decisions;
have a better understanding of the role of testing in language
teaching;
analyse and assess different kinds of tests;
identify the different purposes of testing;
identify the way in which testing can encourage good
teaching and learning;
learn how teachers can test the main skills, the language
system and beyond;
learn and apply techniques of test construction and
administration;
design tests that can assist/ complete good teaching and
learning;
develop techniques of self-learning;
appreciate the variety of interesting issues in evaluation and
testing that will be covered in subsequent chapters.
vi
Introduction
vii
Introduction
6. Point(s) to Ponder
Points to Ponder include aphorisms and quotations which may
be starting points for personal reflection on various issues/
controversies.
Point to Ponder is signalled by the icon on the left.
Introduction
The unit
containing
the SAA
SAA no. 1
SAA no. 2
Unit 2
Unit 4
1
1
SAA no. 3
Unit 6
SAA no. 4
Unit 8
1
1
1
2
1
2
100%
100%
50%
50%
25%
75%
Weight of
each SAA in
the final
assessment
10%
5%
5%
20%
9.
Further readings
Before starting studying this textbook, I recommend you read
two books in Romanian:
Vagler, Jean (2000) Evaluarea n nvmntul preuniversitar,
translated by Ctlina Grba i Ionela Blu, Iai: Polirom
Pavelcu, Vasile (1968) Principii de docimologie, Bucureti: EDP
ix
Introduction
Week
Unit
Number of
study hours
1
2
3
4
5
6
7
7
8
9
10
11
12
13
14
Introduction
Unit 1
Unit 2
Unit 3
2
4
Unit 4
Unit 5
Unit 6
Unit 7
Unit 8
Revision
TOTAL
Assignment
Number of
hours for the
SAAs
SAA no. 1
SAA no. 2
SAA no. 3
SAA no. 4
2
2
28 hours
11. Appendices
To facilitate your acquisition of the main issues, several
appendices have been added:
At the end of each unit you may find a Summary, a list of
Key words and a Checklist. The Further Reading section
gives you a minimal bibliography, indicating the pages where
you may find the information you need.
At the end of the course your final grade will take into account:
attendance of and contribution to face-to-face meetings with your
tutor and assisted activities, solving of SAQs and SAAs 40%
final examination 40%
portfolio (containing your tests/ your models) 20%
Unit 1
INTRODUCTION TO LANGUAGE TESTING
Point to Ponder
To teach without testing is unthinkable.
The Joint Committee of the American Association of School
Administrators
Testing is a very widespread and common management
strategy if we accept the following:
testing represents the explicit codification of the real goals of a
teaching and learning programme. Contemporary trends in testing
show that this management strategy is rarely the decision of the
individual teacher. Rather, unless we speak about formative
tests, it is passed down from the next administrative level, or even
from the Ministry of Education and Research. But how you feel
about it, and the way you let it affect your attitudes and the
attitudes of your pupils, is still under your control.
marking is a form of assessment. It involves giving the pupils a
grade mark. Any assignment, oral or written can be marked.
Marking is one of the most time-consuming parts of a teachers
job. If you want to cope with all this marking, you have to take into
account a number of options. You may:
correct all errors
be selective in choosing particular errors
correct understanding or mistakes of content
suggest/ require corrections to be done
go over areas of common difficulty with the whole class
see individual pupils about their work
display the best papers
simply put a tick to show it has been read
students should know how they are to be assessed
You should avoid:
building up a back lag of unmarked work
marking down a paper because a pupil misbehaved
SAQ 1
What is measured/ tested beyond the walls of your classroom?
Write your answers in the space provided above (in no more than 60 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
to give feedback to
how your teaching
is going
To maximize
learners
motivation
to provide
feedback and
guide
improvement
WHY?
To provide
statistics for
the school
To
enable
grading
WHAT?
WHEN?
WHO?
4
To add variety
to students
learning
experience
To classify
or grade
learners
To enable
student
progression
To diagnose faults
and provide
students with an
essential tool to
put things right
What is to be evaluated?
Possible answers include some/ the entire course; content;
methodology; participants, teachers etc. Always prepare a table of
specifications function of what you want to evaluate.
When is it to be evaluated? During the course? At the end? Before
and after the course?
Who will evaluate? Teachers? Inspectors? An outside party?
Proiectul pentru nvmntul Rural
HOW?
How will the evaluation be carried out? What form will it take? Will it
be a pen-and-paper instrument or be conducted orally? Will it seek to
elicit quantitative or qualitative data, or both?
Among these questions, the why? and the what? are obviously
of major importance. Also, in the real world, there will be constraints
(such as time, space, resources) that will operate on the parameters
of when?, who? and how?.
Once parameters are defined, it is possible to clarify valid
objects for evaluation and to agree on an appropriate methodology.
For small scale evaluation projects in which time is an important
factor, the most convenient vehicle is the pen-and-paper instrument.
At the next stage, that of instrument design, it is necessary to
consider such questions as validity, format and administration.
Do you evaluate
group work or
individual work?
WHAT DO YOU
REALLY WANT TO
ASSESS?
Is assessment formative
or summative?
Is it teaching or learning
that is being assessed?
1.4
Participants in Testing
The participants in language testing are the:
tester
test taker/ the testee
test user
1.5
Points to Ponder
1.6
MOTIVATION
AND
TESTING
What needs are satisfied if all learners experience success, and get
praise and other reinforcement?
Circle the correct answer. Compare your answer to that in the Answers to SAQs
section at the end of the unit.
Proiectul pentru nvmntul Rural
Be creative. Allow
your students to
write one question
of their own in an
exam
Improving
traditional exam
questions
Keep the
language
simple
Avoid trick
questions
Be sure that
learners
understand the
instructions
Offer opportunities
for practising
under simulated
exam conditions
Build-up confidence
(develop revisions and
exam techniques
Help students
identify their
strengths
Coping with
exam failure
Let students
play exams
See failure as
an opportunity
for learning
Give students
opportunities to
reflect on
unsuccessful
exam
performance
1.7 Summary
This unit aimed at sensitizing the learners about the main issues of
testing. We cannot stop testing. Tests seem inevitable because
they are part of a much larger cultural system. Testing can become
a healthy part of an honest and responsive learning atmosphere.
The following statements summarize the major points of the first
unit:
1. Assessment and testing, measurement and evaluation are
essential to sound educational decision making;
2. The concept of assessment is broader than that of testing.
The same is true about measurement. We can measure
characteristics in ways other than by giving tests.
Who wants tests?
1.9 Checklist
Language Test
Assessment
Measurement
Assessment Criteria
Evaluation
Testing
SAQ 2
10
Unit 2
CONDITIONS OF A GOOD TEST
2.1 Unit Objectives ........................................................................................................... 11
2.2 Principles of Good Practice for Assessing Student Learning ...................................... 13
2.3 Validity ........................................................................................................................ 14
2.3.1 Content Relevance .................................................................................................. 16
2.3.2 Content Coverage ................................................................................................... 16
2.3.3 Face Validity ............................................................................................................ 16
2.3.4 Content Validity ....................................................................................................... 16
2.3.5 Predictive Validity .................................................................................................... 17
2.3.6 Construct Validity ..................................................................................................... 17
2.3.7 Curricular Validity .................................................................................................... 19
2.3.8 Criterion Related Validity ......................................................................................... 19
2.3.9 Concurrent Validity .................................................................................................. 20
2.4 Reliability .................................................................................................................... 21
2.4.1 Measuring Reliability ................................................................................................ 21
2.4.1.1 Test-Retest Method .............................................................................................. 21
2.4.1.2 Parallel Forms of the Test to the Same Group ..................................................... 22
2.4.1.3 The Split-Half Method ........................................................................................... 22
2.4.1.4 Factors that Affect Language Scores .................................................................... 23
2.4.1.5 Test Length............................................................................................................ 26
2.5 Discrimination ............................................................................................................. 27
2.6 Feasibility .................................................................................................................... 28
2.7 Washback ................................................................................................................... 29
2.7.1 Negative Washback ................................................................................................. 30
2.7.2 Positive Washback .................................................................................................. 30
2.8 Summary ..................................................................................................................... 31
2.9 Key Concepts .............................................................................................................. 31
2.10 Checklist ................................................................................................................... 31
SAA 1 ................................................................................................................................ 32
2.11 Answers to SAQs ..................................................................................................... 33
2.12 Further Readings ...................................................................................................... 34
11
12
13
SAQ 1
Read the principles of good practice for assessing pupil learning and
try to write your own assessment Decalogue on a single sheet of
paper.
1. ---------------------------------- --------------------------------------------2. ----------------------------------- -------------------------------------------3. ---------------------------------- -------------------------------------------4. ---------------------------------
--------------------------- ----------------
5. --------------------------------- ---------------------------------------------6. -----------------------------------------------------------------------------7. ------------------------------------------------------------------------------8. ------------------------------------------------------------------------------9. ------------------------------------------------------------------------------10. ------------------------------------------------------------------------------Write 10 principles in the spaces provided above. Your choices depend on your
teaching and learning experience.
2.3 Validity
Validity, the most important quality of a test, refers not only to
the degree to which the test actually measures what is intended to
measure, but also to the adequacy and appropriateness of the way
we interpret and use test scores.
A valid test is one in which a testees score gives a true
reflection of his ability on the trait. Statistical and descriptive means
have been used to check validity. Content analysis of tests
determines:
14
Points to Ponder
If a large number of students do poorly on an exam, reconsider
its worth.
When used to describe a test, the term valid should be
accompanied by the preposition for e.g. This test is valid for
Make the first test relatively easy to build up students
confidence.
Never argue with a student about a grade in front of the class.
Offer to meet him/ her the next day. Give him/ her some time to
cool off first.
If the test scores are reliable, then performance on the test cannot
be affected by measurement errors but by other causes. In examining
validity, we consider the relationship between performance and other
types of performance in other contexts. It also implies:
the uses or interpretation we make of the test results
the value systems that justify a given use of test scores
the educational and social consequences of the uses we make of
tests
In test validation we are not examining the validity of the test content
or of even the test scores themselves, but rather the validity of the
way we interpret or use the information gathered through the testing
procedure (Bachman, 1990: 238)
Reliability is a requirement for validity. A test is not valid unless
it meets the conditions of reliability. The investigation of reliability and
validity are complementary aspects identifying, estimating and
interpreting different sources of variance in test scores. Correlation
between scores on parallel test demonstrates reliability. Correlation
between scores on a multiple choice test of grammar and ratings of
grammar on an oral interview demonstrate validity.
Point to Ponder
Reliability is a necessary but not sufficient condition for validity to be
present i.e. it is possible for a test to be reliable without being valid for
a specific purpose, but it is not possible for a test to be valid without
first being reliable.
Validity is a unique concept. The distinction among content
validity, criterion-related validity and construct validity is inadequate.
Practically, they are complementary types of evidence.
15
Write your answer in the space provided above (in no more than 45 words) and
compare it to that in the Answers to SAQs section at the end of the unit.
17
SAQ 3
You want to test competence in vocabulary and grammar. You
decide to use two kinds of tests: a multiple-choice test and a writing
sample. The scores of multiple-choice tests are highly correlated
with other. The correlation between the multiple choice and writing
tests of grammar is poorer. What is the cause of this lack of
correlation?
Write your answers in the space provided above (in no more than 30 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
19
SAQ 4
Match the threats to Test Validity with the practical examples given
below:
1. Misapplication of tests
2. Standardized proficiency tests, developed from a distinct
population, are administered to subject drawn from a qualitatively
different population.
3. Items do not match the objectives or content of instruction
4. Imperfect cooperation of the examinee
a. The examinees are insincere, misinformed or hostile. They
consider that the test is a waste of time. They respond quickly making
a series of answers which do not at all reflect their opinions.
b. A test of reading comprehension designed to measure
achievement in reading comprehension in accordance with the
syllabus of 4th year high school students applied to measure
achievement of 4th year general school students.
c. TOEFL test is a standardized proficiency test of high validity
20
2. . 3. .. 4. ..
Write your answers in the space provided and compare them to those at the end of
the unit.
2.4 Reliability
If we buy a kilo of fruit, each time we weigh the parcel on the
same scales, we expect to get the same weight. The same thing is
expected from a test. In order to be reliable, a test must be consistent
in its measurements i.e if the test is given to the same learners on
different occasions with no further language lessons between the two
dates, the same scores are obtained.
Points to Ponder
21
22
SAQ 5
Circle T(rue) or F(alse)
1. Reliability is a necessary but not sufficient condition for
validity.
2. Reliabilities of the prediction and criterion measures,
group heterogeneity cannot affect validity.
3. Availability of other data, cost of testing and faulty
decisions, selection ratio, success ratio cannot affect
whether a test is valid enough to be useful in decion
making.
4. Reliability is the degree of consistency between two
measures of the same thing.
5. Reliability will be higher when a test is given to a
heterogenous group.
6. All measurement is subject to error.
23
SAQ 6
If all error of measurement could be removed from a testees
score, what would we call the remaining quantity?
Write your answer in the space provided above (in no more than 15 words) and
compare it to that in the Answers to SAQs section at the end of the unit.
Conclusions
A major concern is to minimize the effects of test method,
personal attributes and random factors that are not part of
language ability
The interpretation and use of language test scores must be
moderated by your assessment (or estimates) of the extent to
which these scores reflect personal, test method or random
features.
Points to Ponder
Are you aware that:
Students with neat handwriting get higher marks on essay
tests?
A halo effect exists in the assignment of grades? Students
who performed well on previous essays tend to be rated
higher on subsequent ones, even if the quality diminishes?
Longer essays get rated higher than better shorter essays?
Students with common names get rated higher than students
with unusual names?
Grades have proven of little value in predicting any criteria of
post-school success in any field? (after Ronald L. Partin)
24
MEAN AND
VARIANCE
Example:
Write your answers in the space provided above (in no more than 50 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
25
When the testee must give the answer, the amount of time depends
on the amount of thinking time and the amount of writing involved.
26
SAQ 8
What general relationship exists between test reliability and the
number of the items on the test?
Write your answers in the space provided above (in no more than 30 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
Write your answers in the space provided above (in no more than 40 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
2.5 Discrimination
Another characteristic of a good test is discrimination i.e. a test
has to have the power to discriminate between testees. This is not a
problem with tests for learners at much the same level (e.g. class
achievement tests). In order to discriminate reliably, the test should
be fairy long. Short tests are not always able to discriminate. The
solution is to require the testees who score highly on a short test
given to the majority to take a further longer extension test that meets
this condition.
Discrimination is also a property of individual items in a test.
Each item should contribute to the discrimination power of the test as
a whole. Item analysis answers the following questions:
Is an item answered correctly by candidates who answer most of
the rest of the items right? (good discrimination)
Proiectul pentru nvmnt Rural
27
D=
Nc
7
=
= 0.64
Nc + Lc 7 + 4
where, D = discriminability,
Nc = the number of correct responses
Lc = the number of correct responses in the low group.
The discriminability for item six is 0.64. A discriminability index of
0.67 is considered the lowest acceptable discriminability by this
method.
SAQ 10
Case study. Half of the testees pass a given item and half fail it. If we
take difficulty into account, we would rate this item as an easy one.
Unfortunately, the testees who passed the item were the weaker half
of the testees, and those who failed the item were the better testees in
the ability being measured. What is your conclusion?
Write your answer in the space provided above (in no more than 35 words) and
compare it to that in the Answers to SAQs section at the end of the unit.
2.6 Feasibility
Something that is feasible can be done, made, or achieved.
This requirement has been made possible by a number of
technological developments, such as
28
SAQ 11
What is affected in 1 and 2 below: reliability or validity?
1. a recording for an oral comprehension test is poor in quality
2. b. The quality of the recording is good. A group hears it played
under good acoustics conditions while another group hears it
under poor conditions.
Write your answers in the space provided above (in no more than 40 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
2.7 Washback
Washback is the effect a test has on teaching in the classroom.
It is true we are always recommended not to teach toward a test.
However, we can use tests as teaching tools. Tests (especially
formative tests) may be used as feedback devices that make
teachers aware of the areas where the learners need improvement.
Formal tests may be channels through which learners can receive a
diagnosis of areas of strengths and weakness. Your prompt return of
written tests with your feedback is a must if you really want to use
washback positively. It is also important to comment upon your
evaluation. Give praise for good strengths and offer constructive
criticism of weaknesses. Give learning hints on how a learner might
improve his performance. Encourage learners to seek clarification
about their grades / scores.
Tests have the power of influencing over the method and
content of language courses. Their backwash effect may be positive
or negative. In their turn, the teachers who use such tests and
testees who suffered the negative backwash may be able to
influence the decision of the testing organizations who respond
positively to positive feedback (e.g. the new TOEFL test reflects this
type of feedback).
Washback can be positive (beneficial) or negative.
Proiectul pentru nvmnt Rural
29
Write your answers in the space provided above (in no more than 30 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
Point to Ponder
On the whole, if learners fail to learn it is the fault of the
teacher, the school, the curriculum, or poor curriculum.
30
2.8 Summary
The principal ideas, conclusions, and implications presented in
the chapter Conditions of a Good Test are summarized in the
following statements:
Reliability is a necessary but not sufficient condition for validity;
Validity can be defined as the degree to which certain
inferences can be made from the test scores (or other
measurements). Since a single test may have many different
purposes, there is no single validity index for a test;
Various factors affect validity;
Reliability is the degree of consistency between measures of the
same thing;
The different methods of estimating reliability consider different
sources of error. Which should be used depends upon how one
wishes to use the results of the test;
In general, longer tests are more reliable;
Reliability will be higher when a test is given to a heterogeneous
group.
construct validity
content coverage
content relevance
content validity
criterion related validity
curricula
discrimination
face validity
feasibility
predictive validity
reliability
reliable variance
the Split Half Method
test length
test retest Method
washback
validity
2.10 Checklist
31
Do you ask your learners to evaluate their own work, and set
themselves target?
Do you set each student achievable goals?
SAA No. 1
This activity aims at reviewing unit 2.
Match the principles (numbers) with their main characteristics
(letters)
I. Principles:
a) face validity; b) practicality; c) authenticity; d) content; e) validity;
f) reliability; g) washback; h) discrimination.
II. Caracteristics:
1. a well - constructed format with familiar tasks; timing is clear;
uncomplicated items; crystal clear instructions; a difficulty level that
presents a reasonable challenge;
2. tasks that relate the course work of the learners
3. spending classroom time after reviewing the content; students
discover their areas of strength and weakness; asking students to
use test results as a guide to setting goals for their future effort;
items can serve in diagnostic capacity;
4. the language in the test is as natural as possible; contextualized
items; tasks represent real-world tasks;
5. objective scoring procedures; classroom conditions are equal
and optimal for all students;
6. te test is not expensive and stays within appropiate time
constraints; relatively easy to administer; a scoring procedure that
is specific and time efficient;
Please not that each correct answer will count for 15 points. 10
points will be given for ordering the principles function of their
importance.
The maximum score for this assignment is 100 points.
Do not forget to send your answers to your tutor in due time.
32
SAQ 2
SAQ 3
SAQ 4
SAQ 5
SAQ 6
33
SAQ 7
SAQ 8
SAQ 9
SAQ 10
SAQ 11
SAQ 12
34
Types of Tests I
Unit 3
TYPES OF TESTS I
3.1 Unit Objectives ........................................................................................................... 35
3.2 Informal Assessment .................................................................................................. 36
3.2.1 Informal Assessment of Speaking ........................................................................... 37
3.2.2 Informal Assessment of Writing ............................................................................... 38
3.2.3 Informal Assessment of Listening ............................................................................ 38
3.2.4 Informal Assessment of Reading ............................................................................. 39
3.2.5 Informal Assessment of Non Linguistic Factors .................................................... 39
3.2.6 Informal Assessment of Grammar and Vocabulary ................................................. 39
3.3 Formal Assessment - Types of Tests and Testing ...................................................... 40
3.3.0 Classification by Stimulus Material .......................................................................... 40
3.3.1 The purpose, or use, for which they are intended i.e. the types of decisions to be
made function of the scores .................................................................................... 41
3.3.1.1 Selection Tests ..................................................................................................... 41
3.3.1.2 Entrance Tests ..................................................................................................... 43
3.3.1.3 Readiness Tests ................................................................................................... 43
3.3.1.4 Placement Tests ................................................................................................... 43
3.3.1.5 Diagnostic Tests ................................................................................................... 44
3.3.1.6 Progress Tests ..................................................................................................... 45
3.3.1.7 Achievement/ Attainment Tests ............................................................................ 46
3.3.1.8 Mastery Tests ....................................................................................................... 46
3.3.2 Function of Content ................................................................................................. 48
3.3.2.1 Proficiency Tests .................................................................................................. 48
3.3.2.2 Achievement or Attainment Tests.......................................................................... 52
3.3.2.3 Aptitude or Prognostic Tests ................................................................................. 53
3.3.3 The frame of reference ............................................................................................ 54
3.3.3.1 Norm-Referenced Tests ....................................................................................... 55
3.3.3.2 Criterion Referenced Tests................................................................................. 56
3.4 Summary .................................................................................................................... 57
3.5 Key Concepts ............................................................................................................. 58
3.6 Checklist ..................................................................................................................... 58
3.7 Answers to SAQs ....................................................................................................... 58
3.8 Further Readings ........................................................................................................ 60
35
Types of Tests I
a)
b)
c)
d)
Where?
What?
How?
Why?
36
Types of Tests I
SAQ 1
Which of the following items are assessed informally or formally?
Circle in the margin I (informally) or F (formally)
Written homework
Written grammar activities
Speaking
Projects
Portfolios
Listening tasks
Reading tasks
Writing tasks
Vocabulary activities
Attitude / effort
Participation in class
Group work
Pair work
Organization of work
Presentation of work
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
F
F
F
F
F
F
F
F
F
F
F
F
F
F
F
SAQ 2
What kind of assessment is favoured by teachers? Circle in the margin
I (informally) or F (formally)and compare your answers to those at the
end of the unit.
A. who teach small classes
I F
B. who teach small classes and have
more than two hours a week
I F
C. teach large classes and have
two hours of English each week
I F
Circle in the margin I (informally) or F (formally). Compare your answers to
those in the Answers to SAQs section at the end of the unit.
Proiectul pentru nvmnt Rural
37
Types of Tests I
Circle the letter in the margin L (lower mark), H (higher mark). Compare your
answers to those in the Answers to SAQs section at the end of the unit.
38
Types of Tests I
39
Types of Tests I
40
Types of Tests I
3.3.1 The purpose or use for which they are intended i.e. the types of decisions to
be made function of the scores
3.3.1.1.
Selection Tests
A selection test is a special form of placement test. It excludes
learners who are below a certain percentage. It selects candidates
for a particular job or course of study (success or failure depend on
the number of places available). Proficiency tests are often used for
selection.
The true-false test is the most popular among the selection type
examinations with classroom teachers.
Weaknesses:
Its fifty fifty chance of guessing the correct answer encourages
students to guess wildly
It does not discriminate well between those examinees receiving
the highest score on the total test and those receiving the lowest
score
It is not as reliable as a multiple choice test of equal length
It is quite difficult to develop statements which can be answered
absolutely true or false
It is seldom applicable to the measurement of complex
understandings and other higher order mutual processes
Strengths. The true-false test may be:
Rapidly and accurately scored by individuals unqualified to teach
the subject matter area being examined
The scoring is completely objective
Extraneous factors have no influence on test scoring
It can be administered relatively quickly (less time per item is
required to answer true-false questions in compositions with any
other item type)
It takes less time to construct and refine the items
The item statement need not include instructions on how to
respond
It is very useful in situations where the measurement of the
acquisition of factual, non interpretative information is desired
(vocabulary, technical terms, formulae, dates, proper names)
Construction
Select from the table of specifications the areas that can be
successfully tested by the true-false test
Write each item on a separate 3x5 piece of paper (it is easier to
place the items in the desired order on the test)
The true false item consists of: a statement, a disagreement with
the statement
The testee is instructed to mark the statement true or false, right or
wrong, yes or no at the beginning of the test
41
Types of Tests I
Examples:
Example:
SAQ 4
Rewrite the following items:
1. In Blakes The Lamb, the lamb does not stand as a symbol
of a child.
2. Only a few men have been elected presidents of the US, after
having been defeated for that office.
Corect the items in the space provided above and compare them to those in
the Answers to SAQs section at the end of the unit.
Types of Tests I
3.3.1.2.
Entrance Tests
They are used to admit pupils to a certain school. They protect
admitting institutions and student funding agencies from too high a
failure rate. Tests are used by universities and other educational
institutions to assess the proficiency and predict the readiness of
applicants to benefit from instruction given in the foreign language.
Examples of entrance tests are ACT (The American College
Testing Programme), SAT (The Scholastic Aptitude Test) that is
required for admission to many colleges in the USA. Applicants to
law schools and medical schools must pass special admission tests
e.g. LSAT (The Law School Admission), MCAT (The Medical College
Admission Test)
3.3.1.3.
Readiness Tests
They assess whether a child is ready to benefit from instruction
in general or from instruction aimed at acquiring a certain skill e.g.
reading readiness.
3.3.1.4.
Placement Tests
Closely related to the notions of diagnosis and selection is the
concept of placement. A placement test is a test which is designed to
place students at an appropriate level or stage in a programme or
language course. Such tests are used to assign learners to groups at
different levels. The term refers only to the purpose for which it is
43
Types of Tests I
Diagnostic Tests
Diagnostic tests are designed to show what knowledge or skills
a learner knows and does not know i.e. the strengths and
weaknesses in learning abilities of the students. As they try to find
out problem areas, diagnostic tests are important for teachers in
order to design mastery learning and work out remedial activities.
The data are also useful for self-assessment. For example a
pronunciation test may become a diagnostic test if it tries to identify
which sounds a learner is or is not able to pronounce. Few tests
serve only as diagnostic tests. Achievement and proficiency tests
may be useful for diagnostic purposes.
Areas of focus that may serve for diagnostic purposes
Phoneme discrimination tests
Grammar and usage tests
Controlled writing tests
They are usually used at the beginning of a language course.
Diagnostic tests are based on error analysis and deficiency analysis
(on learners language deficiencies).
Diagnostic tests
offer feedback to the learners
are set after about eight hours of instruction
are not longer than 15
can be marked by the learners themselves
motivate
reduce anxiety about later summative tests
quickly diagnose errors
prevent compound errors of learning (a weeks poor learning
makes next weeks learning all the more difficult)
are not used for grading or judging
Corrective help
mastery learning also involves a self-correcting system
retakes are allowed
learners are advised to use appropriate instructional materials
correction in group after the test
out-of-class meeting to clear up difficulties
encouraging family, friend to help
corrective learning and retesting continues until mastery has been
achieved
44
Types of Tests I
3.3.1.6.
Progress Tests
A progress test is a small scale test (quiz) and it is an
achievement test linked to a textbook/ a set of teaching materials.
Progress tests are tests prepared by a teacher on the basis of a
textbook/ curriculum/ a particular course of instruction given at the
end of a unit, chapter, course or term. Besides being more
specifically focused, they are narrower in scope than achievement
tests. They are usually designed by the class teacher who can fully
take into consideration the knowledge of the learners, the
programme which they have been following, his/ her own particular
aims and goals.
Teachers should learn how to construct such tests as they are
extremely useful.
Such tests
are based on the language programme or curriculum (textbook,
workbook) which the class has been following
assess learning and teaching
familiarize the teacher with the progress of each of his students
and of the whole class
have positive and motivating backwash effect
reinforce what has been taught
allow the learners to show what they have learned
show high scores if progress has been made
do not require a wide range of performance as in the case of
standardized achievement or proficiency tests (Gausss bell/
shaped curve or the Gaussian curve/ normal curve showing a
distribution of probability associated with different values of a
variant is not valid in this case).
Progress tests are widely used, as they try to measure the
extent to which the pupils have learned what has been taught. They
are usually constructed by the class teacher who can fully evaluate it,
taking into account:
Types of Tests I
Mastery Tests
Formative Assessment refers to the process of providing
information to curriculum developers during the development of a
curriculum or programme. It is also used in syllabus design and the
development of language teaching programmes and materials. A
formative test is given during a course of instruction. It provides
feedback to the teacher and the student. It tests only what has been
taught. The score shows whether the student needs extra work. It is
a pass or fail. If a person fails he or she is able to do more study and
take the test again. All tests in our schools should be criterion
referenced. Criterion referencing is ideal for mastery objectives.
In order to avoid the usual 30 to 40 percent failure, teachers
should adhere to formative assessment by
allowing pupils as much individualized instructions as they feel
they need;
allowing them as much practice as they feel they need;
defining the skill the learners need in order to pass;
46
Types of Tests I
47
Types of Tests I
WHAT DOES
MASTERY
LEARNING
IMPLY?
SAQ7
What are the differences between a progress test and an achievement
test?
Write your answers in the space provided above (in no more than 20 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
Types of Tests I
49
Types of Tests I
Circle the answer. Compare it with that in the Answers to SAQs at the end of the
unit.
50
Types of Tests I
COMMUNICATIVE
COMPETENCE
COMMUNICATIVE
TESTS
51
Types of Tests I
b) Show you are annoyed. Use some phrases which are strong and
firm but without swearing.
Answers:
a) Come off it!
Youre pulling my leg.
Thats not true.
You are having me on.
You cant really mean that!
b) Push off!
Get lost!
Ive had enough! Just stop it.
Leave me alone!
Ive already told you I dont want to discuss that.
Who on earth told you that?
Where on earth have I put my keys (books) etc?
SAQ 9
Identify the main characteristics of proficiency tests in the modern
sense of the word. Tick the correct statements:
Proficiency tests:
a. are based on a language syllabus
b. look forwards
c. look backwards
d. are based on an analysis of the language learner
e. has positive washback effect
f. have a negative washback effect
g. are standardized
h. are made by teachers
i. are used for diagnostic purposes
j. are used for certificates
k. are used for selection
Compare your answers with those in the Answers to SAQs at the end of the unit.
Types of Tests I
Write your answers in the space provided above (in no more than 60 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
LANGUAGE
APTITUDE
53
Types of Tests I
LANGUAGE
APTITUDE
TESTS
Write your answers in the space provided above (in no more than 60 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
Types of Tests I
55
Types of Tests I
SAQ 12
The following list summarizes the chief objectives of language testing:
1. to determine the readiness for instructional programmes
2. to classify or place individuals in appropriate language classes
3. to diagnose the individuals specific strengths
4. to measure aptitude to learning
5. to measure the extent of student achievement of the
instructional goals
6. to evaluate the effectiveness of instruction
Group these six categories under three headings:
1. Aptitude test
2. General Proficiency test
3. Achievement test
Compare your groups with those in the Answers to SAQs at the end of the unit.
Weaknesses
DISADVANTAGES
OF CRT
56
Types of Tests I
bright students, who easily attain the level of mastery, may not be
motivated to reach high standards
the results do not inform decision makers whether children achieve
what they should when they should
SAQ 13
Suppose you give 200 learners a test, choosing the best 40% to attend
a good high school, and the next 60% to attend a vocational school.
Is this test. Tick the right answer.
criterion-referenced
norm-referenced
motivating
demotivating
Compare your answers to those in the Answers to SAQs section at the end of the
unit.
Points to Ponder
3.4 Summary
This unit has been concerned with informal assessment and formal
assessment. Categories of tests have been introduced taking into
account the purpose, the content of the tests and the frame of
reference within which their scores are to be introduced. The following
types of tests have been introduced: selection tests, entrance tests,
placement tests, diagnostic tests, achievement/ attainment tests,
mastery tests, proficiency tests, aptitude or prognostic tests, norm
referenced tests and criterion referenced tests.
Norm-referenced tests are used to interpret a score of an individual
by comparing it with those of other individuals. Criterion-referenced
tests are used to interpret a persons performance by comparing it to
some specified behavioural criterion.
57
Types of Tests I
3.6 Checklist
SAQ 2
SAQ 3
58
Types of Tests I
SAQ 4
SAQ 5
SAQ 6
SAQ 7
SAQ 8
SAQ 9
SAQ 10
SAQ 11
59
Types of Tests I
SAQ 13
3.8
Further Readings
60
Types of Tests II
Unit 4
TYPES OF TESTS II
4.1 Unit Objectives ........................................................................................................... 61
4.2 Formal Assessment - Types of Tests and Testing ...................................................... 62
4.2.1 Scoring Procedures ................................................................................................. 62
4.2.1.1 Subjective Tests .................................................................................................. 63
4.2.1.2 Objective Test ....................................................................................................... 63
4.2.1.3 Performance Tests ............................................................................................... 66
4.2.2 The Specific Technique or Method They Employ .................................................... 67
4.2.2.1 Multiple Choice, Completion, Dictation, Cloze Tests ............................................ 67
4.2.3 The Approach to Test Construction ......................................................................... 79
4.2.3.1 Direct Tests .......................................................................................................... 79
4.2.3.2 Indirect Tests ........................................................................................................ 79
4.2.4 Function of the Number of Elements Tested at a Time ............................................ 79
4.2.4.1 Discrete Point Tests .............................................................................................. 79
4.2.4.2 Integrative Tests ................................................................................................... 79
4.2.5 Speed Tests vs. Power Tests .................................................................................. 80
4.2.6 Other Test Categories ............................................................................................. 80
4.3 Self Assessment ...................................................................................................... 80
4.4 Standardized Tests ..................................................................................................... 85
4.5 Summary .................................................................................................................... 88
4.6 Key Concepts ............................................................................................................. 88
4.7 Checklist ..................................................................................................................... 88
SAA 2 ............................................................................................................................... 89
4.8 Answers to SAQs ....................................................................................................... 89
4.9 Further Readings ........................................................................................................ 91
61
Types of Tests II
62
Types of Tests II
SAQ 1
What is more subjective?
The scoring of an essay
The scoring of short answers in response to questions area
reading passage
Circle your answer. Compare your choice with that in the Answers to SAQs section
at the end of the unit.
4.2.1.1.
Subjective Tests
A subjective test requires scoring by opinionated judgment on
the part of the scorer. An example might be the scoring of free,
written compositions for the presence of creativity as no definition of
creativity is provided. Many tests, such as cloze tests, permitting all
grammatically acceptable responses to systematic deletions from a
context, lie between the extremes of objectivity and subjectivity. It is
true, however, that some subjective tests may be objectified in
scoring. In this case, you have to use a precise rating schedule
clearly specifying the kinds of errors to be quantified or through the
use of multiple independent raters.
4.2.1.2.
Objective Tests
Objective tests can be marked without the use of the
examiners personal judgment. The correctness of the testees
response is determined entirely by predetermined criteria:
examinees responses are compared with a scoring key.
STRENGTHS
OF
OBJECTIVE
TESTS
63
Types of Tests II
WEAKNESSES
OF
OBJECTIVE
TESTS
CONSTRUCTING
OBJECTIVE
TESTS
64
Types of Tests II
1. Conversions
Helen is a very good learner of English.
Helen learns .
Change the following sentences into questions:
I am a student.
We can work together.
2. Gap filling
He will come at half eight.
.high-speed computer is always expensive.
I always take magazine with me.
the United States, 30 million people have successfully kicked
the habit of smoking.
1964, and only one three Americans now smoke.
3. Combination
65
Types of Tests II
Performance Tests
Performance tests are tests of skill the skill with which
learners can identify objects, manipulate objects, perform assigned
tasks, or react to simulated situations. Tests of performance are often
found in tests of typing speed, in simulated situations tests (the
typing of business letters, personal letters). The technique is often
used to assess skills in areas such as chemistry, physics, and foreign
languages. Performance testing has long been neglected from the
classroom testing as teachers are preoccupied with tests of verbal
behavior. Performance tests are:
used to measure the effectiveness of final behavior
used at early grade levels, before paper and pencil tests can be
effectively used (handwriting performance tests; sentence to be copied)
With very young examinees or with those who cannot write, it
may be necessary to use an oral response format. Identification tests
include tasks that may be presented orally, visually from a
reproduction on the exam paper. Examinees are asked to respond
orally and in writing. In the latter case, responses may be short
answers, completion, multiple choice ad matching.
SAQ 2
Which is generally more suitable for each of the following language
areas: an objective or subjective test?
Language
area
Objective
Subjective
pronunciation
vocabulary
grammar
discourse
listening
speaking
reading
Writing
Tick your answers . Compare them to those in the Answers to SAQs section at
the end of the unit.
66
Types of Tests II
Point to Ponder
Darwin concluded that he needed to keep a notebook and pencil with
him at all times, as he found that he remembered evidence in favour of
his theories, but quickly forgot evidence against them! We too tend to
have selective memories about our students work and behavior.
Geoffrey Petty
SAQ3
Objective testing uses a variety of techniques. Classify them into:
discrete point technique and integrative techniques.
Techniques
Discrete point
techniques
Integrative
techniques
transformation
Fill in the blanks
Blank and cue
Joining element
Replacing elements
Adding elements
Arranging elements
Matching elements
True/ false
Multiple choice
Cloze
dictation
Information transfer
Tick your choices in the space provided and compare them to those in the Answers
to SAQs section at the end of the unit.
67
Types of Tests II
68
Types of Tests II
Write your answers in the space provided above (in no more than 30 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
69
Types of Tests II
Strengths
The multiple-choice test is the most flexible and versatile of all
selection-type examinations. It can be used to measure instructional
objectives at all levels of the cognitive domain. (i.e. knowledge,
comprehension, application, analysis, synthesis, and evaluation). It
can also be extremely versatile: multiple choice tests may be
designed for all subject matters and with learners at all grade levels.
Example:
CONSTRUCTING
MULTIPLE
CHOICE TESTS
70
Principles of construction
Each multiple choice item should have only one answer
(absolutely correct) although some instructions require choosing
the best option. There must be no ambiguity in the choices.
Test one feature at a time (it is less confusing for the testee and it
helps to reinforce a particular teaching point). Items that test more
Proiectul pentru nvmnt Rural
Types of Tests II
Examples:
1. If she .. (understand) the situation, she would explain it to
us.
The solution is understood as it is required by would explain.
2. Sometimes an adverbial expression should be used:
He asked me how I was every time I . (see) him.
They . (live) in Europe for years when I first met him.
In constructing the items, use sentences that learners might
encounter or wish to use.
SAQ 5
Why is the following item wrong? Revise it.
American colonists were given the same rights as other Englishmen by:
a. local governors
b. 1542
c. charters
d. Parliament
Rewrite the above item. Compare it to the one in the Answers to SAQs section
at the end of the unit.
Proiectul pentru nvmnt Rural
71
Types of Tests II
The Stem
The stem is the first part of a multiple choice item. The task has
to be clear and concise. No irrelevant details should be included.
After reading the stem, the testee should be able to identify what
exactly the requirements are. The wording should be very clear.
Options such as all of these, none of these should be avoided. Be
careful with negative questions.
The stem may take the following forms:
a. an incomplete statement
Example:
Example:
People think of salt mostly as a seasoning for food, but this use
accounts for less than five percent of the worlds salt production.
A. exclusively
B. nutritionally
C. mainly
D. necessarily
c. a passage
Example:
Example:
72
Types of Tests II
Example:
The item:
I enjoy children playing football in the park.
A. looking for
B. looking to
C. looking about
D. looking at
E. looking on
May be re-written
I enjoy looking the children.
A. for
B. to
C.about
D. at
E. on
SAQ 6
Analyze the following badly-constructed multiple choice items:
1. What was the ring made of?
A. it was made of gold
B. it was made of iron
C. it was made of cotton and rope
D. it was made of light wood
2. Puts his own desires first:
A. egoist
B. egotist
C. altruist
Rewrite the above item. Compare it to the one in the Answers to SAQs section
at the end of the unit.
Types of Tests II
Do not write long stems (not more than 50 words, sentences not
longer than 15 words)
Avoid the use of conditionals
Be careful to choose between: which is the best answer/ which is
the correct answer ( do not forget that multiple choice questions
may have more than one answer)
A good testee should be able to give an answer without seeing the stem
A stem should not include general instructions
Include as much of the problem as possible in the stem, so that the
options can be kept short
Example:
74
Types of Tests II
SAQ 7
Analyze the following distractors:
1. Intimating oneself in anothers good graces.
A. extenuating
B. ingratiating
C. superseding
Circle A,B or C. Compare your answer to that in the Answers to SAQs section at
the end of the unit.
Test Instructions
It is extremely important for the students to clearly understand
the question format. If they do not understand, then we do not
measure what we want i.e. the instructional objectives.
Instruction may be oral, although a combination of written and
oral instructions is probably desirable. The instructions should be
clear, concise and explicit. The instruction should be accompanied by
examples of each type of items. Be extremely careful when an item
type occurs for the first time. Encourage the students to ask
questions.
Test Layout
The layout influences the speed and accuracy of the
examinee. Hints:
use all the space available without hindering readability
make it easy for the examinee to keep track of his place in the
examination
a two-column page may be the best layout for multiple choice or
true false items
if you use various item types in the same examination, group
together the same item types: true false items, multiple choice,
completion. In this way, you reduce the number of shifts in mental
orientation
do not use more than two or three item types on one hour
examinations
in order to reduce test anxiety, arrange test items in order from the
easiest to the most difficult
ordering the items on the basis of their content
Readability is increased if
each item is completed in the columns and on the page in which it is
started
reference materials (paragraphs, graphs) should occur on the same
page as the item
the items that refer to the same reference material should be placed
in the same page, separated from other unrelated terms by dotted
lines
Proiectul pentru nvmnt Rural
75
Types of Tests II
if you use Arabic numbers for the items, use letters to differentiate the
alternatives
the labels on the test should correspond with the labels on the answer
sheet
Formats
Writing Multiple - Choice Tests
A typical multiple choice test will look like the following:
1. Write the correct option in full in the blank space:
The practice of making excellent films based on rather obscure
novels has been going on so long in the United States .
constitute a tradition.
a. being
b. as to
c. so that
d. could
2. Write only the letter of the correct option in the box.
She eating breakfast.
A. is
B. has
C. does
D. no extra word
3. Why that dog following us?
A. is
B. has
C. does
D. no extra word
4. Underline the correct option:
What .. your father do?
A. is
B. has
C. does
D. no extra word
5. Put a circle round the letter at the side of the correct option
As a result of in physics and chemistry, scientists have been
able to make important discoveries in biology and medicine.
A. there is more knowledge
B. what is now known
C. knowing now that
D. known now
The correct option should appear in each position (e.g. A, B, C,
D, E) approximately the same number of times in a test, or the
options may be placed in alphabetical order the first word in each
option. However, figures, dates should be kept in chronological order.
William Shakespeare was born in
A. 1564
B. 1592
C. 1603
76
Types of Tests II
D. 1616
Circle in the margin the letter corresponding to the correct form to
complete the following sentences:
A. is
B. has
C. does
D. no extra word
1. She having lunch.
A BCD
2. What that phrase mean?
A BCD
3. he driven that car before?
A BCD
4. What your mother do?
A BCD
5. John never seen snow.
A BCD
6. Who always knows the answer?
A BCD
7. Why that dog following us?
A BCD
8. Jenny usually eat lunch at school.
A BCD
9. When the bus come?
A BCD
10. What your mother making?
A BCD
Read and circle in the margin the letter corresponding to the tense
and aspect of the verb that you would use to fill in the sentence:
A. simple past
B. past continuous
C. present perfect
D. past perfect
1. He discovered that he (lose) his money
ABCD
2. They came just to (get) breakfast.
ABCD
3. He asked me how I was every time (see).
ABCD
4. The cake would have been better if it (stay) in the oven longer. A B C D
5. While we (wait) for the train, we heard a terrible noise.
ABCD
6. If she (understand) the case, she would explain it to us.
ABCD
7. They (live) in France six months when I first met them.
ABCD
8. Thats the best play this year.
ABCD
Circle in the margin the letter corresponding to the lost appropriate
preposition:
A. back; B. along; C. hrough, D out; E. off; F. up
1. Was he rude? Yes, he told me to get
ABCDEF
2. If you lend him money, youll never get it
ABCDEF
3. Although we were relatives, we didnt get
ABCDEF
4. Be sure to get the tram at the third stop.
ABCDEF
5. I liked the first part of the movie, but I cant get the second. A B C D E F
Circle in the margin the letter corresponding to the word which
correctly completes the sentence:
1. The sister of your father or mother is your
ABCD
A. great aunt
B. uncle
C. stepsister
D. aunt
Proiectul pentru nvmnt Rural
77
Types of Tests II
ABCD
ABCD
Cloze Tests
CLOZE
FROM
CLOSURE
A CLOZE
TEST AN
INTEGRATIVE
TEST
78
Types of Tests II
Compare your solutions to those provided in the Answers to SAQs section at the
end of the unit.
Types of Tests II
Write your answers in the space provided above (in no more than 60
words) and compare them answers to those in the Answers to SAQs
section at the end of the unit.
4.2.6 Other Test Categories
Examinations vs. quizzes
Questionnaires
Rating schedules
Single stage and multiple stage tests
Language skills tests
Language feature tests (verb tense/ aspect/ voice; subject/ verb
agreement; modifiers, comparatives, superlatives, relativization,
embedding)
Memory span tests
Sentence completion tests
Word association tests etc
Types of Tests II
practitioner. Ask questions: What were the main difficulties? Ask students
to draw up an assessment checklist which will aid reflection.
Point of Ponder
Research shows that students are generally quite accurate in their
self-scoring.
Students are often harsher in evaluating their own performance
than the teacher would be. Provide self-assessment questions.
Provide students with model answers after they have completed the
worksheet. The students can then use these to mark their own or
each others work. Self-assessment can be:
Low heat: self-worked tests, quizzes
High heat: presentations or exams
Example:
81
Types of Tests II
Point to Ponder
The ultimate goal of the educational system is to shift to the
individual the burden of pursuing his own education.
J.W. Gardener
SELFASSESSMENT
A WAY TO
STUDENT
AUTONOMY
AUTONOMOUS
LEARNING
SKILLS
82
Types of Tests II
Types of Tests II
Techniques of self-assessment
You can involve your learners in self-assessment if you ask
them to write reports about their English and give them to you, if you
learn their problems from their diaries, if you involve them in rating
their skills in English, if they monitor their language when they edit
their essays, when they use your correction codes, if you ask them to
grade their mistakes. They may also list difficulties (pronunciation
problems), favorite activities, organize group and class surveys to
find out about their learning preferences and problems. You may
begin by distributing a questionnaire to learn how they feel about
their English e.g.
1. Learning English is (difficult, easy, very difficult)
2. Which of these areas of English are easiest for you? Rank them
staring from the easiest area. (speaking, listening, writing, reading,
grammar, vocabulary)
3. Give yourself a mark
4. Is English useful?
5. Do you try to speak English with your class mates?
You may also gather information with the help of th following
questions:
1. Why do you learn English?
2. How do you learn English?
3. How do you tackle an unknown text?
4. What is it to learn a word?
Point to Ponder
You learn from your own mistakes only when you think about them.
Michael Hermis and Paul McCann
Self-assessment
84
Types of Tests II
Points to Ponder
85
Types of Tests II
4. Limited user;
3. Extremely limited user;
2. Intermittent user;
1. Non user.
USA Tests
General English
Placement test
Types of Tests II
Teaching English
SAQ 11
1. Objective-type tests can use either the one correct answer
or the best answer format. Which one would you use? Why
would you use this type over the other?
2. If you were preparing a true-false test would you have more
true than false items? Why/ why not?
Write your answers in the space provided (not more than 60 words) and compare
them to those in the Answers to SAQs section at the end of the unit.
87
Types of Tests II
4.5 Summary
The principal ideas, conclusions, and recommendations
presented in this unit are summarized in the following
statements:
1. Objective tests must be written as simply and clearly as
possible so that all examinees will be able to make the same
interpretation of the items intent
2. Test items should be tailored to fit the age and ability level of
the examinees
3. Technical jargon, and excessively difficult vocabulary should
be avoided
4. Irrelevant clues should be avoided
5. Trivial details should be avoided (otherwise, we encourage
rote memory)
4.7 Checklist
Cloze test
Criterion referenced test
Diagnostic test
Direct test
Discrete point test
Formative evaluation
Indirect test
Integrative tests
Norm referenced test
Objective test
Objectives referenced test
Power test
Speed test
Standardized test
Subjective test
Summative evaluation
88
Types of Tests II
SAA No. 2
Write a multiple choice test made up of 20 items. Use as
distracters some of the mistakes made by your pupils.
Please note that a corect item will count for 5 points.
Do not forget to send your multiple choice test to your tutor.
SAQ 2
SAQ 3
Objective
Subjective
X
X
X
X
X
X
X
X
Discrete point
techniques
X
X
Integrative
techniques
X
X
X
X
X
X
X
X
89
Types of Tests II
Cloze
dication
Information
transfer
SAQ 4
X
X
X
SAQ 5
SAQ 6
SAQ 7
SAQ 8
90
Types of Tests II
SAQ 10
SAQ 11
91
Unit 5
TESTING THE LANGUAGE SKILLS I
5.1 Unit Objectives ............................................................................................................92
5.2 Testing Speaking ........................................................................................................92
5.2.1 What Is Speaking? ....................................................................................................93
5.2.2 Types of Speaking Based on Content and Function .................................................93
5.2.3 Objectives ................................................................................................................94
5.2.4 Types of Speaking Tests ..........................................................................................95
5.3 Testing Listening .......................................................................................................101
5.3.1 How Do We Comprehend? ....................................................................................102
5.3.2 Micro Skills .............................................................................................................102
5.3.3 Informal Evaluation ................................................................................................103
5.3.4 Scoring the Listening Test.......................................................................................106
5.4 Summary ...................................................................................................................110
5.5 Key Concepts ............................................................................................................110
5.6 Checklist ...................................................................................................................110
5.7 Answers to SAQs ......................................................................................................111
5.8 Further Readings .......................................................................................................112
93
to set tasks that are representative samples of the oral tasks that
we expect students to be able to perform
tasks should elicit behavior which truly represents the candidates
ability and which can be scored validly and reliably.
the testing of speaking is the most difficult of all language tests to
design, administer, and score. Why?
it is difficult to choose the criteria in evaluating speaking: which is
more important grammar, vocabulary, pronunciation, fluency,
listening comprehension, correct tone (fear, anxiety), reasoning
ability, and initiative in asking clarification? How can we evaluate
properly each of these criteria? Shall we also include among them
questions of response?
Other difficulties
SPEAKINGA DIFFICULT
SKILL TO TEST
94
Situations
Learner speaks to:
assessor,
learner and assessor,
interlocutor and assessor
THE NATURE
OF SPOKEN
LANGUAGE
Point to Ponder
Evaluation can be done surreptitiously, and it can be done
with flags and trumpets; but it must be done, otherwise the teacher
will not know if learning is taking place
(Geoffrey Petty)
5.2.4 Types of Speaking Tests
reading aloud (problem solving working in pairs)
oral interview the interviewer tends to intimidate the learner or to
dominate the interaction; reduced reliability
The Speaking Tests
limited interaction
lack of authenticity
poor washback effect
doubtful test security (who is the author of the grading and what is
graded is difficult to assess)
limited sample
Linguistic speaking tests have in view the assessment of stress
patterns, intonation, grammatical structure, range of lexical units.
Examples:
95
GUIDED
TESTS
Example:
Example:
Reading Aloud
The testee reads aloud to the assessor a passage of a text, a
dialogue (one of the parts is read by the interviewer), a specialized
technical English text, a descriptive passage, instructions (how to
cook a dish or giving instructions by phone), retelling a story
Variants:
Reading scripted dialogue with someone else reading the other part
Reading text with phonetic markers (sounds, words, technical
vocabulary, idiomatic or conversational expressions), speech
factors (assimilation, liaison or contractors), words or sounds that
are known to cause problems for speakers of a certain language
Reading sentences containing minimal pairs
Spelling aloud (if testees apply for a job in travel agency or for
orders by phone)
Reading from a table figure, abbreviations or initials in different
quantities
SAQ 2
Tests of speaking do not test objectively more than two of the four
components of communicative competence. Circle the correct
competences:
a.
b.
c.
d.
linguistic competence
discourse competence
sociolinguistic competence
strategic competence
Compare your choices to those in the Answers to SAQs section at the end of the
unit.
Advantages
The assessor may choose the topic
The same test may be given to all testees (higher reliability)
Simple to administer and quick to score
Correct production of sentences stems and intonation patterns
suggest a good comprehension
Disadvantages
The technique is not authentic (we rarely read aloud in real life)
Proiectul pentru nvmnt Rural
97
1.
2.
3.
4.
5.
6.
Other examples:
advertisements
pictures for comparison
pictures for instructions
if the picture depicts a story or sequence of events, it is useful to
give the testee one or two sentences as a starter
oral interview
question and answer ( disconnected questions are graded in order
of increasing difficulty; suitable for lower levels; easy to adapt
questions to suit level)
Point to Ponder
Whatever the homework, if it is set it must be seen, marked or
tested by the teacher, otherwise it will be evaded.
Individual Elicitation
Research and specialized testing have devised various ways of
eliciting specific parts of the language. One technique is to use
pictures. The tester talks about part of the picture (for example using
a singular noun, or present tense verbs or questions forms). The
learner responds and the tester notes if the expected plural, past
tense verb, or the inflected form of the verb was present.
Interviews
Interviews are a direct, face-to-face exchange between testee and
interviewer.
Advantages
they are structured
the interviewer maintains firm control, keeps the initiative
more authentic
several topics may be raised
98
Disadvantages
the testee sees the assessor as a superior (the result is only one
style of speech)
many functions are absent
only at intermediate level or below
a candidate may dominate another
Stages
Example
99
SAQ 3
Oral interviews are criticized for at least 3 reasons. Can you identify
them?
Write your answers in the space provided above (in no more than 10 words) and
compare them answers to those in the Answers to SAQs section at the end of the
unit.
SAQ 4
What should you test in order to encourage oral ability?
Write your answers in the space provided above (in no more than 60 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
101
Write your answers in 20 words in the space provided and compare them to
those in the Answers to SAQs section at the end of the unit.
102
103
PROCEDURES
104
the selection once. Weather report. Thank you for calling weather
line. Currently at the downtown weather forecast station the
temperature is 24 degrees. And now our forecast. Overnight, clear
skies and lows in the upper 10s. Clear and sunny tomorrow
morning, with highs around 28. Watch for afternoon showers with
temperature dropping into the low 10s. Questions in the written
form are in front of the testee: What is the current temperature?
What will the high temperature be tomorrow?
Individual responses. In one to one setting you can talk about
pictures and have the student point, you can make reports and
see if the learner can follow them, or if the level is high enough,
you can have conversational interchanges and see if the learner
can respond to what you are saying. I you want to know about
comprehension, you will do most of the talking ands require rather
simple, but unambiguous responses.
Simple paper and pencil test. Normally, you should pre-record
the tests if you want to have objective results i.e. comparable
results with different groups. Be careful! In giving live cues you are
very tempted to respond to the non verbal feedback of the group
and vary your production considerably.
True false. Provide a series of statements that must be
comprehended for their general meanings. They are clearly true or
false. The testees mark a standard answer sheet.
Pictures. You say something about one of the group of 3 or 4
pictures. The testee picks the one you referred to and works the
answer sheet.
Multiple choice. You can make statements, ask questions, or
have short conversations. The test contains 3 or 4 choices which
the testees read. They pick the one most related to what they
hear.
Completion multiple choice. Learners choose the best way of
completing the lines of a conversation from among the 3 or 6
choices and mark their answer sheets.
Oral cloze. Testees see a passage that has blanks. As they listen
the second time, they fill in the blanks.
Note taking. Candidates take notes during the talk. After the talk
is finished they see the questions they have to answer.
SAQ 6
Dictate the following text:
Dear Sir,
I am answering your advertisement/ for an engineer./ I saw it
in the paper yesterday./ Can I come for an interview next week?/ I
left my job last month/ and I am free every day of the week./
I am 25 in August/ and I am not married./ I studied at Bolton
University/ and I finished there in 2004.
What do you test? Why is the dictation test written this way?
Proiectul pentru nvmnt Rural
105
Write your answers in the space provided above (in no more than 60 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
SAQ 7
Which humanistic approach is recommended for developing and
testing listening?
Write your answer in the space provided above (in no more than 5 words) and
compare it to that in the Answers to SAQs section at the end of the unit.
106
A check list of listening task types that may be used in formal and informal testing
Type
1. Listen and Do: During or
after listening, students are
asked to perform some
action
2. Listen and Do Nothing:
no output
3. Listen and Follow:
students may be given a
map or picture and match
what they hear with what
they see
4. Listen and Respond:
Students are asked for an
affective response
5. Listen and Answer: The
traditional type of question
task
6. Listen and Compare:
Listening for similarities /
discrepancies between two
(or more) inputs
7. Listen and Complete:
Gap-filling
Example
Numbering a drawing, completing a
map, ordering items in a list, matching
items, labeling, ticking.
Listening to a story or a poem
107
Point to Ponder
tomorrow?
In your textbook, you read:
a. Will Mary be traveling tomorrow?
b. What are Marys plans for tomorrow?
c. Who will be with Mary tomorrow?
d. Does Mary have to do it tomorrow?
The correct answer is b
PART B 15 short conversations
In Part B, you will hear short conversations between two people.
After each question, a third person will ask a question about what
was said. Read the four possible answers and decide which one is
the best answer to the question you heard. Then, on your own
answer sheet, find the number of the question and fill the space that
corresponds to the letter of the answer you have chosen.
Listen to an example: A man tells a woman that he doesnt like
the painting either. The question is: What does the man mean?
In your textbook, you read:
a. he doesnt like the painting either
b. it doesnt know how to paint
c. he doesnt love any paintings
d. he doesnt know what to do
The correct answer is A.
PART C
In this part of the test, you will hear longer conversations and talks. After
each conversation or talk, you will be asked some questions. After you
hear a question, read the four possible answers in your textbook and
decide which is the best answer to the question you heard.
Listen to an example: The topic is computer animation.
Question: What is the main purpose of the program?
In your textbook, you read:
a. to demonstrate the latest use of computer graphics
b. to discuss the possibility of an economic depression
c. to explain the workings of the brain
d. to dramatize a famous mystery story
The correct answer is C.
Question: Why does the speaker recommend watching the
program?
In your textbook, you read:
a. it is required of all science engines
b. it will never be shown again
c. it can help viewers improve their memory skills
d. it will help with coursework
The correct answer is D.
Comment upon the above test:
a. Is it a discrete/ integrative test?
b. What language areas does it cover?
c. What type of test is it?
d. How do you evaluate such a test: easy/ difficult
Write your answers in the space provided above (in no more than 10 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
Proiectul pentru nvmnt Rural
109
5.4 Summary
This unit has been concerned with testing speaking and listening
specific procedures have been introduced for formal and informal
assessment of speaking and listening. Changing emphasis in the
assessment of speaking and listening is a move towards integrative
tests that cover all four elements of communicative competence,
towards objective scoring and higher reliability and authenticity. We
have also identified other characteristics:
Assessing processes
Internal (during course assessment instead of external end of
course assessment)
Use of a variety of methods
Criterion referencing
Formative identification of strengths and weaknesses and
recording of positive achievement instead of pass/ fail summative
assessment.
Oral interview
Linguistic competence
Discourse competence
Sociolinguistic competence
Strategic competence
Transactional
Information transmitting
Interactional
Transactional
Authenticity
Top-down
Bottom up
Interlocutor/ assessor
5.6 Checklist
Do students draw up checklists of criteria for success?
Do your learners get frequent reinforcement, e.g. marks,
comments, praise, etc.?
Does your reinforcement or recognition of success come as
quickly as possible after the student has completed the work?
Are the standards you set seen as work achieving by your
students, as well as being achievable by them?
Do you test regularly, and set well-managed deadlines for
students work?
110
SAQ 2
integrative
objective
high reliability
authentic
direct
good validity
SAQ 3
SAQ 4
SAQ 5
SAQ 6
111
students time to check what they have written before collecting the
answer paper.
SAQ7
SAQ 8
112
Unit 6
TESTING THE LANGUAGE SKILLS II
6.1 Unit Objectives ......................................................................................................... 113
6.2 Testing Reading ....................................................................................................... 114
6.2.1 Types of Reading based on Content and Function ................................................ 114
6.2.2 Types of Reading based on Context and Processing Variables ............................ 114
6.2.3 Types of Reading according to Purpose ................................................................ 115
6.2.4 Cloze Passages ..................................................................................................... 116
6.2.5 Passages with Questions ...................................................................................... 117
6.2.6 Microskills .............................................................................................................. 117
6.2.7 True False Dont Know Checks ....................................................................... 118
6.2.8 Other Reading Techniques .................................................................................... 118
6.2.9 Assessing Overall Comprehension ........................................................................ 118
6.2.10 Issues in Teaching Reading ................................................................................ 119
6.2.10.1 Narrative Text. Reading for Pleasure ................................................................ 120
6.2.10.2 Reading for Information .................................................................................... 120
6.2.10.3 An Instructive Test ............................................................................................ 120
6.2.10.4 Types of Test Procedures ................................................................................. 121
6.3 Testing Writing .......................................................................................................... 121
6.3.1 Conditions under which Writing Takes Place ......................................................... 122
6.3.2 Current Theories of Writing with Particular Reference to
Foreign Language Writing ...................................................................................... 123
6.3.2.1 Writing as a Product ........................................................................................... 123
6.3.2.2 Writing as a Process ........................................................................................... 124
6.3.2.3 Writing as a Social Activity .................................................................................. 124
6.3.3 The Main Approach to Teaching Writing. Text Based Approaches .................... 125
6.3.3.1 Grammatical Form Practice ................................................................................ 125
6.3.3.2 A Communicative Approach ............................................................................... 125
6.3.3.3 Writer Based Approach .................................................................................... 125
6.3.4 Various Choices of Writing Tasks .......................................................................... 126
6.3.4.1 Scoring Essay Type Tests .................................................................................. 126
6.3.4.2 The Point Score Method ..................................................................................... 128
6.4 Summary .................................................................................................................. 130
6.5 Key Concepts ............................................................................................................ 130
6.6 Checklist ................................................................................................................... 131
SAA 3 ............................................................................................................................. 131
6.7 Answers to SAQs ..................................................................................................... 132
6.8 Further Readings ...................................................................................................... 132
113
HOW DO WE
READ?
PROCEDURES
115
Write your answers in 20 words in the space provided and compare them to
those in the Answers to SAQs section at the end of the unit.
POINT TO PONDER
Before testing reading in a foreign language, the native language
reading skills of the testee must be assessed. If some testing skills
are not yet developed by the testee in his native language, you can
be sure that the learner encounters difficulties in reading in the
foreign language. However, do not conclude that reading skills from
the native language can be transferred to the foreign language.
Reading tests can follow the general outline of listening tests i.e
the same type of tests that are used for listening can be used for
reading. They can be handled individually, with true false
questions, written instructions, and so on. As easy as reading is to
organize informally, however it is surprisingly difficult to measure
formally.
6.2.4 Cloze Passages
WHEN IS A
CLOZE
PASSAGE
RELIABLE ?
116
Cloze passages are easy to prepare, but the fill in type can be
difficult to write. Multiple choice cloze tests appear to do much the
same thing, being only a little longer to prepare, and as many times
easier to score. You have to supply the correct choice and two or
three other attractive distractors (that have the right meaning but
Proiectul pentru nvmnt Rural
wrong grammatical form, or somehow look right but have the wrong
meaning, and so on). Cloze passage of over 40 blanks can approach
reliability of 85 and above rather easily. But once made, such tests
are difficult to revise. You cannot just cut out the parts of the passage
that do not give good results as you can do with multiple choice tests
or other test types.
6.2.5 Passages with Questions
Many reading tests use passages followed by 3 or 5 questions
(for example, the TOEFL reading section). These tests are extremely
difficult to construct. First, it takes a lot of time to find suitable
passages that have content which is equally familiar to all testees.
The questions tend to overlap, and the answer to one question often
gives cues to the answers of the others. Reading through the
questions in fact can often tell you what the passage is about and
allow a high level of performance even if you do not read the
passage. Such tests take a lot of time per item, and in some ways
require more academic and study skills than reading skills.
6.2.6
Microskills
READING
SKILLS
CRITERIA
LEVELS
117
SAQ 2
What happens if in a nine point scale, the best paper is assigned
a 8 and the worst paper assigned a 2 because the teacher
considers that the extreme scores should not be used because no
paper is good enough to receive the top score and no paper is bad
enough to receive the bottom score.
Write your answers in the space provided above (in no more than 40 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
119
Drawing a map
Labeling
Completing a table
6.2.10.3 An Instructive Test
Sequencing
Following instructions
120
121
How Do We Write?
122
123
125
Sorting Method
Decide on the number of groups to be used ( for example 5) prior
to scoring the papers
Read the papers
Sort the papers and place them into several groups ranging from high
to low
Resort the questionable papers or the border-line ones
Take care to see that the better papers in the better group are
superior to the top papers in the poorer group
If a predetermined number if papers are assigned to each group
function of the size of the class, the number of points assigned to
the question (for example if the number of groups is five then two
papers may be assigned to the first/ top group, four to the second,
eight to the third, four to the fourth, two to the fifth. The teacher is
not required to conform exactly to expected distribution. The
method establishes an expected standard measuring. All
groupings should be used in order to increase the reliability of the
examination.
SAQ 5
Imagine you are in a library. Find your reference materials in one of
the sections of the library.
Which of these items relate to:
a. study skills
b. reference skills
1. Where might you look for a book about submarines?
a. section J; b. section G; c. section H
2. Where would you find the latest issue of Time magazine?
a. section A; b. section B; c. section K
Proiectul pentru nvmnt Rural
127
128
You will have 30 minutes to plan, write, and correct your essay.
Your essay will be graded on its overall quality
1. When the supervisor tells you to begin, read the essay question
carefully.
2. Think before you write. Making notes may help you to organize
your essay. Below the essay question is a space marked NOTES.
Use only this area to outline your essay or make notes.
3. Write only on this topic. If you write an essay on a different topic, it
will not be scored. Write clearly and precisely. How well you write
is much more important than how much you write, but to cover the
topic adequately, you may want to write more than one paragraph.
4. Write neatly and legibly. Do not skip lines. Do not write in very
large letters or leave large margins.
5. Check your work. Allow a few minutes before times are called to
read over your essay and make small changes.
6. After thirty minutes, the supervisor will tell you to stop. You must
stop writing and put your pencil down. If you continue to write, it
will be considered cheating.
Essay Question
(30 minutes)
Do you agree or disagree with the following statement?
A zoo has no useful purpose.
Use specific reasons and examples to explain your answer.
Notes:
VOCABULARY
ORGANIZATION
CONTENT
Score
Level Criteria
30-27
26-22
21-17
16-13
VERY POOR: does not show knowledge of the subject noin substantive not
pertinet OR not enough to evaluate
20-18
17-14
GOOD TO AVERAGE: somewhat choppy loosely organized but main ideas stand
out limited support logical but incomplete sequencing
13-10
9-7
20-18
17-14
13-10
FAIR TO POOR: limited range frequent errors of word/idiom form, choice, usage
meaning confused or obscured
9-7
Comme
nts
129
MECHANICS
LANGUAGE USE
25-22
21-18
17-11
10-5
Total
score
Reader
capitalization,
Comments
6.4 Summary
The assessment and testing of reading and writing, especially in a
communicative oriented classroom is a thorny issue. Because
reading, like listening comprehension, is totally unobservable, it is
important in reading as it is is in other skills to be able to accurately
assess students comprehension and development of skills. The
following overt response indicate comprehending: doing, choosing,
transferring, summarizing, condensing, extending (providing an
ending to a story), duplicating (translating), modeling (after reading
instructions), conversing (engaging in a conversation that indicates
appropriate processing of information. Six general categories form
the basis for the evaluation of student writing: meaning, organization,
content, vocabulary, discourse (sentences, grammar), syntax,
mechanics (spelling, punctuation). This unit provides a wide range of
procedures that can be applied to your own situation.
130
Schema theory
Skimming
Scanning
Silent reading
Reading aloud
Bottom up approach
Top down approach
Process vs product
Authenticity
Interaction
Proiectul pentru nvmnt Rural
Correction symbols
Communicative approach
Writer based approach
6.6 Checklist
Do students draw up checklists of criteria for success?
Do your learners get frequent reinforcement, e.g. marks,
comments, praise, etc.?
Does your reinforcement or recognition of success come as
quickly as possible after the student has completed the work?
Are the standards you set seen as work achieving by your
students, as well as being achievable by them?
Do you test regularly, and set well-managed deadlines for
students work?
Are the questions realistic in terms of difficulty, time allowed the
student to respond, complexity of test
Does the essay question establish a framework to guide the
student to the expected answer
a) Is the problem delimited?
b) Are descriptive words used (compare, contrast, define instead of
discuss, explain)
SAA No. 3
1. Evaluate the paragraphs below: use the levels and criteria from
the L2 composition profile. Fil in the description in
a. In the beginning of life there was no classroom, but we read
about many people have a big deal of knowledge. There was no
classroom told the first man in the world how to plan, how to build
his huts. I read about many potteries that have good poems in the
first and second centuries, they knew hoe these poems without any
classroom. In ancient the women knew how to sewing there
chesses without any teacher.
b. I believe that we get more knowledge out side the classroom
than we do inside. A classroom can give us only limited kinds of
information. If we look at the beginning of civilization, foe example,
we will note that people back then did not have formal classrooms,
yet many of them were well informed. There were no classrooms to
teach the first men how to plant or how to build huts. The great
early poets of the first and second centuries didnt learn their
poems in a classroom, nor did the women find out how to sew their
clothes there.
a. Meaning ............................................
Organization .......................................
Content ...........................................
Vocabulary .........................................
Sentences ..........................................
Grammar ..........................................
Mechanics ..............................................
Grade:
b. Meaning .............................................
Proiectul pentru nvmnt Rural
131
Organization .........................................
Content .............................................
Vocabulary ..........................................
Sentences ...........................................
Grammar ...........................................
Mechanics ................................................
Grade:
SAQ 2
SAQ 3
SAQ 4
SAQ 5
132
Unit 7
TESTING THE LANGUAGE SYSTEM AND BEYOND
7.1 Unit Objectives ......................................................................................................... 133
7.2 Testing Pronunciation ................................................................................................ 133
7.3 Testing Grammar and Usage .................................................................................... 138
7.3.1 Multiple- Choice Fill In ........................................................................................ 138
7.3.2 Modify and Fill In ................................................................................................. 138
7.4 Testing Vocabulary .................................................................................................... 140
7.4.1 Cloze ..................................................................................................................... 142
7.4.2 Multiple Choice Fill- In Type ................................................................................ 142
7.4.3 Multiple Choice Synonym Type .......................................................................... 143
7.4.4 Matching ................................................................................................................ 143
7.4.5 Simple Prompts ..................................................................................................... 143
7.4.6 Selection of the Words to Be Tested ..................................................................... 143
7.4.7 Translation ............................................................................................................. 143
7.4.8 True/ False ............................................................................................................. 143
7.4.9 Checklist Tests ...................................................................................................... 143
7.5 Testing Beyond Language Form .............................................................................. 144
7.5.1 Discourse and Culture ........................................................................................... 145
7.5.2 Speech events ....................................................................................................... 147
7.5.3 Literature ............................................................................................................... 148
7.6 Summary .................................................................................................................. 149
7.7 Key Concepts ........................................................................................................... 149
7.8 Checklist ................................................................................................................... 149
SAA 4 ............................................................................................................................. 150
7.9 Answers to SAQs ..................................................................................................... 150
7.10 Further Readings .................................................................................................... 150
133
A TOP-DOWN
OR
BOTTOM-UO
APPROACH
MICROSKILLS
134
Examples:
You can use pictures while you say (keeping the intonation
identical in both cases):
The sheep is in the lake. The ship is in the lake.
The testee points to the right picture.
PICTURES
AND OTHER
TYPES OF
TESTS
135
additional examples: cheap, sheep, lip, chip, sheep, meet, skip, leap,
ship, leap. Be sure to use the same intonation.
You can also use triplets:
Cheap, chip, cheap
Meat, meet, meet
Ship, sheep, ship
Ask testees to pronounce the contrasting word. If you say
ship, they say sheep and so on.
Then have them repeat phrases:
Cheap ship
Meet the jeep
Leap in the jeap
Ship the chips
When all the three words are the same (AAA), sometimes only
the first and the second words are (AAB). Which is the correct answer?
Ship sheep ship
AAA AAB ABA ABC
bad bad bat
AAA AAB ABA ABC
bed bad beard
AAA AAB ABA ABC
road rod nod
AAA AAB ABA ABC
Then have them give full sentences. You might use the pictures from
discrimination tests: The sheep in the lake. The ship is in the lake.
They may make up sentences that contrast the sounds: The ship has
a leak. The sheep has pink lips.
SAQ 2
Study the following questions and circle T (true) or F (false):
1. Written examinations were introduced because
oral examinations were found not to be valid.
2. Objective tests were introduced because it was
found that traditional techniques lacked reliability.
3. Because of their format, objective tests can be
assumed to possess reliability and validity.
4. Tests, unlike examinations, give accurate
information about a testees abilities.
5. By eliminating marker variability, validity is insured.
6. Some people are more variable in their
performance than others.
T F
T F
T F
T F
T F
T F
OTHER
PROCEDURES
Example
Read the passage clearly and expressively into the microphone with
your tape recorder set at record:
Joe: Where are you going, Betty?
Betty: Hello, Joe, I am going shopping. Ive just moved and I need
some things for my room. Would you like to come with me, or
are you going to work?
J: Thanks, Id like to come. I want to buy a few things too.
B: Im going to look for chairs, a rug, and perhaps a picture.
J: A rug? How big? Did you measure your room?
B: Oh, no. Im only going to get a little one. A big one would be very
expensive. I havent got much money.
J: I havent either. First, lets go to that old shop the one near the
railway station.
B: OK. My boyfriend told me that was a good place to start.
137
ABCD
ABCD
ABCD
ABCD
ABCD
ABCD
ABCD
138
Sylvia (come) here about a month ago. She (leave) her village
because her father (die) and (leave) her with a cruel step
mother and sister. Sylvia (not hear) from her since she left. She
(live) in a flat with two roommates for the last three weeks. They
(make) her do most of the work while they (go out) to have a good
Proiectul pentru nvmnt Rural
time. Last night she (finished) her work early and (go) to the
local disco. She (have) a good time but (leave) at midnight
because her feet (hurt). Since then she (find) more interesting
things to do and so she (feel) much happier now.
Range
No evidence of correct
morphological and syntactic
structures
No control of sttructures
Errors of all types
Limited range of
morphologic and syntactic
structures
Complete range of
morphologic and syntactic
structures
Complete range of
morphologic and syntactic
structures
No systematic errors
Accuracy
139
SAQ 3
True or false?
Grammar has been the skeleton around which most testing has
developed techniques.
Circle T or F and compare your choice to that in the Answers to SAQs section
at the end of the unit.
VOCABULARY
AND
STATISTICS
140
IMPROVING
VOCABULARY
TEST SCORES
RULES FOR
TESTING
VOCABULARY
141
SOURCES
OF
VOCABULARY
TESTS
7.4.1 Cloze
Cloze is indirectly a vocabulary test.
Example
Great Britain is an island that ...1 the Atlantic Ocean and the North
Sea. It 2 the mainlands of England, Wales and Scotland. Ireland
3 the west coast of Great Britain.
(Answers: 1. is surrounded by; 2. comprises, consists of, is
composed of; 3. lies off
Point to ponder
Words are one of our chief means of adjusting to all the situations of life.
Bergen Evans
142
143
Examples:
1. To test yourself on the vocabulary, fill in the missing letters in
the incomplete words:
A superstition is an untrue b- - - - f held by many p - - - - e based on
fear of n - - - e. The ground hog s - - - y is one of the c - - - - - - t
superstitions.
The matching lexical cloze is a similar type of test. The words
are listed below. In a true matching lexical cloze the words are
omitted according to a system.
2. Choose appropriate words from the list below to complete the
passage. You may need to change the forms of some of the words.
Capable, permit, privilege, employ, complaint
Women in the United States were looked upon, for a long time, as
being less than men. This is the why they were not to have
as many as men.
3. Circle in the margin the letter corresponding to the most
appropriate completion for the following sentences when
A = back; B = along; C = through; D = out; E = off; F = up
I liked the first volume, but I cant get the second. A B C D E F
Be sure to get the bus at the second stop.
ABCDEF
4. Circle in the margin the letter corresponding to the phrase which
correctly completes the sentence:
The mother of your father or mother is your A B C D
A. stepmother; B. grandmother; C. godmother; D. mother-in-law
5. Sets. Three of the four words in each line are similar in meaning or
share some common features. Draw a circle around the word that
does not fit:
1. conference, congress, meeting, ethics
2. collapse, dissipate, speculate, decay
6. Multiple choice in context.
Later I ... to them for my bad behavior.
a. apologized
b. applauded
c. enquired
d. entertained
find out how other people think and live. Cross cultural and even
multicultural learning is considered in our days to be highly desirable.
Point to Ponder
Were Shakespeare suddenly to materialize in London or New York
today, he would be able to understand, on the average, only five out
of every nine words in our vocabulary. The Bard would be a semiliterate.
Toffler
7.5.1 Discourse and Culture
GRICES
MAXIMS
DISCOURSE
FUNCTIONS
145
COHESION
Example:
147
7.5.3 Literature
HOW TO
SELECT
LITERARY
TEXTS IN TEFL
TEACHING
CULTURE
148
When you teach culture, you teach about a culture (family, size,
customs, holidays, and educational system), attitudes towards a
culture, and behavior appropriate for a culture (how to behave in a
family, act during a ceremony, etc). When we think about teaching a
culture we have to think of what is essential to teach in order to
survive in the respective culture. Becoming native would take many
years of effort. In other words, choosing the right things to teach
seems more important for culture than it does for language.
7.6 Summary
This unit explained how to assess mastery of the subskills of English
i.e. to test how well each component has been mastered as a subskill
of the four main skills. Of course, tests of grammar, vocabulary,
pronunciation do not show exactly how well a person uses English,
but they can help teachers identify students strengths and
weaknesses in oral or written communication. Choosing which
procedure to use depends on the learners age and language ability
as well as on the kind of skill being taught. It is true that tests devoted
exclusively to pronunciation are rare today. This does not mean that
testing pronunciation is useless. It simply means that this subskill is
assessed in conjunction with listening and speaking, incorporating
context. Pronunciation items can be useful as they may measure
progress made on specific points of pronunciation.
Top-down
Bottom-up
Discrimination test
Minimal pairs
Intonation
Modify and fill in
Discourse
Grices rules
Cohesion
Coherence
Speech events
Sheltered academic programme
Culture
7.8 Checklist
Do all students achieve some success and get some
reinforcement?
Do you ask your students to set themselves tasks?
Do you adopt assessment methods which do not rely exclusively
on written assessments?
Do you ask questions equally of males and females?
Do all your students get some measure of success in their
learning? Does this success get quickly reinforced?
Do you encourage self evaluation and student responsibility?
Do your homework assignments combine maximum learning value
with minimum marking effort?
Are you rigorous about setting colleting and marking homework?
And do your students parents know this?
Do you use homework marks for report, or record cards?
Proiectul pentru nvmnt Rural
149
SAA No. 4
Give your learners three tests of one grammatical feature e.g. form
and use of personal pronoun, a multiple choice test, a cloze test,
and ten sentences to translate into English. Make graphs of the
number of errors made by the learners on each test. Repeat the tests
in a different order for another feature e.g. Past tense versus Present
Perfect and examine the results as well. Write an analysis of what
this experiment has revealed about the relative difficulty and
discriminatory power of the tests, and the most persistent problems
for the students who are learning these features.
SAQ2
SAQ3
SAQ4
Unit 8
NEW TRENDS IN TESTING
8.1 Unit Objectives ......................................................................................................... 151
8.2 General Trends.......................................................................................................... 151
8.3 Computer- Based Language Testing ........................................................................ 152
8.4 Alternative Assessment ............................................................................................ 156
8.4.1 Techniques ............................................................................................................ 156
8.4.2 Journals ................................................................................................................. 156
8.4.3 Conferences .......................................................................................................... 157
8.4.4 Cooperative test construction ................................................................................ 157
8.5 Portfolios ................................................................................................................... 157
8.5.1 Characteristics ....................................................................................................... 158
8.5.2 Assessing Portfolios .............................................................................................. 159
8.5.3 Portfolio Content .................................................................................................... 160
8.5.4 Useful advice on development of portfolios ........................................................... 161
8.6 Summary .................................................................................................................. 162
8.7 Key Concepts ........................................................................................................... 163
SAA 5 .............................................................................................................................. 163
8.8 Answers to SAQs ..................................................................................................... 164
8.9 Further Readings ....................................................................................................... 164
151
153
SAQ 1
What kind of tests are these questions specific to:
1. This is a test about the use of the Past Tense and Present Perfect in
English. Put a cross on the number which shows how well you think
you can use these tenses:
I make few mistakes when I use these tenses. 10, 9, 8, 7, 6, 5, 4, 3,2,1, 0
I always make mistakes when I use these tenses.
2. Do you think you can answer this question correctly? Yes Not sure
No
Write your answers in the space provided above (in no more than 20 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
154
REASEARCH
ON
INTELLIGENCE
Alternative
Summative
Formative
Oriented to product
Oriented to process
Interactive performance
155
156
8.4.3 Conferences
Conferencing implies a one-to-one interaction between teacher
and student. The role of the teacher assumes the role of the
facilitator and guide, an ally of the student. It develops in your learner
self-reflection attitudes. These alternative types of assessment are
formative looking forward towards further development.
8.4.4 Cooperative test construction
Ask the testees about the things they have learned and that
should be in a test. Then ask them to formulate the actual test
questions. The teacher makes them aware that the real test will
contain some of the questions they have selected. The cooperation
test becomes in this view a way to stimulate review and integration.
SAQ 2
List and explain three advantageous features of a system for the
machine construction of tests.
Write your answers in the space provided above (in no more than 30 words) and
compare them to those in the Answers to SAQs section at the end of the unit.
8.5 Portfolios
The increasing dissatisfaction with traditional, quantitative forms
of assessment has led to the development of alternative assessment
approaches. The theories behind the use of the portfolio for both
assessment and learning purposes are the constructivist learning
theories (which see learners as actively making sense of new
knowledge and deciding how to integrate it with previously held
concepts) and Vygotskys notion of the zone of proximal
development. The use of portfolios for both assessment and learning
purposes provides opportunities for demonstrating learning and for
the development of important learning dispositions, processes and
strategies.
Point to Ponder
Lev Semyonovich Vygotsky (1896 1934) Russian psychologist,
born in Orsha. He studied various social sciences at Moscow
University, and turned to psychology when aged 28. This last
decade of his life, when he was at the Institute of Psychology in
Moscow (1924 34) was his productive period. His theory of
cognitive development, especially his view of the relationship
between language and thinking, have strongly influenced western
Proiectul pentru nvmnt Rural
157
159
Journal and
logs
Audiotapes of
presentations
Examples of
written work
Videotapes of
student
performance
A portfolio:
Contains evidence of a
Mind map and
notes
students achievement,
skills,
Tests and
quizzes
accomplishments)
Charts,
graphs
List of books
read/summaries
Questionnaire
results
160
Self
evaluation
Set a limited
number of
objectives
Ask students to
reflect on which
items are worth
including
Organize
conferences to
review students
portfolios
Useful advice on
development of
Help the parent
examine the portfolio
(make him/her aware
of evidence of
progress and areas of
needed improvement)
portfolios
(after Ronald L. Portin)
Be sure each
item is dated (to
assess the
evaluation of
progress)
161
SAQ 3
Why people who do well in intelligence tests usually are better
learners than those with excellent memories?
Write your answers in 20 words in the space provided and compare them to
those in the Answers to SAQs section at the end of the unit.
8.6 Summary
This unit pinpoints the main trends in contemporary assessment
and testing that betray a move from quantitative to qualitative
testing, from paper based tests (PBT) to computer based tests,
from traditional testing to alternative testing, including selfassessment. Advantages of CAT (Computer Adaptive Testing) may
be described as follows:
Individual testing time may be reduced
Frustration and fatigue are minimized
Boredom is reduced
Test scores may be provided immediately
Diagnostic feedback may be given immediately
Test security may be enhanced
Record-keeping functions are improved
Reporting, research and evaluation capabilities are
expanded.
A constructivist approach to testing, one of the alternatives to
traditional testing, is portfolio evaluation. Several advantages are:
It promotes cooperation rather than competition
It enhances professional communication
It requires no technical knowledge of quantitative evaluation
procedures
Ideas are conserved for future application in other classes
Other characteristics of contemporary trends include:
Continuous long term assessment
Formative tests
Criterion referenced score
Interactive and motivation
162
Alternative assessment
Bodily kinaesthetic intelligence
Constructivist approach
Computer
Computer adaptive testing
Computer administered test
Computer assisted instruction (CAI)
Proiectul pentru nvmnt Rural
SAA No. 5
This paper aims at making you identify, clarify and develop ou
informed, comprehensive personal philosophy of grading that is
consistent with your philosophy of teaching and evaluation.
Examine the following items and circle the figures for all items that
should be included in your set of criteria for determining a final mark.
Write in a percentage the weight you would assign to each circled
item (obviously the total percentage is 100 %)
1. Language perfomance of the learner (based on tests, quizzes,
other objective tests) ..........
2. Your informal observation of the learner's language ......
3. Oral participation in class activities ..............
4. Attitudes and behaviour: degree of cooperation, politeness,
disruption in the classroom ..............
5. Effort ...........
6. Motivation .................
7. Punctuality and attendance .............
8. Self-assessment ...............
Write an essay about your philosophy of grading taking into account
your answers to the questionnaire above. Do not write more than
three pages. Consider the following questions:
Do you consider yourself consistently impeccable in your objectivity?
Can you capture the totality of your student's competence only
through formal tests? What is the value of alternatives in
assessment?
Example:
I base my final marks/grades on student language performance
because I think grades should represent .................. Grades should
not be contaminated by ........................... I discourage the inclusion
of ..............................
Proiectul pentru nvmnt Rural
163
SAQ 2
SAQ 3
164
Bibliography
Bibliography
165