Session 1: Key concepts in language testing

3.- Which of these two test is more ​valid​ as a test of writing? why? Which is the more
reliable​ test? why?

- According to the two examples, test B is more ​valid ​since it measures what it is
claimed to measure, for instance​ writing ability,​ asking the students to write
something for the class. The more ​reliable ​is test A. Even though teachers have a
marking key with the correct answers, the test asked the student to complete a
missing word from each sentences rather than testing writing abilities.

3.1.- Can you think of ways to make the more reliable test more valid? Make a list of
suggestions. Then, make a list of suggestions for making the more valid test more reliable.

Reliable test more valid Valid test more reliable

- Select 5 sentences and write a short - Take the test twice to the same
story related to real life group

- Using face validity - Take the same test twice within

- Test being checked by experts in
writing abilities
- Make the test cover all the areas of
the related skill.
10-15 days to check improvement.
- Change the order teachers marks
the composition.
- Make a different version of the test
containing the same knowledged or
skilled taught to measure.

4​.-​ Validity is commonly deconstructed into the following types of validity. What do you
think is meant by each of these?
 Types of validity Meaning

Face validity It is the agreement the students show to

approve the test, whether it measures the
ability which it is supposed to measure.

Content validity Whether the test is relevant to real-life

situations

Construct validity It deals with whether the assessment

accurately measure the ability or skill that
were taught.

Predictive validity If two separated test but related test

produce similar results

Concurrent validity The results from an well known test are

measure in different time with same or
similar results.

Do you think any of the five types of validity is more important than any other?

- No, I think all five types of validity are important as far as teachers are well prepared
to make test which measures what are supposed to measure.

Can you rank them in a kind of general order of importance?

- If I were to rank these types in order of importance, it would be:

-
- Content validity
- Construct validity
- Concurrent validity
- Predictive validity
- Face validity

Can you think of any specific situation where one type of validity may be much more
important than usual?

- Construct validity might be much important for an English study program since it
needs a panel of teacher experts to examine and determine whether the items
assess what are supposed to measure.

Which of these five types of validity do you think is most problematic for classroom
teachers?
 - I think that Content Validity is an issue for classroom teacher since most of them
work with standardized textbooks which do not reflect the real context of the school
neither the environment in which it is taken.

5.- Reliability

Think of a least three factors within each of these areas which might affect how reliable
test results might be.

Areas Factors

Testing condition - Not meaningful to the students

- Human changeability
- Out of context

Test formal/ individual - Not similarity between tests

- Test participants ( age, gender,
environment)
- Not measuring what it claimed to

Scoring conditions - Bias in scoring

- Previous knowledge of the test
- Not a clear agreement between
raters

What do you think exactly is meant by the following?

- Item reliability All items in a test measure the same thing

- Inter-rater- reliability It is the balance between various scores on

what degree markers scores shows
equality.

- Intra-rater-reliability Agreement among a repeated administration

of a diagnostic test done by a rater

What might adversely affect these?

What might negatively affect these is the less training teachers have with these concepts
when assessing test to students.
Practical ideas for maximizing reliability.

Why these could lead to great reliability. Can you see any disadvantages or difficulties
these may present to other aspects of test writing and delivering?

These proposal present great ideas for reliability because make us provide clear and direct
questions, avoiding ambiguity, for all type of students the same difficulty without
discriminating between the weaker and stronger ones and tell us use material the students
are familiar with. However, identifying students by number appears not to be correct since
we are dealing with people and teaching people who think and feel, and the purpose of
educating students is humanizing the current education.

Ways to test reliability

Three common ways to test reliability are:

- Test-retest
- Split-half- reliability
- Check individual item reliability

The problem with the first two of these are that the first measures the stability of a test over
time; and the second works with large questionnaires in order to divide it into two done by
comparing the results of one half of the test with the other half of the test to see whether
they provide the same results.

Victor Machuca Carrera