Validity & Reliability

the characteristics of a good test

EDU 480: PRINCIPLES OF ASSESSMENT AND EVALUATION
Faculty of Education, UiTM
Nadia Ainuddin Dahlan
WHAT IS A GOOD TEST?

VALIDITY
Does the test measure what it
is supposed to measure?
ANY GOOD TEST
SHOULD BE…
RELIABILITY
How consistent are the
✓ VALID results of the test?
✓ RELIABLE
✓ PRACTICAL PRACTICALITY
How practical is the test to
administer?
1. Validity
Know what is the purpose of the test & how the results will be used!
Ensuring the accuracy of the test & its desired interpretation

WHAT IS THE PURPOSE OF THE TEST?

Why do the students need to take it? + WHAT WILL THE RESULTS BE USED FOR?
What can we accurately say about the
results?

1. To compare students with one another?

A
SPM NRT O
2. To find out how much the student has L
mastered some aspect of the curriculum?
Mid-term test
A
3. To see to what extent the students CRT
possesses certain characteristics? MEdSI F
L
Types of Validity (content, criterion, face, consequential)
A. Content validity
• The degree to which a test sufficiently samples the intended domains
• What the test intends to measure should be reflected in the content (IOs, LOs & TOS)
• Representative sample of domain

5 questions
Aina gets an Form 4 syllabus
‘A’ for a
subject in SPM
+ 35 questions
Form 5 syllabus

Domain Representativeness
What are the Are the number of items
domains/areas/parameters/ selected adequate/enough to
content being tested? measure students’
understanding?
B. Criterion validity
• The extent to which test results can be used to:
Predictive
• Infer/predict students' future performance
• Two measures obtained at different occasions (there is time interval)
Concurrent
• Estimate students' current performance
• Two measures obtained concurrently (no time interval between tests)
C. Construct validity
• The extent to which test results can be inferred to reflect a particular/various
constructs
• The legitimacy of interpretations made from students' test results to a construct
• Deals with human characteristics/quality
• Mathematical reasoning abilities, reading comprehension, creativity etc.
• Sociability, anxiety, leadership etc.
• Can be used to explain behaviour

Intelligence test MENSA High IQ vs Low IQ

Personality test MEDSI Teacher vs Not Teacher

D. Face validity
• Appearance of validity = perceived to be meaningful
• But may or may not actually be valid = need to have a close look at the items

E. Consequential
• Consequences of a test = social, psychological
• Effects of the test (intended & unintended) on students & teachers
• Positive (increased motivation) vs. negative (demotivated)
2. Reliability
A test is reliable when it produces consistent results
Does the test measure what it intends to measure in a reliable manner?

1st time
Score: 75
Measurement error
MEdSI When M.E. is less = The test is more reliable
2nd time
Score: 77

Reliability is a necessary but not sufficient condition for validity

Just because a test is reliable does not mean that it is also valid!
The conditions of validity must still be met
Types of Reliability
A. Test-retest reliability
• Administering the same test twice – at two different times
• Time interval can be from minutes to years

B. Equivalent-forms/Parallel-forms reliability
• Administering two forms of the test that measures the same contents,
domains or behaviours
• Are scores consistent over different forms?
• Forms of test are different but equivalent (e.g. Set 1, Set 2 etc.)
• Time interval should not be too long if achievement tests
C. Inter-rater reliability
• Consistency of judgement across different assessors
• Assessors use same tasks/procedures/standards
• Improved with clear scoring rules & training
• Correlation of scores between the different assessors
•
D. Internal consistency reliability
• The degree to which items in a test is consistent with one another
Split-half
• Administer test once
• Score odd & even items separately – as though dividing a test in two equal
halves
• Correlation of sub scores – estimates the reliability of a full assessment
Excercise
What do you want to measure? What will you use the results for?
Current level of students’ knowledge
in Form 4 Modern Mathematics
+ To monitor students’ ongoing progress
To improve instruction

What kind of test should be used?

Is this test valid? Is this test reliable?

A word about types of tests…
Achievement test
1. INSAK – Inventori Sahsiah Keguruan
• Amanah, taat setia, ikhlas dan dedikasi, kawalan emosi, disiplin, kepekaan,
kepimpinan, motivasi, kreatif dan inovatif & kematangan fikiran

2. MEdSI – Malaysian Educators Selection Inventory Psychometric test

• Personality, minat kerjaya, nilai integrity & kecerdasan emosi

3. IKep - Iventori Kecerdasan Pelbagai

• https://www.slideshare.net/MohdNoorNoor/pentadbiran-ikp-ting-3

4. UASR - Ujian Aptitud Sekolah Rendah Aptitude/Ability

• https://www.slideshare.net/norlianaramli7/aptitut-tahun-6 test
EXAMPLE INSAK
EXAMPLE MEdSI
EXAMPLE
Relationship of Validity & Reliability