ANALYZING AND INTERPRETING OF TEST ITEMS

LEONCIO P. LUMABAN, Ph.D. Education Program Supervisor II

Meaning of Numbers

Numbers as codes for labeling purposes(nominal) 1= girl 2=boy 1= RURAL 2= URBAN 1=PGMA 2=RR 3=EV 4=PL 5=FPJ

1=1 12

Stud No. 1 2

Gender Location

Choice

1
2

1
2

1
1

What can you infer from the data given Meaning of Numbers on table below?

Numbers as codes for labeling purposes(nominal) 1= girl 2=boy 1= RURAL 2= URBAN 1=PGMA 2=RR 3=EV 4=PL 5=FPJ

1=1 12

Stud No. 1 2

Gender Location

Choice

1
2

1
2

1
1

Meaning of Numbers

Numbers as ranks for ordering purposes (ordinal)

1<2
5= outstanding 4= very satisfactory 3= satisfactory 2= fairly satisfactory 1= needs improvement

<3

12 3

Application in the classroom

Numbers to label categories

1 = Fail 2 = Pass 3 = Advanced level 2 = Intermediate 1 = Beginning

Stud. No 1

Lang Prof 2

Grade 2

2
3 N

1
3 1

1
2 1

Frequency distribution by group

Based on the table, when do you fail or Application in the classroom pass a student?

Numbers to label categories

1 = Fail 2 = Pass 3 = Advanced level 2 = Intermediate 1 = Beginning

Stud. No 1

Lang Prof 2

Grade 2

2
3 N

1
3 1

1
2 1

Frequency distribution by group

Numbers as arbitrary measures for indicating amount or degree based on an arbitrary zero point (interval) Math = 60 Attitude Scale = 25 temperature = 30 degrees

0
0

Numbers arbitrary measures If your as score is zero in a for indicating amount or degree based on measuring instrument that an arbitrary zero point (interval) is inMath interval scale, how = 60 will Attitude you interpret it? Scale = 25 temperature = 30 degrees

0
0

Numbers as true measures indicating measures using scales with true zero point ( ratio) 5 meters 0 n 3 kilos true numerical scales

If your score is zero in a measuring instrument that is in ratio scale, how Numbers as true measures indicating should you interpret it? measures using scales with true zero
point ( ratio) 5 meters 3 kilos

true numerical scales

Test Results

Raw scores

Raw score (RS) numerical summary of a students test performance number of items correctly answered ( one correct item gets one point) 30 correct items out of 50 items number of points received on a test ( regardless of the number of points given to an item/task) 30 points out of 50 possible points ( Each task = 10 max points)

Some assumptions prior to analysis..

Scores are coming from good tests! A good test consists of good items! What is a good item?

Measures a significant learning outcome.

(CONTENT VALIDITY)

Discriminates between one who knows and one who doesnt know; between high-achieving and lowachieving

(CONSTRUCT VALIDITY)

Distracters effective (multiple-choice items)

ITEM ANALYSIS for Summative Tests (norm-referenced perspective)

Item analysis is good for planning future normreferenced tests Can be used for instructional purposes

I-A data provide basis for efficient class discussion of the test results I-A data provide a basis for remedial work I-A data provide a basis for the general improvement of classroom instruction I-A procedures provide a basis for increased skill in test construction

ITEM ANALYSIS for Summative Tests (norm-referenced perspective)

Ability to Discriminate
It should not be very easy every one will get it! It should not be very difficult No one will get it!

Simplified I-A procedures

Classroom

Compare the performance of the highest-scoring students and the lowest-scoring students

Large class top ten vs bottom ten Small class top half vs bottom half

Sample Item Analysis Data

Item Card Subject: _____________ Unit: Weather Objective: Identifies use of instruments Item: Which of the following is most useful in weather forecasting? A. Anemometer * B. Barometer C. Thermometer D. Rain gauge

Sample
(Back of item card) Item Analysis Data
Group 1 U-10 L-10 A 1 2 B 8 4 C 0 1 D 1 3 Diff. Disc Index Index .60 0.40

How was the difficulty/discrimination index Sample computed?

(Back of item card) Item Analysis Data
Group 1 U-10 L-10 A 1 2 B 8 4 C 0 1 D 1 3 Diff. Disc Index Index 0.60 0.40

ITEM ANALYSIS for Summative Tests (norm-referenced perspective)

Indices:

Difficulty Index = No. of students in upper group getting item correct + No. of students in lower group getting item correct/ total no. of students who took the test

p = RH + RL N

Discrimination Index = No of students in upper group getting item correct No of students in lower group getting item correct / of the no. of students who took the test

r = (RH RL) Ng

ITEM ANALYSIS for Summative Tests (norm-referenced perspective)

Where R = # of correct responses H = high group L = low group N = # of students g = size or # of students in either high/low group

Difficulty (p) indices Discrimination indices 0.91 - 0.95 very easy 0.40 or higher Very Good Item 0.76 - 0.90 easy 0.30 0.39 - Good Item 0.25 - 0.75 average 0. 20 0.29 - Reasonably Good Item 0.10 0.24 difficult 0.10- 0.19 Marginal Item 0.05 0.09 very difficult Below 0.10- Poor Item

In the discussion of the results of the item analysis, the item needs to be revised or rejected when: Almost all in both groups are able to answer the item Both groups find the item too difficult such that almost none are able to answer the item correctly When a negative d is obtained (i.e., more in the low group answered the item Additional information to help you decide which items to eliminate from a test, before it is again administered is to analyze the distractors or options.

Conditions: Miskeying the upper group chooses a distractor more frequently than the key A B C D* Example : 1 1 9 2 Guessing each option is chosen with about equal frequency. Example : A B C* D 4 3 3 3 Ambiguity one of the distractors is chosen with about the same frequency as the correct answer by the upper group Example : A B C D* 6 0 1 6

 Poor distractor Some distractors are not chosen at all by all students in the upper group. Example : A 4 B 0 C 0 D* 9

ITEM improvement and selection Guidelines: 1. Consideration based on difficulty index and discrimination index Discrimination Index 0.40 and above (Very Good item) 0.30 0.39 (Good item) 0.20 0.29 (Reasonably Good item) 0.10 -0.19 (Marginal Item) Below 0.10 (Poor item) Difficult Difficult Difficult Rejected Difficulty Index
0.24 or lower 0.25 0.75 0.76 or higher

Acceptable Improvable Marginal Rejected (Kept for

Easy Easy Easy Rejected Mastery)