NORM REFERENCED VS CRITERION REFERENCED TESTING

Norm Referenced Tests: Norms are performance levels of representative samples.

Norms are established for creating an anchoring point/a kind of benchmark which
may be used for comparing the performance of elements from population with the
performance levels given by representative sample.
Any one type of Standard scores is used for establishing norms (Standard scores by
themselves are not norms).
Stages in Developing Norms:-
1. Calculating the raw scores of representative sample
2. Converting the raw scores to any one type of standard scores.
3. Generating Norm Tables for Performance levels of representative sample using
the standard scores.
(Criteria used for generating Norms can be: Any one of the criterion of Age, Gender,
Education qualification, Language known, National or various combinations of these
criteria)
Limitations of Norm Referenced Tests:
1. Tests may differ in their content though may be bearing similar labels.
2. Scale units may not be comparable. SD may differ.
3. Composition of standardization samples used in establishing norms may vary.
4. Psychological test norms are not universal or permanent.
To overcome all these limitations the alternative of Criterion Referenced Test can be
adopted.
Criterion Referenced Tests- These tests use a Specified Content Domain rather than
a specified population of persons. In this type of testing no score comparison is done.
Test performance may be reported in terms of specific kinds of operations test taker
has mastered. It is useful in computer-assisted, Computer-managed & other
individualized self-paced programs.
1. In this type of testing the focus is clearly on what a person can do & what he
knows and not on How he compares with others.
2. These tests can best be adopted for testing basic skills which can have highly
specific objectives determined.
3. One important feature is procedure of mastery testing which can tell whether
individual has or has not attained the pre-established level of mastery.
4. Before administering the test, using some source the likely level of potential of
test takers is identified and they would be exposed to only those items which
have difficulty level matching to potentials of test takers.

In mastery testing individual differences in performance are of little or no interest.
These tests minimize individual differences in performances after appropriate
training. No. of items & proportion of items to be correct are important questions for
mastery testing.

Limitations of Criterion Referenced Test -

1. Lot of questions for each level of difficulty need to be prepared.
2. Determining level of potential of each individual prior to exposing to test,
becomes tough task.

ITEM WRITING
1. Define clearly what you what you want to measure. To do this command on
theoretical understanding is very important.
2. Generate an pool of items. Theoretically all items should be chosen randomly
from the universe of item content.
3. Avoid exceptionally long items.
4. Keep the level of reading difficulty appropriate for those who will complete the
test.
5. Avoid double barreled items which convey two or more ideas at the same
time.
6. Consider mixing positively & negatively worded items properly.

PRECAUTIONS
1. Avoid using difficult words unless the test measures vocabulary and word
recognition
2. Avoid complex grammatical constructions.
3. Replace emotionally loaded language with neutral language.
4. Rephrase items that imply bias towards particular section of society
5. Tests with more than one item type should be avoided as it complicates item
analysis.

ITEM FORMATS
ABILITY TESTS: While designing items for a test, important consideration needs to
be given to Item Format or Type.
Free Response (or open-ended) item type: In this type Essay or Short answer
questions are considered. Essay questions ask for descriptive answers in paragraph
or sentence form. In short answers test takers have to give brief responses, as a word
or two or one or two sentences.
Fixed Response (or close-ended) item type: This type is also called objective item
type.
Fixed Response items provide the test takers all permissible responses. Examples of
this item type are multiple-choice, true-false, matching, ranking, and rearrangement
items.
Advantages of Fixed response items:
1. These items bring more agreement among scorers, or interrater reliability. This is
possible because the tests author specifies the right answers unambiguously before
administration. In open-ended questions role of subjective element in assessment of
answers is high.

2. These items are a better measure of the target characteristics as the smaller, briefer
items give wider coverage.
3. Improved coverage reduces error by chance and thus makes the test fairer to the
test takers.
4. The tests with this format take less time to complete and score.

PERSONALITY TESTS: Personality tests also like ability tests employ both free as
well as fixed-response items.
Fixed-Response Item Type: This type is broken down into four groups: Dichotomus
type:- These items force the test takers to choose between two alternatives. Rating-
scale type:- These may also be labeled as continuous items. The responses fall on a
rough continuum. Usually the test takers have to respond on a five-point scale from
agree to disagree. Another method may be each point is given clear description.
Midpoints type: These tend to attract a large number of responses. This usually
makes the scale uninformative.
Forced-choice type: In this type the test takers are forced to make one choice very
clearly.
Free Response Item Type: Free-response personality tests are projective tests. In these
tests test takers are given vague stimuli like inkblots or shadowy pictures.

ITEM ANALYSIS
ITEM DIFFICULTY: This index tells about difficulty level of each and every item in
the test. Large number of Too difficult or Too easy items in the test, make it
uninformative. (For KSA tests Difficulty Index is expressed in terms of p value which
indicates proportion of people passing on any given item.
For personality tests Item Difficulty is considered in terms of item attractiveness.
Items attractiveness is its capacity to elicit a positive response. Test developer should
be careful in designing the items in such a way that all or most of the items are not
eliciting only positive responses.
ITEM DISCRIMINATION INDEX: This index denotes the difference between the
members of high and low scoring groups who have given the keyed response. For
KSA tests Item Discrimination Index can be calculated with the help of any of the
two methods, viz. Point Biserial Correlation (a special form of correlation) or Extreme
group method. Item Discrimination Index develops item-test correlation and helps
determine which Items are useful in discriminating between potentials and are
worthy of retaining in the test.
For Personality Tests, Discrimination Index is calculated using Extreme Group
Method
(FORMULA: 1. r
pbis
= Y
1
- Y x p/1-p )
s
(FORMULA: 2. Extreme Group Method- d
i
= (n
high
/n
h
) (n
low
/ n
l
)