Psychological Testing Test Administration1

Kaplan, R.M., & Sacucuzzo, D.P. (2011). Psychological Testing.
Cengage Learning Asia Pte. Ltd: Singapore.

Objectives
1) Discuss how the relationship between the
examiner and the test taker can affect test
scores
2) Explain how an expectancy effect might
affect a test score
3) Examine the relationship between test
performance and reinforcing particular
responses

Objectives
4) Outline some of the advantages of
computer-assisted test administration
5) List what characteristic of the state of the
subject should be considered when a test
score is evaluated
6) Know what problems you would need to
consider in training your observers if you
were in-charge of a behavioral observation
study
OUTLINE
I. The Examiner and the Examinee
Relationship
between examiner
and examinee
Training of Test
Administrators
Computer-Assisted
Administration
Race of the
Examiner
Expectancy Effects

Mode of
Administration
Language of the
Examinee (Test
Taker)
Effects of
Reinforcing
Responses
Subject Variables

II. Behavioral Assessment and Methodology
Reactivity Expectancies Statistical Control
of Rating Errors
Drift Deception
Kaplan, R.M., & Sacucuzzo, D.P. (2011). Psychological Testing. Cengage Learning Asia Pte. Ltd: Singapore.

We have a tendency to think that an observed
score really represents the TRUE ability or
TRAIT we are trying to measure

Does a score really represents TRUE
ABILITY or TRAIT? Explain
WHAT concept is implicated in the
relationship between a TRUE and
OBSERVED score?

What is the focus of RELIABILITY?

What does a RELIABILITY estimate
SIGNIFY?

What does it mean if a test has HIGH
RELIABILITY estimate?
Reliability theory is
primarily concerned
with random sources
of error
What are the other
potential sources of
ERROR?

1) Testing situation
2) Examiner (tester)
characteristic
3) Examinee (test-taker)
characteristic
OUTLINE
Relationship
between examiner
and examinee
Training of Test
Administrators
Computer-Assisted
Administration
Race of the
Examiner
Expectancy Effects

Mode of
Administration
Language of the
Examinee (Test
Taker)
Effects of
Reinforcing
Responses
Subject Variables

Reactivity Expectancies Statistical Control
of Rating Errors
Drift Deception
The Examiner and the
Examinee
Both the behavior of the examiner and his or
her relationship to the examinee can affect
test scores

1. Enhanced rapport vs. neutral rapport
2. Approving comments vs. Disapproving
comments
3. Familiarity vs. Unfamiliarity with proctor
4. Substantial interviewer effects

** Examiners should be aware that their rapport
with test takers can influence the results
WISC
EXAMINERs friendly conversation and
verbal reinforcement

higher IQ scores for 5
th
to 9
th
grader
than those 5
th
to 9
th
grader who had a
NEUTRAL examiner BUT little effect
on 1
st
to 3
rd
grader (Feldman &
Sullivan, 1960)

Saying GOOD or FINE vs.
I thought you could do better than that or

Children who took the test under a
disapproving examiner received lower
scores than did children exposed to a
neutral or approving examiner (Witmer,
Bernstein, & Dunham, 1971)
Among 137 young children who took a
reading test, reading scores were
significantly lower when the proctor was
unfamiliar (DeRosa & Patalano, 1991)

In Fuchs and Fuchs (1986) quantitative review
of literature (22 different studies which
involved 1489 children)
the average test performance was roughly 4 IQ
points higher when examiner was familiar with the
examinee
For those children from low SES, familiarity
accounted for 7.6 IQ points

Familiarity can either positively or negatively
bias test results

Respondents may give response that they
perceive to be expected by the interviewer

People tend to disclose more information in a
self-report format forms than they do to an
interview (Moun, 1998)

People report more symptoms and health
problems in a mailed questionnaire than they
do in face-to-face interview (McHorney, 1999)

Computer administration is at least as
reliable as traditional test
administration (Bergstrom & Lunz,
1999; Campbell, Rohlman,
Storzbach, & Binder, 1999)

In most testing situations,
examiners should be aware that
their rapport with test takers can
influence the results

The Race of the Examiner
Racial Bias

Sattler (2002, 2004) concluded that there is little
evidence that the race of the examiner
significantly affects intelligence scores

Only few studies (4 out of 29) have referrence to
the belief that white examiners impede the
performance of African American children
(Sattler, 1979a)
Why Few Studies Show Effects of
Examiners Race on IQ tests

The procedures for properly administering an
IQ test are specific
Most standardized test require a strict
administration procedure. However, the
examiner can still communicate a hostile or a
friendly atmosphere, or an inquisitorial or
therapeutic role
Examiner effects tend to increase when they
are given more discretion about the use of the
test
OLSAT test self reading vs. audiotaped
instruction
Bias in the way Items are
presented
Comparison between African American and
White Children on Preschool Language
Assessment Instrument
2 conditions: standard vs. use of context and
themes
AA children scored higher when items are
administered in a thematic mode
Significant increase in performance on more
complex and difficult items (Fagundes, Haynes,
Haak, & Moran, 1998)

Otis-Lennon School Ability Test
(OLSAT)

It was hypothesized that AA children
may score lower on IQ tests due to
poorer reading skills
4
th
to 5
th
grade AA students who heard
an audiotaped version scored
significantly higher on the test (Warner-
Benson, 2001)

Although standardization is required,
VARIATIONS from the standard might reveal
some unnoticed skills in disadvantaged test
takers

What can be done?

Though the examiners race have minimal
effects on IQ scores, should it be taken for
granted or disregarded? Why?

Language of the Test Taker
Issues concerning testing individuals with
different linguistic backgrounds
Some tests are inappropriate for people who
have limited knowledge of the language used in
a test- What is compromised?
In original test vs. translated test, what may be
the primary concern?
If examinee is bilingual, which version of the
test should be used?
Any opinion on test interpreters?
Training of Examiners
Different assessment procedures require
different levels of training (ex. License for
psychometrician)
SCID- Structured Clinical Interview for
DSM-IV
Used for psychiatric diagnosis, are
administered by licensed psychologists
WAIS-R
10 practice administrations to develop
competence

Expectancy Effects
Expectancies shape our
judgments in many important
ways (Kirsch, 1999)

Lasallians, UP students,
Thomasians, Paulinians?

Expectancy Effects
Also called Rosenthal Effects (Robert
Rosenthal from Harvard University)
2 groups of experimenters were led to believe
that average responses will fall on success or
failure (1 group expects success, the other,
failure)
Subjects provided data that that confirm the
experimenters expectancies
Experimenters influence is not limited to
human subjects = maze bright vs maze dull

2 aspects of Expectancy Effects in the
Use of Standardized Tests (Rosenthal)
1. Observed even when all of the experimenters
followed a standardized script; expectancy
effect results from subtle non-verbal
communication between experimenter and
the subject (Rosenthal)
2. Has a small and subtle effect on scores and
occurs in some situation but not in others
May affect in the scoring of tests (told that
subject is bright)

*Examiners must be aware that interpersonal and cognitive
processes can affect objectivity in administering or scoring
of tests.
A review of many studies suggests that an
expectancy effect exists in some but not in all
situations
What person characteristic may be implicated
in expectancy effect?

It is important to pay attention to the potentially
biasing effect of expectancy and to eliminate it
as possible

Effects of Reinforcing
Responses
1. Examiners should always administer tests
under controlled conditions
2. Inconsistent use of feedback can damage the
reliability & validity of test scores
3. Most potent effects of reinforcement arise in
attitudinal scales
4. Reinforcement & feedback guide the
examinee toward a preferred response
* Examiners should exert strict control over the use of
feedback
Psychological Testing in
Everyday Life: Incentive
Scoreboard
Effects of token and social reinforcement
Sattler (1988) reviewed 34 studies
Evaluated the effect of incentives: tokens, praise,
candy, money, and social reinforcement
Subjects in the experiments are normal and
handicapped children of various ethnic groups
41 % (14 studies): Incentives or feedback did not
affect performance
38 % (13 studies): Mixed results
21 % (7 studies) : Evidence that reinforcement either
improved or hindered performance
One issue raised by research on incentive
effects concerns what the results imply for test
interpretation

What important characteristic of the test is
affected?

Test Manuals
1. Should clearly indicate directions for
administration
2. Should be detailed to be duplicated in all
situations in which the test is given
3. A good manual includes the exact words to
be read to the subjects
4. Include FAQs and how examiners should
answer them
*Standardized Administration: Ensures the reliability &
validity of the test
Advantages of Computer-Assisted
Test Administration
1. Excellence of standardization
2. Individually tailored sequential administration
3. Precision of timing responses
4. Release of human testers for other duties
5. Subjects are not rushed
6. Control of bias
7. Objective and cost-effective
8. Allow for more experimental control
9. Reduced scoring errors
*What may be the disadvantages/possible
problems?
Mode of Administration
Tests that are self administered vs test
that are administered by an examiner or
trained interviewer

People are more likely to show good health
during interviews than self-completed
questionnaires

Phone interviews yielded higher health scores
than filling out questionnaires

Example
Test Administration Types and Self-
Reporting
Colon and rectal cancer screening tests
Mail, phone, face-to-face
Mode of administration did not make
much difference; equally accurate when
compared to medical records

In educational testing
Reading test administered to K to 12
students
Computer-administered vs paper-and-
pencil tests
No significant differences between
modes (Wang, Jiao, Young, Brooks, &
Olson, 2008)

Mode of asking questions makes a
difference (psychiatric disability)

For younger, more distress and disability
were reported in questionnaires than in
interviews (using the same
questionnaire)

OUTLINE
Relationship between
examiner and examinee

Training of Test
Administrators
Computer-Assisted
Administration
Race of the Examiner
Expectancy Effects

Mode of Administration

Language of the Examinee
(Test Taker)
Effects of Reinforcing
Responses
Subject Variables

Reactivity Expectancies Statistical Control of Rating
Errors
Drift Deception
Subject Variables
A serious source of error
Motivation and anxiety can greatly affect test
scores
Test anxiety difficulty focusing attention on the
test items and distracted by other thoughts
I am not doing well, I am running out of time
(Sapp, 1999)
3 components: Worry, Emotionality, &Lack of self-
confidence
Illness, health concerns, hormones may also
affect perceptual, spatial and motor performances

Activities for Next Week
1. Lab on Test Administration (Manual)
2. Quiz
3. Discussion on Interviewing
Techniques (read Chapter 8; pp 182-
220)


Psychological Testing Test Administration1

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Psychological Testing Test Administration1

Încărcat de

Drepturi de autor:

Formate disponibile

Kaplan, R.M., & Sacucuzzo, D.P. (2011). Psychological Testing.

Cengage Learning Asia Pte. Ltd: Singapore.

S-ar putea să vă placă și