Curs 7.

Metode de masurare
Curs Metodologie (II) / Dragos Iliescu
 O sa vorbim despre ...
1. Utilitatea metodelor: Screening vs.
comprehensive
2. Calitativ vs. Cantitativ: lupta pentru
cantitativizare
3. Tipuri de instrumente: Self-assessment vs.
altele
4. Norm-referenced vs. Criterion-referenced:
necesitatea normelor
5. Cum gasim si utilizam instrumentele de care
avem nevoie
Research process - summarized
as 5-step sequence
Statement of the problem

Design of research study

Measurement of variables

Analysis of data

Conclusions from research

 Cum masuram variabilele?
Fidelitate, validitate, blabla

Utilitate
Eficienta (ating toate obiectivele)
Parcimonie (fara sa arunc cu resurse)

Utilitatea are si sens financiar

(see next slide)
 Utilitatea are si sens financiar
Utility in psihologia aplicata (in special in I/O) este cel
mai puternic indicator al unei metode de masurare
Spune daca mai degraba iesim pe plus sau pe minus din
aplicarea unei metode

Face apel la principiul validitatii incrementale

Fiecare metoda in plus contribuie cu varianta mai putina si mai
putina si mai putina
Fiecare procent de variant prezisa se reflecta in gain financiar,
iar asta e cuantificabil
Cunoastem costurile pentru fiecare test
Momentul cand costul per test este mai mare decat gainul per
test este momentul cand utilitatea testului este sub zero
 Screening vs. comprehensive
Comprehensive assessment:
evaluare completa
cea mai buna evaluare posibil de realizat
Golden Standards, e.g.
Conners-3 pentru ADHD
Wechsler (e.g. WISC-IV) pentru inteligenta
ADOS pentru autism
etc.

Screening:
quick and dirty
 Un exemplu
Imaginati un studiu (aplicativ) in care se
evalueaza populatia scolara din clasele mici
(clasa 0) pentru intarzieri de dezvoltare

WISC-IV

Denver-II (opusuri, defineste cuvinte, Numara,

intelege propozitii simple, cunoaste utilizarea catorva
obiecte)
 Quantitative vs. qualitative
Calitativ vs. cantitativ: care e diferenta?
 Quantitative vs. qualitative
Nu sunt de fapt toate metodele
observationale?

Observatie
...
Interviu
...
Self-report
...
etc...
 Calitativ: mai la indemana
uneori, insa ...
Observatia - limitari
frecventa comportamentelor,
reactivitatea,
observer bias
etc.
Interviul - limitari
onestitatea,
interpretarea intrebarilor,
self-observer bias,
utilizarea pe anumite populatii, de ex. copii
etc
 Insa noi avem nevoie de
MASURARE
Lupta pentru cantitativizare
chiar in evaluare calitativa (de ex. observatie):
cantitativizare pe timp sau pe frecventa etc.
Exemplu:
BASC-2 SOS (Student Observation System)
 Exemplu: BASC-SOS
(Lista de comportamente)
 Exemplu: BASC-SOS
(Fisa de observatie a frecventei)
 Tipuri de instrumente
(measures)
autoevaluare (self-report)
heteroevaluare (ratings by others)
masuri obiective (objective measures)
masuri de laborator (laboratory measures)
 Auto-evaluare (Self-report)
2 limite majore:
response bias
subiectivitate (dar other-report e la fel de
subiectiv)
Impression management
common method variance
effect statistic de inflatare a corelatiilor in
studiile care se bazeaza exclusive pe self-
report
 Distorsiuni in autoevaluare
(Response bias)
distorsiune intentionata
factori care tin de instrument
comportamente dezirabile social
comportamente sensibile
factori care tin de administrare
presiune situationala
factori care tin de persoana evaluata
sensibilitate pe anumite dimensiuni
self-disclosure
distorsiune neintentionata
auto-amagire (crede despre sine lucruri ne-reale)
necunoasterea realitatii (nu cunoaste realitatea)
raspunsuri top of mind (raspunsuri negandite)
 Common Method Variance
Method variance refers to variance that is attributable to the
measurement method rather than to the construct of interest. The
term method refers to the form of measurement at different levels of
abstraction, such as the content of specific items, scale type,
response format, and the general context (Fiske, 1982, pp. 8184).
At a more abstract level, method effects might be interpreted in
terms of response biases such as halo effects, social desirability,
acquiescence, leniency effects, or yea- and nay-saying. (p. 426)
 Subiectivitate vs. Lupta pentru
masurare obiectiva in self-report
ascunderea intentiilor in masurare
de ex. utilizarea de indicatori comportamentali (verificabili) mai
degraba decat indicatori psihologici (emotii, cognitii)
insa studiile arata ca distorsiunea nu scade in aceste conditii

totusi, exista studii care arata ca declararea intentiilor

creste de fapt (in anumite situatii) fidelitatea si
validitatea masurarii
in masurarea personalitatii
in studii de marketing (discutiile despre domeniu ofera un fel de
priming)
 Subiectivitate vs ...
... iar subiectivitatea nu este chiar atat de
invalida uneori
Ex.: Self-Assessed Intelligence (Furnham, 2001)
 Other-report
ce pot sa evalueze alte persoane despre
aspectele psihologice ale unei persoane
focale (focal person)?
dispozitii (caracteristici)
comportamente
 Masurarea de dispozitii prin
other-report
Cum e posibil?
chestionarele clasice de personalitate au si
intrebari care se refera la emotii si cognitii

Solutia:
liste de adjective, e.g.
Adjective Check-List (Gough & Heilbrun, 1983)
Big Five Adjectives (Caprara, Barbaranelli, &
Steca, 2002)
 Big Five Adjectives
Acest chestionar conine o serie de adjective. Citii-le unele dup altele,
decidei n ce msur v caracterizeaz fiecare adjectiv si apoi marcai
rspunsul dumneavoastr n dreptul lui.
1 - COMPLET NECARACTERISTIC PENTRU MINE
2 - NECARACTERISTIC PENTRU MINE
3 - DESTUL DE NECARACTERISTIC PENTRU MINE
4 - NEUTRU
5 - DESTUL DE CARACTERISTIC PENTRU MINE
6 - CARACTERISTIC PENTRU MINE
7 - FOARTE CARACTERISTIC PENTRU MINE

Sociabil, ntelept, Suparacios, Nonconformist, Traditionalist, Expert,

Sistematic, Avar, Afectuos, Logic, Inactiv, Simpatic, Iritabil, Istet, Dominant,
Informat, Instabil, Scrupulos, Competitiv, Curajos, Echilibrat, ntreprinzator,
Curios, Energic, Distant, Capabil, Primitor, Constant, Ordonat, Inconstant,
Indiferent, Dinamic, Vesel, Nemultumit, Deprimat, Perseverent, Inteligent,
Imprecis, Impulsiv, ...
 Masurarea de comportamente
prin other-report
Exemplu:
Conners-3 (Conners, 2008, 2013)
Standardul de aur pentru masurarea ADHD
system multi-evaluator (Self, Teacher, Parent)
Exemplu: Conners-3
 Masuri obiective
(Objective measures)
Exemple:
aptitudini matematice
aptitudini de citire
inteligenta generala
creativitatea
intelegerea verbala
teste scolare
deprinderi motrice
Teste obiective
raspunsuri corecte si gresite
scorul = numarul de raspunsuri corecte sau o variatiune pe
aceasta tema (atenuat pentru ghicit etc.)
 Masuri de laborator
(Laboratory measures)
Physiological measures
diverse metode de imagistica (e.g.,
EEG/electroencefalograma, MRI/magnetic resonance
imaging etc.)
masuri biochimice (e.g., cantitatea de
neurotransmitatori)
activitatea sistemului nervos periferic (e.g., tensiune
musculara).
Response measures
frecventa raspunsurilor (e.g., numarul de cuvinte
reamintite, numarul de stimuli identificati corect)
timp de reactie in sarcina (sau latent in reactive)
 Norm-referenced vs. Criterion-
referenced
Masuratori raportate la criteriu
obiective
mult utilizate in domeniul clinic
dar nu uitati ca si criteriile sunt de fapt stabilite
subiectiv (vezi homosexualitatea in DSM)

Masuratori raportate la norma

relativitate in felul in care se stabilesc cut-off-
uri
 Masuratori raportate la criteriu
(Exemplu)
PTSD (Post-Traumatic Stress Disorder)
DSM 5: Trauma- and Stressor-Related
Disorders
Posttraumatic Stress Disorder: Diagnostic
Criteria: 309.81 (F43.10)
8 criterii:
A. Exposure
B. Presence of intrusion symptoms
C. Avoidance of stimuli
D. Alterations in cognitions and mood
E. Alterations in arousal and reactivity
...
 DSM Criteria for PTSD:
A. Exposure
A. Exposure to actual or threatened death, serious injury, or sexual
violence in one (or more) of the following ways:

1. Directly experiencing the traumatic event(s).

2. Witnessing, in person, the event(s) as it occurred to others.

3. Learning that the traumatic event(s) occurred to a close family member

or close friend. In cases of actual or threatened death of a family member or
friend, the event(s) must have been violent or accidental.

4. Experiencing repeated or extreme exposure to aversive details of the

traumatic event(s) (e.g., first responders collecting human remains: police
officers repeatedly exposed to details of child abuse).
Note: Criterion A4 does not apply to exposure through electronic media, television, movies, or pictures,
unless this exposure is work related.
 DSM Criteria for PTSD:
B. Presence of intrusion symptoms
B. Presence of one (or more) of the following intrusion symptoms associated
with the traumatic event(s), beginning after the traumatic event(s) occurred:

1. Recurrent, involuntary, and intrusive distressing memories of the traumatic

event(s).
Note: In children older than 6 years, repetitive play may occur in which themes or aspects of the traumatic
event(s) are expressed.

2. Recurrent distressing dreams in which the content and/or affect of the dream are
related to the traumatic event(s).
Note: In children, there may be frightening dreams without recognizable content.

3. Dissociative reactions (e.g., flashbacks) in which the individual feels or acts as if

the traumatic event(s) were recurring. (Such reactions may occur on a continuum,
with the most extreme expression being a complete loss of awareness of present
surroundings.)
Note: In children, trauma-specific reenactment may occur in play.

4. Intense or prolonged psychological distress at exposure to internal or external

cues that symbolize or resemble an aspect of the traumatic event(s).

5. Marked physiological reactions to internal or external cues that symbolize or

resemble an aspect of the traumatic event(s).
 DSM Criteria for PTSD:
C. Avoidance of stimuli
C. Persistent avoidance of stimuli associated with
the traumatic event(s), beginning after the traumatic
event(s) occurred, as evidenced by one or both of
the following:

1. Avoidance of or efforts to avoid distressing memories,

thoughts, or feelings about or closely associated with the
traumatic event(s).

2. Avoidance of or efforts to avoid external reminders

(people, places, conversations, activities, objects,
situations) that arouse distressing memories, thoughts,
or feelings about or closely associated with the traumatic
event(s).
 DSM Criteria for PTSD:
D. Alterations in cognitions and mood
D. Negative alterations in cognitions and mood associated with the traumatic event(s),
beginning or worsening after the traumatic event(s) occurred, as evidenced by two (or
more) of the following:

1. Inability to remember an important aspect of the traumatic event(s) (typically due to dissociative
amnesia and not to other factors such as head injury, alcohol, or drugs).

2. Persistent and exaggerated negative beliefs or expectations about oneself, others, or the world
(e.g., I am bad, No one can be trusted, The world is completely dangerous, My whole
nervous system is permanently ruined).

3. Persistent, distorted cognitions about the cause or consequences of the traumatic event(s) that
lead the individual to blame himself/herself or others.

4. Persistent negative emotional state (e.g., fear, horror, anger, guilt, or shame).

5. Markedly diminished interest or participation in significant activities.

6. Feelings of detachment or estrangement from others.

7. Persistent inability to experience positive emotions (e.g., inability to experience happiness,

satisfaction, or loving feelings).
 DSM Criteria for PTSD: ...
Cum ar trebui sa arate un test de PTSD?
evident, intreaba explicit despre fiecare criteriu
Cum se scoreaza o persoana ca pozitiva la
acest test
evident, pe baza regulilor enuntate de DSM

e.g.:
Life Event Checklist (Blake, Weathers, Nagy,
Kaloupek, Charney, & Keane, 1995)
Exemplu: Life Event Checklist
 Instrumente raportate la norma
marea majoritate a instrumentelor psihologice
pentru ca nu exista consortiul pentru extraversie,
initiativa pentru diagnoza nevrotismului etc.

consideram o trasatura psihologica (e.g. un

comportament, o atitudine, o dispozitie) ca fiind
mai accentuate sau mai putin accentuate in
functie de cum se prezinta ea in populatie
Standardizarea scorurilor
 Tipuri de scoruri standardizate
Scoruri bazate pe transformari liniare
e.g. z, T, IQ, stanine, STEN

Scoruri bazate pe volume

e.g. centile, decile, cvartile etc.
 Transformari liniare
Scoruri bazate pe transformari liniare
e.g. z, T, IQ, stanine, STEN

utilizabile in cercetare

fara probleme si pentru statistici

nu este necesara standardizarea:

dau EXACT acelasi rezultat ca si scorurile brute

se recomanda cu putere standardizarea in cateva conditii mai

speciale
pentru centrare (centrarea variabilelor) de ex. in regresie
cand sunt combinate mai multe esantioane diferentiate intre ele
etc.
 Transformari liniare
(Caveat)
atentia la transformarile liniare aplicate
unor distributii non-liniare
de ex. pentru testele clinice, unde
distributia este de obicei puternic
asimetrica
 Centile
Scoruri bazate pe volume
e.g. centile, decile, cvartile etc.

utilizabile in cercetare

doar pentru criterii de incluziune/excluziune, pentru generarea

grupurilor etc

NICIODATA PENTRU STATISTICI !!!

 Cum putem gasi instrumentele
necesare?
Testing books
Testing journals
Testing organizations
Test & test review repositories
...

A Primer on Finding Tests (APA)

http://www.apa.org/science/programs/testing/find-
tests.aspx
 Testing books
(Compendii de scale)
 Testing journals
Applied Psychological Measurement
Behavior Research Methods, Instrumentation, and Computers
Behavioral Assessment
British Journal of Mathematics and Statistical Psychology
Educational Measurement: Issues and Practice
Educational and Psychological Measurement
International Journal of Testing
Journal of Applied Psychology
Journal of Behavioral Assessment
Journal of Educational Measurement
Journal of Personality Assessment
Journal of Psychoeducational Assessment
Measurement and Evaluation in Guidance
Multivariate Behavioral Research
Psychological Assessment
Psychological Bulletin
Psychometrika
 Testing organizations
IAAP (Division 2) The International
Association for Applied Psychology
ITC The International Test Commission
EAPA The European Association for
Psychological Assessment
PS The Psychometric Society
 Test & test review repositories
Buros Institute of Mental Measurements (www.unl.edu/buros)
Mental Measurements Yearbook
Tests in Print
Educational Testing Service (ETS)
Test Collection Bibliographies
Bine, bine: de unde facem noi rost de
instrumente (in Romania)?
Online (atentie la traducere/adaptare)

Profesori

ResearchCentral:
www.researchcentral.ro
 Cum identificam instrumentele
adecvate?
Fundament
Autor
Teorie
Generare
Adaptare
Normare

Dovezi empirice
Dovezi privind validitatea

Dovezi contextuale
Materialele testarii: manualul testului, alte materiale
Referinte in compendii
 Exemplu de rationament:
Despre autorul testului
e nevoie sa fie psiholog sau psihometrician
e nevoie sa fie specialist in domeniul respectiv
e nevoie sa aiba pedigree: recunoastere in zona universitara, sau de constructie de
test

de exemplu:

Testul de inteligenta emotionala MACH

Autor:?
Testul de inteligenta emotionala MSCEIT
John D. Mayer & Peter Salovey

Testul de leadership LP
Autor: echipa x
Testul de leadership MLQ
Bernard Bass & Bruce Avolio
 Obligatie in raportarea cercetarii:
referinte complete despre instrument
...
 Deci, cum identificam
instrumentele adecvate?
instrument suficient de reputat
bibliografie
documentatie: inclusiv scorare, interpretare
etc.

nu neaparat si adaptare culturala

nu neaparat si norme
 Cum obtinem instrumentele
The ITC Statement On the Use of Tests and Other
Assessment Instruments for Research Purposes
 ITC Statement
1. Permission to use tests in research
2. Permission for (re)printing
3. Modification of the test or its components
4. Ethical test use
5. Documentation
6. Conflicts of interest
7. Spread of research editions into
professional practice