Measurements and mismeasurements in

psychology
Instructor: Assistant Professor Kai Qin Chan
Ashoka University
Spring 2016

Course description
Psychologists are obsessed with measuring outcomes. But measuring
psychological constructs such as happiness is not quite the same as measuring
physical constructs such as the length of a table. Does it mean that psychologists
should give up on measuring psychological constructs, or should the question be
how best can we measure them? What sort of mistakes do we make during
measurements, and what sort of consequences do these errors have with regards
to our inferences about other people? The aim of this course is to nurture a
critical eye at the nature of measurement in psychology. At the end of the course,
students will appreciate what measurement really is about.
Prerequisites
None.
Course goals
By the end of this course, you will be able to:

Understand the merits and limitations of measurements

Evaluate statistical information with a critical eye
Understand what makes a good measurement
Understand that even the best methodology cannot salvage the worst
measurements.

Evaluation
Research Topics summary: 20% (i.e., 10 summaries, 2% each)
Presentation: 10%
Project: 30%
Exam essay: 40%
Optional book
Smith, G. (2012). Standard deviations. London, UK: Duckworth Publishing.
Wanjek, C. (2003). Bad medicine. Hoboken, NJ: Wiley.
1

Seminar schedule
We
ek
1

Session 1

19/1

Introduction:
Measurement

21/1

Session 2
(Measurement challenges
in)
Psychometrics

No class. Complete these three questionnaires before 26 Jan:

http://www.surveygizmo.com/s3/2494655/CTS-Personality
We will discuss Questionnaires (1) and (2) in Week 3; we will discuss (3) in
the Week 15.
2/2
Personality psychology
4/2
Personality assessment

9/2

*Research topic: MBTI (do

not confuse it with MMPI)
Clinical psychology:
Projective tools

11/2

Clinical diagnoses: DSM-IV

and other clinical
assessment tools

18/2

Nonverbal communication

*Research topic: Rorsarchs

Inkblot Test

6
7
8

16/2

8/3

15/3

*Research topic: Thematic

Apperception Test
Law: Polygraphy

*Research topic: Lie detector

Cancelled because of Jat riots
Midsem break (27/2 6/3)
Organizational
10/3 Personnel selection
psychology: Graphology
*Research topic:
Handwriting analysis
Biophysiological
psychology:
Dermatoglyphics

17/3

Neuroscience

*Research topic: Fingerprint

analysis (dermatoglyphics);
Im not talking about those
used by forensic
psychologists in criminal
investigations.
2

*Research topic: 2D:4D

ratio. In your summary, also
write down the length of
your index (2nd) and fourth
finger.
10

22/3

Intelligence

24/3

*Research topic: IQ tests

(there are many out there,
you can select the ones that
interests you the most)

11

12

13

14

29/3

5/4

12/4

19/4

Rationality

Student presentation: 5,
6, 7, 8
You will be grading your
classmates presentation,
just as they would be
grading yours when its your
turn.
Student presentation:
13, 14, 15, 16
You will be grading your
classmates presentation,
just as they would be
grading yours when its your
turn.
Student presentation:
21, 22
You will be grading your
classmates presentation,
just as they would be
grading yours when its your
turn.

Holi Day
No class.

31/3

7/4

Student presentation: 1,
2, 3, 4
You will be grading your
classmates presentation,
just as they would be grading
yours when its your turn.
Student presentation: 9,
10, 11, 12
You will be grading your
classmates presentation,
just as they would be grading
yours when its your turn.

14/4

Student presentation: 17,

18, 19, 20
You will be grading your
classmates presentation,
just as they would be grading
yours when its your turn.

21/4

Methodological reasoning
*Research topic: Paul, the
World Cup psychic octopus.
Did Paul indeed have psychic
powers?

In the second half of the

3

class, we will discuss

consciousness.

15

26/4

*Research topic: Find out

about quantum
consciousness in Google.
Do you think the theory is
true?
Summary: The Ten
Commandments of
Scientific Thinking

28/4

Open consultation for takehome exam

*Research Topic Summaries (20%, i.e., 2% for each submission)

There are 10 Research Topics. Research on the topic for that week. Often, using
Google will be enough. Submit a summary (max. 1 page) about the topic. In your
summary, be sure to outline general principles. See example on BrainCap TM in
Week 1s lecture. Write your summary in proper prose form, not bullet points or
single-sentence paragraphs.
Each submission will be graded as follows:
0% - No submission or minimal effort
1% - Adequate summary
2% - Excellent summary
We will discuss in class whether this is a measurement or mismeasurement. In
this class discussion, you will share with your class what you found and your
thoughts about it.
Each Research Topic Summary is worth 2%. You should submit each Research
Topic Summary at the start of that particular class (the exception is when you
have an excused absence); otherwise you will receive 0%.
Optional readings
Wee
k
1

References
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological
tests. Psychological Bulletin, 52, 281-302.
Boynton, J. (2012). Facilitated communicationwhat harm it can do:
Confessions of a former facilitator. Evidence-based Communication
Assessment and Intervention, 6, 3-13.
Jacobson, J. W., Mulick, J. A., & Schwartz, A. A. (1995). A history of
facilitated communication: Science, pseudoscience, and antiscience
science working group on facilitated communication. American
Psychologist, 50, 750-765.
4

Mostert, M.P. (2001). Facilitated communication since 1995: A review of published studies. Journal of
Autism and Developmental Disorders 31, 287-313.

2
3
4

No lesson
Paul, A. M. (2005). The cult of personality testing. New York, NY: Free
Press.
Lilienfeld, S. (2011). Distinguishing scientific from pseudoscientific
psychotherapies: Evaluating the role of theoretical plausibility, with a little
help from Reverend Bayes. Clinical Psychology: Science and Practice, 18,
105-112.
Rosenthal, R. (2003). Covert communication in laboratories, classrooms,
and the truly real world. Current Directions in Psychological Science, 12,
151-154.
Vrij, A., Granhag, P. A., Mann, S., & Leal, S. (2011). Outsmarting the liars:
Toward a cognitive lie detection approach. Current Directions in
Psychological Science, 20, 28-32.

Chan, D. (2005). Current directions in personnel selection research.

Current Directions in Psychological Science, 14, 220-223.

7
8

Mid-semester break
Bennett, C., Miller, M., & Wolford, G. (2012). Neural correlates of
interspecies perspective taking in the post-mortem Atlantic salmon: An
argument for multiple comparisons correction. NeuroImage, S125-S125.

10

Miller, G. A., & Keller, J. (2000). Psychology and neuroscience: Making

peace. Current Directions in Psychological Science, 9, 212-215.
Weinberg, R. (1989). Intelligence and IQ: Landmark issues and great
debates. American Psychologist, 44, 98-104.
Stanovich, K. (2009). What intelligence tests miss: The psychology of
rational thought. New Haven, CT: Yale University Press.
Bargh, J. A., & Williams, E. L. (2006). The automaticity of social life.
Current Directions in Psychological Science, 15, 1-4.

Westen, D. (1999). The scientific status of unconscious processes: Is

Freud really dead? Journal of the American Psychoanalytic
Association, 47, 1061-1106.
11
12
13
14

15

Student presentation week

Bartoshuk, L. (2014). The measurement of pleasure and pain.
Perspectives on Psychological Science, 9, 91-93.
Boot, W. R., Simons, D. J., Stothart, C., & Stutts, C. (2013). The pervasive
problem with placebos in psychology: Why active control groups are not
sufficient to rule out placebo effects. Perspectives on Psychological
Science, 8, 445-454.
De Craen, A. J. M, Kaptchuk, T. J., Tijssen, J. G. P., Kleijnen, J. (1999).
Placebos and placebo effects in medicine: Historical overview. Journal of
the Royal Society of Medicine, 92, 511-515.
Lilienfeld, S. O. (2005). The 10 commandments of helping students
distinguish science from pseudoscience in psychology. Observer, 18.
5

Available here:
http://www.psychologicalscience.org/index.php/publications/observer/200
5/september-05/the-10-commandments-of-helping-students-distinguishscience-from-pseudoscience-in-psychology.html
Project (30%)
In this course, you would be exposed to many psychological measurements
(MBTI, 2D:4D, IQ tests, handwriting analysis, Rorsarchs Inkblot, Thematic
Apperception, Mousetracking, Eyetracking, fMRI, polygraph, fingerprint analysis,
etc.), where the validity of them may be good, some questionable, or some just
plain bogus. In this project, you will devise a fake psychological test, but you
have to make it sound convincing. The aim is learn how to think and write good
science, by first learning how to think and write bad science.
Your task is compose a website or brochure to advertise and sell this test. Find
out how other advocates of the measures have sought to convince laypeople
about the merits of their measurements. Then, write down short statements
about the basic principles of the measurements, in order to convince another
layperson that the measurement is scientific. Hence you will be judged how
convincing you are at getting a layperson to believe that your test is valid.
Tips:
1. You should use the critical thinking skills learnt from Weeks 1-10 to help
you.
2. You can be as creative as you want, but also realize that extremely creative
(crazy?) ideas may also be deemed as less credible. You should learn to
strike a balance between being creative and being realistic.
3. Incorporate feedback from your classmates during your Presentation where
appropriate.
Deadline: 21 Apr 2016. For every 24 hours or part thereof that you are late, 5%
will be deducted from your final project grade.
Check the grading rubrics uploaded on my website to know what I am looking for.

Presentation (10%)
Each student has maximum 20 mins. Prepare a 10-15-minute presentation of
your website/brochure, leaving the rest of the time to gather feedback from your
classmates. Treat this as a sales pitch to sell your product. Your classmates will
score you on the below grading components. The average of their scores will be
calculated and this would form 10% of your grade for this course.
Grading components for Presentation
6

Component

I would pay for the product.

1
NO

2
no

A highly educated person would be convinced.

NO

no

A person without much education would be

convinced.
The arguments are believable.

NO

no

NO

no

I would trust the results of the product.

NO

no

I would not hesitate to use the product.

NO

no

3
mayb
e
mayb
e
mayb
e

4
yes

5
YES

yes

YES

yes

YES

mayb
e
mayb
e
mayb
e

yes

YES

yes

YES

yes

YES

Exam essay (40%) Max 2000 words

Answer only ONE of the questions:
(1) How can we differentiate measurement from mismeasurement? Discuss
with examples, wherever possible.
(2) Why do people belief in pseudoscientific psychology?
Deadline: 4 May 2016. For every 24 hours or part thereof that you are late, 5%
will be deducted from your final exam essay grade.
Grading criteria for both essay and project
See Grading Rubric.
Office hours
I am most free on Mondays and Wednesdays. Just come by Office 608 no need
to make appointment.