WHAT IS A QUESTIONNAIRE?

An instrument (form) to
collect

answers to questions
collect factual data

gathers information or measures

A series of written questions/items in a fixed,

rational order

WHY USING A QUESTIONNAIRE?

A well designed questionnaire:
Gives accurate and relevant information to your
research question
Minimizes potential sources of bias
Will more likely be completed
As simple and focused as possible

Statistical Terms

Population: Discrete group whose members can be defined by a set

of at least one shared characteristic
Sample: A selection of that target population (can also be a
subsample)
Random Sample: Sample in which every member of a given
population has an equal chance of being selected
Margin of Sampling Error (MOSE): Every random sample has a
margin of error (statistical price you pay for not interviewing
everyone)
Response Rate: Rate at which people agreed to participate in
survey. If those who refuse are different from those who agree,
results may be skewed (higher response rate = less chance for skew)

Statistical Terms

Population: Discrete group whose members can be defined by a set

of at least one shared characteristic
Sample: A selection of that target population (can also be a
subsample)
Random Sample: Sample in which every member of a given
population has an equal chance of being selected
Margin of Sampling Error (MOSE): Every random sample has a
margin of error (statistical price you pay for not interviewing
everyone)
Response Rate: Rate at which people agreed to participate in
survey. If those who refuse are different from those who agree,
results may be skewed (higher response rate = less chance for skew)

ADVANTAGES OF QUESTIONNAIRES
Can

reach a large number of people

relatively easily and economically

Provide

quantifiable answers

Relatively

easy to analyse

DISADVANTAGES OF QUESTIONNAIRES
Provides

problem

only limited insight into

Limited

response allowed by questions

Maybe not the right questions are asked

Varying

response

Misunderstanding/misinterpretation

Need

to get it right first time

Hard

to chase after missing data

TYPES OF QUESTIONNAIRE
Face

to face
Telephone
Interviewer
-administrated
By

mail
E-mail/Internet
Self-administrated

SELF-ADMINISTERED QUESTIONNAIRE

Advantages:
Cheap

and easy to administer

Preserves confidentiality
Completed at respondent's convenience
No influence by interviewer

SELF-ADMINISTERED QUESTIONNAIRE

Disadvantages:
Low

response rate
Questions can be misunderstood
No control by interviewer
Time and resouces loss

INTERVIEW-ADMINISTERED
QUESTIONNAIRE

Advantages:

Participation

by illiterate people
Clarification of ambiguity
Quick answers

INTERVIEW-ADMINISTERED
QUESTIONNAIRE

Disadvantages:
Interviewer

bias
Needs more resources
Only short questionnaires possible

Especially on telephone

Difficult

for sensitive issues

Basic Survey Process

Steps:
1. Deciding on a mechanism
2. Drawing a sample
3. Designing the questionnaire
4. Fielding a survey and collecting the data
5. Analysis
6. Reporting

STAGES IN DESIGNING A QUESTIONNAIRE (1)

Planning the study:

Decide on goals
Identify risk factors
Know

the subject

Literature,

Formulate
Postmen

for getting bitten by dogs

experts on dog bites

a hypothesis

more likely to get bitten by dogs than the

normal population

Define

information needed to test hypothesis

Occupation,

owning dog, outdoor activities, attitude

towards dogs

STAGES IN DESIGNING A QUESTIONNAIRE

(2)
Determine study population:
Know

the respondents

Occupation
Special

sensitivities
Education
Ethnic
Language

Questionnaire needs to be adapted to your

population, not the opposite!

STAGES IN DESIGNING A
QUESTIONNAIRE (3)
Design questions:
Content
Format

of the questions

of the questions

Presentation
Coding

and layout

schedule (if appropriate)

Pilot and refine questionnaire

WHAT DO YOU PREFER?

WHAT DO YOU PREFER?

WHAT MAKES A WELL DESIGNED

QUESTIONNAIRE?

Good appearance

easy on the eye

Short and simple

Relevant and logical

High response rate
Easy data summarisation and analysis

QUESTION ORDER

Decide on order of items/questions

Easy

difficult
General particular
Factual
abstract

Where to place sensitive questions?

Be aware of ordering effects!

QUESTION ORDER
Questions should be ordered so as to seem
logical to the respondent
First questions should be relevant and easy
Questions are effectively ordered from most
salient to least salient
Demographic questions should not be
covered at the beginning
Potentially objectionable questions are
placed near the end

QUESTION ORDER
Group

questions by topic/ response options

Starting questions
Simple
With

closed format
Relevant to main subject
Non-offending
Neither demographic nor personal questions
Dont

put most important item last

QUESTIONNAIRE INTRODUCTION

Survey/interview introduction
Who

you are/ what is your ILP/thesis topic

Why you are investigating
Where you obtained the respondents name
How and where you can be contacted
Guarantee of confidentiality
Length of interview (be honest)

Usefulness of study should be clear

to all respondents

CONTENT OF QUESTIONS

Clear focus on research question

Avoid

sidetracking

Avoid

unnecessary information

Demographic information

Contact information (if non-anonymised)

DO:
Use simple wording
Be brief
Be specific

DO NOT:
Be vague
Be condescending or talk down to
respondent
Use biased wording
Use abbreviations or scientific jargon
Use objectionable questions
Be redundant

FORMAT OF QUESTIONS
Adjust to responding audience
Professionals vs. public
Keep sentences simple and short
Define key words
Remember option dont know

FORMAT OF QUESTIONS

Ask for one information at a time

Do you own a dog or have frequent contacts
with dogs?

Yes
No

Use mutually exclusive and exhaustive answer

options
Vertical order of answer options

BE ACCURATE
Do you often touch dogs?

Yes
No

vs.

How often did you touch a dog during the

past 3 months?

Once

Twice

Three times or more

Not at all
Dont know

BE APPROPRIATE
Are you a drunk?
Yes

No

vs.
How often have you consumed
alcoholic beverages during the past 6
months?
Daily

2-6 times/week
Once a week
Less than once a week
Dont know

BE OBJECTIVE
Did you drink the strange brownish drink in
Prague?
Yes
No

Vs.
Which beverage did you consume?
Water
Beer
Wine
Karkad
None of them
Dont know

BE SIMPLE
Did you smoke not less than a mean
amount
of 7 cigarettes/2 days from 1999 onwards?
Yes
No

vs.

Did you smoke an average of 2 pack of

cigarettes/week for the last 5 years?
Yes

No

Dont know

BIAS
Bias = systematic differences in the
measurement of a response

INFORMATION BIAS

Recall bias
Cases

more likely to remember than controls

Observer bias
Different

interviewer different interpretations

Different interpretation of similar questions
Reduce by structured questionnaire

NON-RESPONSE BIAS

Those who respond are different from those who

do not
Telephone

Reduce

Ensure

interviews: more females, elderly

high response rate

Random choice of interview partners
Correct during analysis (eg age, sex)

FORMAT OF QUESTIONS
Two main question formats
Closed

format forced choice

Yes
No
Dont know
Open

Always
Sometimes
Never

format free text

What is your most distressing symptom?

Please describe:
________________________________________________
________________________________________________

OPEN OR CLOSED?
Closed
Advantages:
Simple

and quick
Reduces discrimination against less literate
Easy to code, record, analyze
Easy to compare
Easy to report results

CLOSED QUESTIONS

Disadvantages:

Restricted number of possible answers

Loss of information

Possible compromise:

Insert field others

OPEN QUESTIONS

Advantages:
Not

directive
Allows exploration of issues
to generate hypothesis

qualitative research, focus groups, trawling questionnaires

Used

even if no comprehensive range of alternative choices

Good for exploring knowledge and attitudes
Detailed and unexpected answers possible

OPEN QUESTIONS

Disadvantages:
Interviewer

bias
Time-consuming
Coding problems
Difficult to analyze!
Difficult to compare groups

CLOSED QUESTIONS
1.

Straightforward response
What is your age in years?
How long have you owned a dog?
years
What is your sex (gender)?
Male
Female

Did you stay in Hotel X on 23/7/05?

Yes
No
Dont know

___ years
___

CLOSED QUESTIONS
2. Checklist
Which of the following outdoor
activities
did you do last week?

Running
Walking
Hiking
Cycling
Swimming

CLOSED QUESTIONS
3. Rating scale
Did you do use sunscreen during the following
outdoor activities during the past six months?
Always Sometimes Seldomly Never
Running
Walking
Cycling

CLOSED QUESTIONS

4. Rating scale
Numerical
How useful would you think that information
on
the risk of biting from stray dogs would be?
(please circle)
1
2
3
4
5
6
7
Not at all useful
Very useful
Analogue
How much is your pain severe (put the tick
on the line)
010

CLOSED QUESTIONS
5. Scales for measuring attitude (Lickert)
Stray dogs carry a higher risk of rabies
No, I strongly disagree
No, I disagree quite a lot
No, I disagree just a little
Im not sure about this
Yes, I agree just a little
Yes, I agree quite a lot
Yes, I strongly agree

PROBLEMS AND PITFALLS

Avoid questions that ask two things at once you wont know which bit people are answering:
Have you ever had stomach ache and
diarrhoea?
Ambiguity.....
Do you go to the woods a lot?

PROBLEMS AND PITFALLS

Avoid

jargon/abbreviations/slang

How often do you get up at night to PU?

(pass urine)
Should IVDUs be treated in the community?
Avoid

not mutually exclusive options

What age are you?

16-20
20-25
25-30
35-40

PROBLEMS AND PITFALLS

Avoid leading questions

Do you think that the food in the hotel
made you sick?
Did the hotel staff seem unhygenic to
you?
Do you agree that the hospital staff were
close to exhaustion?

Avoid making questionnaire too long

Typographical / spelling errors

PILOTING AND EVALUATION

Pilot with a similar group of people to your
intended subjects
Highlights problems before starting

Effects

of alternative wording
Overall impression on respondents and interviewers
Final polishing after several amendments

PRESENTATION AND LAYOUT

Clear consistent layout

Adequate

space to answer

Large

font size

Appropriate
Avoid

page breaks

experimental layouts
fancy logos
printed on recycled paper/is an equal opportunity employer etc

PRESENTATION AND LAYOUT

Using colour or printing questionnaire on coloured

paper may help
Use filter questions, if necessary
Give clear instructions about how to answer the
questions

WAYS TO INCREASE PERCEIVED

REWARD

Show positive regard

Say thank you
Ask them for advice
Give social validation
Give a tangible reward
Make the questionnaire interesting
State an upcoming deadline

WAYS TO DECREASE PERCEIVED

COST

Avoid subordinating language

Do not embarrass the respondent
Do not inconvenience the respondent
Make questions appear short and easy
Avoid asking personal information (if it must
be asked, use soft wording and inform about
confidentiality)
Keep subsequent requests similar

CREATE A NAVIGATIONAL PATH

Instructions should be placed right where
they are needed
Matrices are confusing: it is best to order
questions from top to bottom, and left to
right

Use larger font to attract attention

Use color shading to attract attention or

show groupings
Use spacing and similarities to show
groupings

WAYS TO ESTABLISH TRUST

Provide a token of appreciation

Provide a sense of legitimate authority
Make completing the questionnaire seem
important
Remind respondent of previous relationship
with sponsor (if applicable)

SUMMARY
A well designed questionnaire:
Will give appropriate data which allow to answer
your research question
Will minimise potential sources of bias, thus
increasing the validity of the questionnaire
Will much more likely be completed

FINALLY, KEEP YOUR

QUESTIONNAIRE SHORT
AND THE QUESTIONS
SIMPLE, FOCUSED AND
APPROPRIATE