ELEMENT 3a

Collecting data and asking questions

3a: COLLECTING DATA AND ASKING

QUESTIONS
Katherine Weare

Introduction
The conclusions drawn from any piece of research are only as good as
the data they are based on. In qualitative, as in quantitative, research you
must take great care not only in selecting whom you ask for information,
but what you ask them. This, along with techniques for recording your
data, is the main focus of this Element.
By the end of this Element you should:
be clearer about some of the general issues involved in data collection
know what considerations need to be taken into account in asking
questions, including:
question wording
question sequence
asking open, semi-directed or closed questions
some of the 'do's and donts of question wording
asking particular types of question

Recommended reading
The recommended readings in this Element come from these two books.
In the text we shall refer to them by their authors surnames only.
Cohen, L.
Manion, L and
Morrison, K.

(2000)

Research Methods in Education, 5th Edition,

RoutledgeFalmer.

Oppenheim,
A. N.

(1992)

Questionnaire Design, Interviewing and

Attitude Measurement, Pinter.

3a.1

ELEMENT 3a

Collecting data and asking questions

Section 1: Overview of methods of data

collection
You probably already know more than you think about methods of data
collection and recording.

T
Allow yourself 30
minutes for this
task

Task 1
Do not look anything up for this task. Use only the knowledge you
have so far acquired about research methods to make a list of as
many of the following that you can think of:
research methods
methods of recording data.

Now turn to the next page for the feedback on this task.

3a.2

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 1
You might have included the following:
Research methods
Unstructured fieldwork
participant observation
interviews: from open ended to highly structured
face-to-face interviews
telephone interviews
focus groups
projective method for self completion (e.g. draw and write, bubbles)
diaries
video taping
observation
self-completion questionnaire: from open-ended to highly structured
documentary analysis (secondary sources).
Methods of recording data
Tape recording e.g. of interviews of individuals or groups in
discussion
making notes at the time (e.g. of interviews, observations)
making notes afterwards (e.g. of interviews, observations)
projective methods for self completion (e.g. draw and write, bubbles)
diaries
video taping
observation schedules: covert
observation schedules: overt
questionnaire filled in by the researcher during the interview
'postal' questionnaire.

Choosing your method of data collection

How do you go about choosing from this long list? The essential
requirement is clear thinking, about what you want to achieve on the one
hand and the resources available to you in your endeavour on the other.
Task 2 should help you begin to clarify your thinking about this. Before
getting involved in practical considerations, think carefully and decide
the single most important factor influencing your decision.

3a.3

ELEMENT 3a

T
Allow yourself
about 10 minutes
for this

Collecting data and asking questions

Task 2
Look again briefly at Chapter 1 of Cohen, Manion and Morrison.
What do you think should be your main consideration in deciding
which method to use?

Feedback on Task 2
We hope you said that your choice is fundamentally dictated by why you
are collecting data. If you are not sure of your aims, get those clear first
before even considering collecting data.
Now you are ready to move on to the practicalities. Are you going to use
one method, or perhaps two to provide backup/greater depth?

T
Allow yourself
about 30 minutes
to do this

Task 3
Let us assume then that you already know why you are collecting
data. Again, in the light of your reading of Chapter 1 of Cohen,
Manion and Morrison, what practical considerations might you
take into account in choosing a method or methods of data
collection?
Here is an example to start you off.
How much structure/lack of structure can your methods
comfortably accommodate? (How open/closed can they be?)

3a.4

ELEMENT 3a

Collecting data and asking questions

Now look on the next page for a list of considerations to take into
account in choosing your data collection method(s).

3a.5

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 3
Considerations you might have raised include:
How the method used affects the validity of what is being studied:
what is the relationship between what the respondent says or does
while data is collected and what they say or do in other contexts? How
intrusive are the methods?
How time consuming are they to administer, complete and/or to
analyse afterwards? Do you have the time to undertake them in
general, and have you left enough time for each stage of the process?
How verifiable are they by another researcher: can they be checked at
all? If they are very 'subjective', should you think about involving
someone else in this way? Is anyone actually available?
How reliable are they: would another researcher agree with your way
of analysing your data? Again, do you have the facility to check this?
What skills do they demand from the researcher? e.g. tact, empathy,
charm, logicality, ability to handle numerical data etc. To what extent
do you possess these?
What skills do they demand from the respondent? e.g. level of
literacy, verbal fluency. Do your respondents possess these?
How interesting are they are for the respondent? They will need to
be to arouse their interest and encourage them to cooperate with you.
You need to consider all these issues, and make your choice accordingly.
Once you have chosen your method(s) there are some further points to
consider before you wade in.
If data is not needed or not likely to be useful, do not collect it. Never
'ask for the sake of it'.
The method of recording data used must be systematic, logical,
rational, clear, neat, easy to read etc. Make sure that your data is
labelled carefully (who, what, when etc) so you can go back to it later
after a break and still make sense of it.
Never rely on anyone other than yourself to collect data (unless they
are a paid research assistant and, even then, liaise with them about
research progress on a regular basis and random check their work).
This includes distribution of questionnaires etc. It is best to collect a
small amount yourself and know it is reliable. If you use others, in the
hope for more data, you are liable to wind up with inconsistently
collected data, or at least data you cannot absolutely rely on.
Look after your data carefully. The method used and accompanying
documents must be understandable to an outsider. They must be
durable, and you should preserve them at least until your report is
circulated/published and is becoming 'old news' or your dissertation

3a.6

ELEMENT 3a

Collecting data and asking questions

has 'passed'. Never cut up original data collection instruments. If they
are on paper make photocopies and do the actual work with them.

Section 2: Asking questions

Imagine you are having dinner at a restaurant and you ask, Do you have
any mustard? The waiter replies, Yes, but makes no move to fetch it
for you. Did you ask the right question? Should you have asked May I
have some mustard please? Asking questions is a tricky business. There
are six main issues involved.

Issue 1: Preliminary considerations

Where do you, the questioner stand? What are your axioms and
attitudes?

T
This should take
you about 20
minutes

Task 4
Do not do any further reading for this task. Base your answer on
your work so far and your common sense. There are a number of
important questions you must ask yourself before you inflict any
on your informants. What are they? List them here.

Now turn to the next page for the feedback on this task.

3a.7

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 4
You could have said all kinds of things in response to this task. Some
issues you might like to consider include:

This is vital for both

quantitative and qualitative
data. You must have a fairly
clear idea of what you
intend to do with your data
before you start to collect
it.

Do I know why I am asking these questions, of these respondents?

Are the aims of my study sufficiently clear to me and to my
respondents?
Have these questions been asked in other studies? If so, why am I
asking them? To replicate? To compare? What does previous
research lead me to expect?
Is my respondent likely to possess the information I am seeking?
Can I legitimately ask these questions? Is it sufficiently impersonal,
and not too sensitive an issue?
Do I know how I am going to record the data I collect? Do I have the
equipment? Have I practised the skills?
Do I know how I am going to process the data I collect? If it is going
to be very open ended, do I have the time to do this?

Issue 2: Question wording and question sequence

The Preface of Oppenheims book starts, The world is full of wellmeaning people who believe that anyone who can write plain English
and has a modicum of common sense can produce a good questionnaire.
Obviously this is not so. His 300 page book has become almost a bible
for those who realise this.

T
Allow about 1 hour
for the reading and
20 minutes for
writing

Task 5
Consider Chapter 8 of Oppenheim. Identify the problems of
question wording
question order.
Make some notes on the main general issues to bear in mind
when you are working these out.

3a.8

ELEMENT 3a

Collecting data and asking questions

Now turn to the next page to find the feedback on this task.

3a.9

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 5
You might have mentioned some of the following issues.
Question wording is much harder than it looks. Suppose you are hoping
to discover the nature of the respondent's attitude to a particular concept
but he/she has none. Your attempt to discover and code the 'attitudes'
may produce distortion, through problems of e.g. understanding,
knowledge, wishful thinking, desire for social approval or desire and
ability to communicate. You must:
decide clearly what you want to know
think about each word in the question and what it will mean to a
range of respondents
phrase the question tentatively so as not to load it with the
assumption that the respondent has the knowledge or attitude you are
seeking
use familiar words
keep the questions short
avoid double negative questions
clarify the 'how often' options (which often produce 'wishful
thinking' answers)
avoid 'leading questions' which suggest the 'correct' response
be aware of 'prestige bias', and try to phrase your question so that a
'low prestige' answer is possible/acceptable
avoid questions which require admissions of foolish behaviour
try to avoid the respondent having to 'lose face' (vital in some
cultures)
maintain rapport, respect and consideration
keep open-ended questions to a minimum
leave questions about the respondent's personal details to the end
avoid asking unnecessary questions.
Question sequence is vital.
You should start with the easy, impersonal ones.
You must not suggest answers to later questions through the phrasing
of the earlier ones.
You should use filter questions to get rid of those respondents who do
not fit your research question.

3a.10

ELEMENT 3a

Collecting data and asking questions

Issue 3: Open or closed questions

We have just suggested that you minimise the number of open questions.
Why? Let us look at this in more detail.

T
This should take
you about 40
minutes

Task 6
Again, using Oppenheim's Chapter 8 to help you, have a go at
phrasing one example of each of three types of question.
Completely open ended
semi-structured/ semi-directed
closed.
When would you use each?

Now turn to the next page for the feedback on this task.

3a.11

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 6
Before deciding what type of question you want to ask, you need to
decide what sort of information you want to obtain. Open-ended
questions explore the views of the respondent. Closed questions are
suitable if you want to count the number of people in different
categories.
Completely open-ended questions are the stuff of which phenomenology
(the description and classification of phenomena) is made. This is too big
to be expected of you here. Nevertheless, within a study with a more
structured intention, you can often use them in the design stage to
formulate pre-coded questions or when quotable responses are required.
Sometimes they are the only way of obtaining information.
Their disadvantage is that they have to be coded later, so they would not
be suitable for a postal questionnaire because analysing them would be
time consuming, or even impossible. Here is an example of an openended question.
"What do you hope to do when you leave school?"
A semi-directed question is a compromise. It partly limits the response.
For example:
What is it about this form of exercise that appeals to you?"
A closed question is easy to ask and easy to code. It can have an openended response, for example:
"How many cups of coffee did you drink yesterday?"
Closed questions may also be answered in a pre-coded way (with boxes
to tick, a continuum to put a ring round etc). Factual questions are often
asked by this method (but you must know what categories of answer you
expect.) The categories must be mutually exclusive and exhaustive. In
other words they must not overlap and they must cover all possible
answers.)

3a.12

ELEMENT 3a

Collecting data and asking questions

Issue 4: Asking closed questions

Closed questions are very useful to the researcher. They are clear-cut and
easy to categorise. Properly formulated they are surprisingly versatile.

T
Give yourself about
30 minutes for this
one

Task 7
Think about closed questions. How many different types can you
think of? (Think about the different types of answer you might get
from these questions.)

Now turn to the next page for the feedback on this task.

3a.13

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 7
You might have included some of the following types of closed question.
Dichotomous questions are those where only one of two answers is
possible. They are the easiest to ask, to answer and to code, and there
is little danger of interviewer bias. Yes/no questions are the bestknown example, e.g.
"Did you, yourself, take exercise yesterday?" YES/NO
You should include a 'don't know' category where appropriate.
Multi-choice or cafeteria (choose from this selection) questions list a
choice of answers. Care must be taken with exclusivity. For example,
it would not be helpful to list categories for age as:
20-30
30-40
40-50
50 or over.
What would you tick if you were 30? Categories must be exhaustive.
That is, they must include every possible response. It is almost always
essential to include 'other' as a category. You must take care that all
the important categories are specifically mentioned, if not some
may be under-reported.
Value judgement (rating scales). These are of two types:
Ordinal: e.g. where people are asked to rank different foods by
preference. They are not straightforward to code.
Interval: (e.g. semantic differential questions). These provide a
scale with extreme options at either end.
e.g. Please place a tick on the scale.
The dinner yesterday was:
Terrible- - - - - - - - - - - - - Fantastically good
The main problem with this kind of pre-coded questions is that
respondents are forced into making a choice, and they may not have such
a clear opinion. e.g.
Do you think that the last teacher you visited was good or bad?
In most cases you need more than just two categories, or perhaps more
questions on the issue

3a.14

ELEMENT 3a

Collecting data and asking questions

Issue 5: Types of wording to avoid

Now for some of the donts.

T
Give yourself about
an hour for the
reading and 30
minutes for the
notes

Task 8
Consider Oppenheim's Chapter 11. What type of questions do you
think we should avoid? Make notes on some pitfalls.

Now look at the next page for the feedback on this task.

3a.15

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 8
You might have mentioned the following as questions to avoid:

Non-specific questions
questions using complex and difficult words or jargon
ambiguous questions
questions which depend on vague words e.g. occasionally; how often;
regularly; many; why did you; etc without defining what you mean by
them
Leading or loaded questions, e.g. You don't think do you that......?
Shouldn't something be done about.....? Do you agree that.....?'
long and complicated questions
double-barrelled questions: two questions in one
questions that assume e.g. 'How many cigarettes do you smoke per
day?'
hypothetical questions
double negatives
patronising questions
unneccessarily personal or sensitive questions.

Issue 6: Phrasing questions

Once you have decided what you want to ask questions about, in what
order you want to ask them and what sort of questions you want to ask,
you must think about the actual words you are going to use.

T
Allow about 1 hour
for this task

Task 9
Refer again to Oppenheim's Chapter 11 in some detail. Make
some notes on what you think are the main issues to bear in mind
in phrasing questions on the following.
Facts

Opinions

Motivations

3a.16

ELEMENT 3a

Collecting data and asking questions

Knowledge

Questions involving memory. (Think about sources of error)

Sensitive questions. (Think about the methods you would suggest

for dealing with sensitive questions)

Now turn to the next page for the feedback on this task.

3a.17

ELEMENT 3a

Collecting data and asking questions

Feedback on Task 9
You might have mentioned the following issues.
Factual questions
The words used must be clearly defined, to both the interviewer and the
respondent (if relevant). Here are some problem questions:
How long have you been working here?
How many bedrooms have you in your house?
What is your occupation?
Can you see why these might be problem questions?
Opinion questions
These can be difficult to get right, as opinions may not exist, may be
many-sided and may be strong or weak. Responses to these are very
sensitive to changes in question wording and sequence. It is difficult to
cover all possible questions in a pre-coded question.
Motivation questions
e.g. Why did you do this course?
These can be either open-ended or multiple choice. They share many of
the difficulties of opinion questions. Answers should be graded if
necessary (some factors might have a strong influence on your results,
others a weak influence).
Knowledge questions
These are often open-ended and designed to sort people into categories of
informed/not informed. They may be tests or more subtle questions
where the respondent may think that he is stating an opinion. Care must
be taken not to sound patronizing, or to make the respondent feel inferior.
Questions involving memory
Most factual questions involve memory. Two factors influence memory:
length of time since event.
relative importance of event to respondent (salience).

3a.18

ELEMENT 3a

Collecting data and asking questions

Two types of error occur:
recall loss: the respondent may forget something in the time period.
Use 'aided-recall methods'. e.g. ask a question like this
What different magazines did you read last month?
and give a list of possibilities.
the telescoping effect: the respondent may claim to have done
something in the time period when he has not. Use bounded-recall
methods to try and fix time, e.g. Was that before or after you ...?
Sensitive questions
These include:
socially unacceptable questions (about sexual perversions, excessive
alcohol consumption, drug abuse, breaking the laws or customs of a
society)
questions about private matters (income, sex, religion, personal
health)
questions which elicit low prestige answers.
These are quite a large number of techniques for asking questions on
sensitive topics. These include:
a casual, matter-of-fact approach (as if you do not think the
respondents action or attitude is odd or especially important)
a numbered card approach whereby the respondent ranks items
anonymously
an everybody' approach, i.e. Everybody has some kind of little 'vice'
that they know is not healthy but which they enjoy. What would you
say yours are?
an 'other people' approach; this is much the same as the 'everybody'
approach
a sealed ballot technique to guarantee anonymity
the use of familiar words
questions about past behaviour before you ask about present
behaviour
an embedding of the topic of interest in a list of more and less
threatening topics
an open-ended question approach
the use of sentence completion or cartoons
a reverse approach; where the respondent may not like to admit that
he has not done something. Do you intend to... may get to this
You can make an indirect check on validity by asking your respondent
if they were made uneasy by the question.

3a.19

ELEMENT 3a

S
This should take
about 20 minutes

Collecting data and asking questions

Summarising task
Make your own summary of this Element. What do you feel are
the key things you have learned from this chapter about collecting
data and asking questions?

You will have a chance to start practising these skills in a rather more
'real life' situation in the next Element. It is about interviewing, and you
will be asked to construct and administer an interview.
You should also keep the work of this Element in mind when you go on
to look in more detail later at quantitative methods and survey design, as
the skills of questioning are vital there too.

3a.20