Volume 3 Issue 12 December 2011

Researchers Corner Problems and Issues in Eliciting Data from People

Once a National Information Centre in Bangalore conducted a half-day free seminar on online searching to local invited participants and offered free lunch at the end. In order to elicit feedback, a representative of the organization distributed a (A5 size) short questionnaire along with desserts in the canteen premises. The prominent question was Do you find the seminar useful? Interesting sub-question after two options, namely Yes or No, is If not, why? Yes, eliciting data from people is a primary method of data collection. Numerous surveys, particularly use and user studies, have used questionnaire as a method of data collection and repeatedly asked the question to users about visiting/ using library, borrowing/ reading books and journals and even sought frequency of visit/ use and/ or quantity of use. Are these ways of asking people (respondents) appropriate/ inevitable? And how about the reliability of the data so collected? Questionnaires are also called opinionnaires as they only get opinion of respondents and not facts or accurate data. When more accurate data are available through alternative methods/ sources/ records, we should never resort to opinnionnaires. When inevitable, one should consider critical incident technique, projective techniques and scaling techniques for obtaining more reliable data. Even then, one has to be extremely careful about potential problems in collecting data from people. Critical incident technique is a method of gathering facts (incidents) from respondents on the task just completed on the spot focusing on a particular incident or set of incidents. Projective techniques are disguised / indirect techniques to overcome barriers of awareness, irrationality, inadmissibility, selfincrimination & politeness by way of presenting ambiguous or unstructured stimuli to the respondent so that he brings up his subconscious operations to conscious level in a disguised & projected form & reveals his inner characteristics. It is easier to measure variables than attributes and scaling is a method of changing attributes (a series of qualitative facts) into variables (a quantitative series). In other words, it is a procedure for assignment of numbers or symbols (i.e., quantitative measures) to subjective abstract concepts (or properties of objects). Some problems like irrationality, inadmissibility, ignorance, bias, hallo effect and response set on the part of respondents as well as the very presence of researcher and disclosure of respondents identity apart from the way and the context in which respondents are

questioned can drastically affect the reliability of data. Most problems may be due to disclosure of identity of researcher and respondent. Researchers have to not only assure privacy but also do everything possible to maintain it. Here is a check list of such problems to overcome in executing a questionnaire survey. A. Respondent related issues 1. 2. 3. 4. 5. 6. 7. Awareness, own motives and attitudes and inadmissibility or desire to cover up guilty feeling of falling short of norms Irrationality (since society places high premium on sensible rational & logical behaviour) Self incrimination (protecting self esteem) Politeness - Not preferring to say negative, unpleasant or critical things Ignorance & misunderstanding Non response, disinterestedness & carelessness Biases, and prejudices; Due to the self-report nature of surveys, the results are susceptible to the problem of reactivity and could potentially elicit a response bias from respondents. One way to ameliorate this problem is to include "filler" questions 8. Hallo effect: Instead of giving attention separately to each question / item, respondent lets himself be influenced by an overall feeling of like or dislike a particular issue (i.e., respondent stepping into the shoes of researcher) 9. Response sets - tendency to reply to attitude scale items in a particular way almost independent of contents (For example, median response, reactions to red & green); Causes are (I) Social desirability (ii) Acquiescence (tendency towards assent rather than dissent) (iii) Rigidity, dogmatism & authoritarianism B. Tool related issues (ways of asking) 1. 2. 3. Generally, the language, style & wording should be simple, neutral, exact and shared vocabulary with uniform meaning and no embarrassing too personal questions No unspoken, unwarranted assumptions / presumptions about words or deeds; For example, How often do you use library? presumes that respondent uses library No hypothetical and ambiguous question (clear, specific and sensible questions only). For example, (a) If charges are introduced for library services your use would increase / decrease/ no change (b) what percent of useful information do you get from important secondary journals in your field?

4.

Questions should not lead to median replies. For example, How much time do you spend in reading? (replies depend on the options in terms of range and the median will attract more responses)

5. 6.

No leading questions (suggesting answer and indicating why the question is asked). For example, Do you think people like e-books? No loaded words / phrase in questions (emotionally colored with suggestion of approval or disapproval). For example, Are you one of those few elite scientists who use (international) online databases?

7. 8.

No in built coercion. The above cited example of asking the question after a free lunch, do you find the course useful? And if no, why? is nothing but coercion. No double barreled question (which affects precision of replies). For example, Charges for library services should be introduced but it should be for additional services only as an option for question on introducing charges for library services.

9. No double negatives (which lead to incorrect answers). For example, Charge for library services should not be introduced for basic services - agree / disagree / neither.

M S Sridhar sridhar@informindia.co.in --------------------------------------------------------------------------------------------------------------------http://informindia.co.in/iil_newsletter_researhc.asp