UNIT III DATA COLLECTION Business Research Methods (BA9227) Types of data Primary Vs Secondary data Methods of primary

ry data collection Survey Vs Observation Experiments Construction of questionnaire and instrument Validation of questionnaire Sampling plan Sample size determinants optimal sample size sampling techniques Probability Vs Non probability sampling methods.

Primary Data & Methods of primary data collection:-

PRIMARY data are data that you collect yourself using such methods as:

direct observation - lets you focus on details of importance to you; lets you see a system in real rather than theoretical use (othen faults are unlikely or trivial in theory but quite real and annoying in practice);

surveys - written surveys let you collect considerable quantities of detailed data. You have to either trust the honesty of the people surveyed or build in self-verifying questions (e.g. questions 9 and 24 ask basically the same thing but using different words - different answers may indicate the surveyed person is being inconsistent, dishonest or inattentive).

interviews - slow, expensive, and they take people away from their regular jobs, but they allow indepth questioning and follow-up questions. They also show non-verbal communication such as face-pulling, fidgeting, shrugging, hand gestures, sarcastic expressions that add further meaning to spoken words. e.g. "I think it's a GREAT system" could mean vastly different things depending on whether the person was sneering at the time! A problem with interviews is that people might say what they think the interviewer wants to hear; they might avoid being honestly critical in case their jobs or reputation might suffer.

logs (e.g. fault logs, error logs, complaint logs, transaction logs). Good, empirical, objective data sources (usually, if they are used well). Can yield lots of valuable data about system performance over time under different conditions.

Primary data can be relied on because you know where it came from and what was done to it. It's like cooking something yourself. You know what went into it. SECONDARY SECONDARY data are collected from external sources such as:

TV, radio, internet magazines, newspapers reviews research articles stories told by people you know

There's a lot more secondary data than primary data, and secondary data are a whole lot cheaper and easier to acquire than primary data. The problem is that often the reliability, accuracy and integrity of the data are uncertain. Who collected it? Can they be trusted? Did they do any preprocessing of the data? Is it biased? How old is it? Where was it collected? Can the data be verified, or does it have to be taken on faith? Often secondary data have been pre-processed to give totals or averages and the original details are lost so you can't verify it by replicating the methods used by the original data collectors. In short, primary data are expensive and difficult to acquire, but they are trustworthy. Secondary data are cheap and easy to collect, but must be treated with caution. Primary research entails the use of immediate data in determining the survival of the market. The popular ways to collect primary data consist of surveys, interviews and focus groups, which shows that direct relationship between potential customers and the companies. Whereas secondary research is a means to

reprocess and reuse collected information as an indication for betterments of the service or product. Both primary and secondary data are useful for businesses but both may differ from each other in various aspects. In secondary data, information relates to a past period. Hence, it lacks aptness and therefore, it has unsatisfactory value. Primary data is more accommodating as it shows latest information. Secondary data is obtained from some other organization than the one instantaneously interested with current research project. Secondary data was collected and analyzed by the organization to convene the requirements of various research objectives. Primary data is accumulated by the researcher particularly to meet up the research objective of the subsisting project. Secondary data though old may be the only possible source of the desired data on the subjects, which cannot have primary data at all. For example, survey reports or secret records already collected by a business group can offer information that cannot be obtained from original sources. Firm in which secondary data are accumulated and delivered may not accommodate the exact needs and particular requirements of the current research study. Many a time, alteration or modifications to the exact needs of the investigator may not be sufficient. To that amount usefulness of secondary data will be lost. Primary data is completely tailor-made and there is no problem of adjustments. Secondary data is available effortlessly, rapidly and inexpensively. Primary data takes a lot of time and the unit cost of such data is relatively high.

Survey Vs Observation Participant observation studies and surveys are easily differentiated, although they have some common characteristics. Social researchers, like marketing professionals or anthropologists, use both systems as qualitative research techniques. According to Minnesota State University, the method of participant observation is used to describe thoroughly people's behavior in their every day lives. Unlike surveys, participation observation methods require researchers to immerse themselves in a culture. Anthropological research methods, like participant observation, may require months of fieldwork, while surveys can be completed quickly and through different media, such as phones, mailers, online and inperson

Experimental Research The word experimental research has a range of definitions. In the strict sense, experimental research is what we call a true Experiment . This is an experiment where the researcher manipulates one variable, and control / randomizes the rest of the variables. It has a Control Group, the subjects have been randomly assigned between the groups, and the researcher only tests one effect at a time. It is also important to know what variable(s) you want to test and measure.

Construction of Questionnaire and Instrument Schedules and questionnaires are the most common instruments of data collection. These two types of tools have much in common. Both of them contain a set of questions logically related to a problem under study; both aim at eliciting responses from the respondents; in both cases the content, response structure, the wordings of questions, question sequence, etc. are the same for all respondents. Then why should they be denoted by the different terms: 'schedule' and 'questionnaires'? This is because the methods for which they are used are different. While a schedule is used as a tool for interviewing, a questionnaire is used for mailing.This difference in usage gives rise to a subtle difference between these two recording forms. That is, the interviewer in a face-to-face interviewing fills a schedule, whereas the respondent himself fills in a questionnaire. Hence the need for using two different terms. The tool is referred to as a schedule when it is used for interviewing; and it is called a questionnaire when it is sent to a respondent for completion and return.

A questionnaire is a self-report data collection instrument that is filled out by research participants. Questionnaires are usually paper-and-pencil instruments, but they are increasingly being placed on the web for participants to go to and fill out. Questionnaires are also know as survey instruments, but the actual questionnaire should not be called the survey. The word survey refers to the process of using

a questionnaire or interview protocol to collect data; it does not refer to the survey instrument (questionnaire). You might conduct a survey of teacher attitudes about inclusion; the instrument of data collection would be called the questionnaire or the survey instrument. Survey research has a long history and this is how survey researchers use these terms.

Questionnaires are used to measure individuals thinking about behavior, experiences, attitudes, opinions, beliefs, values, knowledge, and background or demographic information. Each of these can be asked about with reference to the past, present, or future. In short, a lot of information can be obtained via questionnaires. We believe that a useful way to learn about constructing questionnaires is to understand the principles of questionnaire construction and to examine examples of model questionnaires (i.e., questionnaires that follow the principles explained in this chapter). Principles of Questionnaire Construction Principle 1: Make sure the questionnaire items match your research objectives. Principle 2: Understand your research participants. Your participants (not you!) will be filling out the questionnaire. Consider the demographic and cultural characteristics of your potential participants so that you can make it understandable to them. Principle 3: Use natural and familiar language. Familiar language is comforting; jargon is not.

Principle 4: Write items that are clear, precise, and relatively short. If your participants don't understand the items, your data will be invalid (i.e., your research study will have the garbage in, garbage out, GIGO, syndrome). Short items are more easily understood and less stressful than long items.

Principle 5: Do not use "leading" or "loaded" questions. Leading questions lead the participant to where you want him or her to be. Loaded questions include loaded words (i.e., words that create an emotional reaction or response by your participants). Always remember that you do not want the participant's response to be the result of how you worded the question. Always use neutral wording.

Principle 6: Avoid double-barreled questions. A double-barreled question combines two or more issues in a single question (e.g., here is a double barreled question: Do you elicit information from parents and other teachers? Its double barreled because if someone answered it, you would not know whether they were referring to parents or teachers or both). Does the question include the word "and"? If yes, it might be a double-barreled question. Answers to double-barreled questions are ambiguous because two or more ideas are confounded. Principle 7: Avoid double negatives. Does the answer provided by the participant require combining two negatives? (e.g., "I disagree that teachers should not be required to supervise their students during library time"). If yes, rewrite it.

Principle 8: Determine whether an open-ended or a closed ended question is needed. Open-ended questions provide qualitative data in the participants' own words. Here is an open ended question: How can your principal improve the morale at your school? On the questionnaire, make sure you provide plenty of blank space for responses; do not provide blank lines for responses.

Closed-ended questions provide quantitative data based on the researcher's response categories. Here is an example of a closed-ended question:

Open-ended questions are common in exploratory research and closed-ended questions are common in confirmatory research.

Principle 9: Use mutually exclusive and exhaustive response categories for closed-ended questions. Mutually exclusive categories do not overlap (e.g., ages 0-10, 10-20, 20-30 are NOT mutually exclusive and should be rewritten as less than 10, 10-19, 20-29, 30-39, ...). Exhaustive categories include all possible responses (e.g., if you are doing a national survey of adult citizens (i.e., 18 or older) then the these categories (18-19, 20-29, 30-39, 40-49, 50-59, 60-69) are NOT exhaustive because there is no where to put someone who is 70 years old or older.

Principle 10: Consider the different types of response categories available for closed-ended questionnaire items. Rating scales are the most commonly used, including:

Numerical rating scales (where the endpoints are anchored; sometimes the center point or area is also labeled).

1 Very Low

7 Very High

Fully anchored rating scales (where all the points on the scale are anchored).

1 Strongly Disagree

2 Disagree Neutral

3 Agree

4 Strongly Agree

1 Strongly Disagree

2 Disagree

3 Agree

4 Strongly Agree

Omitting the center point on a rating scale (e.g., using a 4-point rather than a 5-point rating scale) does not appreciably affect the response pattern. Some researchers prefer 5- point rating scales; other researchers prefer 4-point rating scales. Both generally work well.

You should use somewhere from four to eleven points on your rating scale. Personally, I like the 4 and 5-point scales because all of the points are easily anchored.

I do not recommend a 1 to 10 scale because too many respondents mistakenly view the 5 as the center point. If you want to use a wide scale like this, use a 0 to 10 scale (where the 5 is the middle point) and label the 5 with the anchor medium or some other appropriate anchor.

Rankings (i.e., where participants put their responses into rank order, such as most important, second most important, and third most important).

Semantic differential (i.e., where one item stem and multiple scales, that are anchored with polar opposites or antonyms, are included and are rated by the participants).

Checklists (i.e., where participants "check all of the responses in a list that apply to them"). Checklists or multiple response items should be avoided except in cases where description is all that is needed; only use when you dont need to examine the relationship between the multiple response item and other items in your questionnaire.

Principle 11: Use multiple items to measure abstract constructs. This is required if you want your measures to have high reliability and validity. One approach is to use a summated rating scale (such as the Rosenberg Self-Esteem Scale that is composed of 10 items, with each item measuring self-esteem). Another name for a summated rating scale is a Likert Scale because the summated rating scale was pretty much invented by the famous social psychologist named Rensis Likert. See the Rosenberg Self-Esteem Scale (Figure 7.1 in your book), which is a summated rating scale.

Principle 12: Consider using multiple methods when measuring abstract constructs. The idea here is that if you only use one method of measurement, then your measurement may be an artifact of that method of measurement.

On the other hand, if you use two or more methods of measurement you will be able to see whether the answers depend on the method (i.e., are the answers corroborated across the methods of measurement or do you get different answers for the different methods?). For example, you might measure students self-esteem via the Rosenberg Scale just shown (which is used in a self-report form) as well as using teachers ratings of the students self-esteem; you might even want to observe the students in situations that should provide indications of high and low self-esteem. Also, use open and closed ended items on questionnaire.

Principle 13: Use extreme caution if you reverse the wording in some of the items to prevent response sets in multi-item scales. (A response set is the tendency of a participant to respond in a specific direction to items regardless of the item content.) Reversing the wording of some items can help ensure that participants don't just "speed through" the instrument, checking "yes" or "strongly agree" for all the items. On the other hand, you may want to avoid reverse wording if it creates a double negative. Also, recent research suggests that the use of reverse wording reduces the reliability and validity of scales. Therefore, you should generally use reverse wording sparingly, if at all. Principle 14: Develop a questionnaire that is properly organized and easy for the participant to use. The participant must not get confused or lost anywhere in the questionnaire. Make sure that the directions are clear and that any contingency questions used are easy to follow. Principle 15: Always pilot test your questionnaire. You will always find some problems that you have overlooked! The best pilot tests are with people similar to the ones to be included in your research study. After pilot testing your questionnaire, revise it and pilot test it again, until it works correctly.

Now, lets put it all together.

10

First, examine the mixed questionnaire shown in Exhibit 7.3 (see your textbook). You can use that as a model questionnaire to get you started in writing your own questionnaire. Second, use the checklist for questionnaire construction (Table 7.3 in your textbook) as you develop your own questionnaire or as you critique other questionnaires to make sure good practice has been followed. Third, follow the steps in the questionnaire construction flowchart provided in Figure 7.2 in your textbook Questionnaire Development and Validation There are two basic goals (6) in questionnaire design. The first is to obtain information relevant to the purposes of the survey and the second is to collect this information with maximal reliability and validity. The question before the researcher is that the questionnaire he is adopting will measure what it is supposed to measure and will do this in a consistent manner? Now, before we start discussing the answer to this question let us go through the definitions for and methods of establishing the validity and reliability of a questionnaire. Validity Validity is the degree to which an assessment measures what it is supposed to measure. Known Groups Validity Known groups validity is a form of construct validation in which the validity is determined by the degree to which an instrument can demonstrate different scores for groups know to vary on the variables being measured. Longitudinal Validity Longitudinal validity is the extent to which changes on one measure will correlate with changes on another measure Concurrent Validity To validate a new measure, the results of the measure are compared to the results of the gold standard obtained at approximately the same point in time (concurrently), so they both reflect the same construct. This approach is useful in situations when a new or untested tool is potentially more efficient, easier to administer, more practical, or safer than another more established method and is being proposed as an alternative instrument.

11

Construct Validity Reflects the ability of an instrument to measure an abstract concept, or construct. For some attributes, no gold standard exists. In the absence of a gold standard, construct validation occurs, where theories about the attribute of interest are formed, and then the extent to which the measure under investigation provides results that are consistent with these theories are assessed. Content Validity Refers to the extent to which a measure represents all aspects of a given social concept. Example: A depression scale may lack content validity if it only assesses the affective dimension of depression but fails to take into account the behavioral dimension. Criterion Validity Examines the extent to which a measure provides results that are consistent with a gold standard. It is typically divided into concurrent validity and predictive validity Discriminant Validity Measures that should not be related are not. Discriminant validity examines the extent to which a measure correlates with measures of attributes that are different from the attribute the measure is intended to assess. Face Validity A form of content validity, face validity is assessed by having 'experts' (this could be clinicians, clients, or researchers) review the contents of the test to see if the items seem appropriate. Because this method has inherent subjectivity, it is typically only used during the initial phases of test construction. Reliability Reliability can be defined in a variety of ways. It is generally understood to be the extent to which a measure is stable or consistent and produces similar results when administered repeatedly. A more technical definition of reliability is that it is the proportion of "true" variation in scores derived from a particular measure. The total variation in any given score may be thought of as consisting of true variation (the variation of interest) and error variation (which includes random error as well as systematic

12

error). True variation is that variation which actually reflects differences in the construct under study, e.g., the actual severity of neurological impairment. Random error refers to "noise" in the scores due to chance factors, e.g., a loud noise distracts a patient thus affecting his performance, which, in turn, affects the score. Systematic error refers to bias that influences scores in a specific direction in a fairly consistent way, e.g., one neurologist in a group tends to rate all patients as being more disabled than do other neurologists in the group. There are many variations on the measurement of reliability including alternate-forms, internal, inter-rater agreement, intra-rater agreement, and test-retest Test-retest Reliability A way of estimating the reliability of a scale in which individuals are administered the same scale on two different occasions and then the two scores are assessed for consistency. This method of evaluating reliability is appropriate only if the phenomenon that the scale measures is known to be stable over the interval between assessments. If the phenomenon being measured fluctuates substantially over time, then the test-retest paradigm may significantly underestimate reliability. In using test-retest reliability, the investigator needs to take into account the possibility of practice effects, which can artificially inflate the estimate of reliability. Gold Standard A method of measuring reliability. Inter-rater reliability determines the extent to which two or more Questionnaire Validation Made Easy This is a type of reliability assessment in which the same assessment is completed by the same later on two or more occasions. These different ratings are then compared, generally by means of correlation. Since the same individual is completing both assessments, the rater's subsequent ratings are contaminated by knowledge of earlier ratings. Sensitivity Sensitivity refers to the probability that a diagnostic technique will detect a particular disease or condition when it does indeed exist. Specificity Specificity refers to the probability that a diagnostic technique will indicate a negative test result when the condition is absent (true negative). Standardized Response Mean

13

The standardized response mean (SRM) is calculated by dividing the mean change by the standard deviation of the change scores. Floor Effect The floor effect is when data cannot take on a value lower than some particular number. Thus, it represents a sub sample for which clinical decline may not register as a change in score, even if there is worsening of function/behavior etc. because there are no items or scaling within the test that measure decline from the lowest possible score. Intra-class Correlation Coefficient (ICC) Intra-class correlation (ICC) is used to measure inter-rater reliability for two or more raters. It may also be used to assess test-retest reliability. ICC may be conceptualized as the ratio of between-groups variance to total variance. Cronbach's Alpha Cronbach's alpha is a coefficient (a number between 0 and 1) that is used to rate the internal consistency (homogeneity) or the correlation of the items in a test. A good test is one that assesses different aspects of the trait being studied. If a test has a strong internal consistency most measurement experts agree that it should show only moderate correlation among items (.70 to 0.90). If correlations between items are too low, it is likely that they are measuring different traits (e.g. both depression and quality of life items are mixed together) and therefore should not all be included in a test that is supposed to measure one trait. If item correlations are too high, it is likely that some items are redundant and should be removed from the test. Known groups method is a typical method to support construct validity and is provided when a test can discriminate between a group of individuals known to have a particular trait and a group who do not have the trait. Similarly, known groups may be studied using groups of individuals with differing levels/severities of a trait. Again the known groups methods will evaluate the test's ability to discriminate between the groups based on the groups demonstrating different mean scores on the test. For example,

14

a group of individuals known to be not depressed should have lower scores on a depression scale then the group known to be depressed . Difference between test/retest and Internal Consistency The primary difference between test/retest and internal consistency estimates of reliability is that test/retest involves two administrations of the measurement instrument, whereas the internal consistency method involves only one administration of that instrument. Bias Bias often occurs due to distortions in procedures and characteristics of instruments, observers, and investigators. While bias can occur due to intentional acts on the part of researchers, of particular concern here is bias that is unintentional, arising from instruments. Bias is far more problematic than random error, most often investigators attempt to uncover its sources and design instruments or methods to avoid it and in fact much effort in designing studies involves the avoidance of bias. Validated Questionnaire A validated questionnaire is one which has undergone a validation procedure to show that it accurately measures what it aims to do, regardless of who responds, when they respond, and to whom they respond or when self administered.The instrument is compared against the available Gold standard. It is compared with other sources of data. The reliability is also examined.Even if a questionnaire is not completely valid (which it hardly ever will be) the reliability of the instrument has a value of its own. If an instrument is reliable this offers the opportunity to compare results from one study with work that has been done by others. This is of special value if measurements are taken at different points in time or in the different regions.(raters obtain the same result when using the same instrument to measure a concept.

Advantages of Validating a Questionnaire

15

This reduces bias by detecting ambiguities and misinterpretations which can then be minimized. The ambiguities when minimized then the instrument aims at high degree of specific objectivity. The instrument is compared with a gold standard questionnaire, as well as the instrument is compared with the other sources of data. The other benefits are that the feasibility, acceptability, time needed to respond, cost etc. are pre-examined. This also Examine variation in response due to data inquiry methods (self administered, personal interview, telephone interview etc .Hence, better quality data will be collected, comparability will be high, efforts will be reduced and the credibility, quality and usefulness of information thus derived will be far superior. We suggest that authors of questionnaire based research papers shall feel proud of making the questionnaire available on-line rather than taking it as a burden and in turn shall receive incentives, like priority in publications, from the editors. Authors of the medical papers shall put the questionnaires on their web pages or better on-line with free asses to the researchers. Submitted papers that were accompanied by a web accessible questionnaire might be granted higher priority for publication. All else being equal, one could argue that a report with an easily accessible questionnaire has greater value to readers. Probability and nonprobability sampling A probability sampling scheme is one in which every unit in the population has a chance (greater than zero) of being selected in the sample, and this probability can be accurately determined. The combination of these traits makes it possible to produce unbiased estimates of population totals, by weighting sampled units according to their probability of selection. Example: We want to estimate the total income of adults living in a given street. We visit each household in that street, identify all adults living there, and randomly select one adult from each household. (For example, we can allocate each person a random number, generated from a uniform distribution between 0 and 1, and select the person with the highest number in each household). We then interview the selected person and find their income. People living on their own are certain to be selected, so we simply add their income to our estimate of the total. But a person living in a household of two adults has only a one-in-two chance of selection. To reflect this, when we come to such a household, we would count the selected person's income twice towards the total. (The person who is selected from that household can be loosely viewed as also representing the person who isn't selected.) In the above example, not everybody has the same probability of selection; what makes it a probability sample is the fact that each person's probability is known. When every element in the population doeshave the same probability of selection, this is known as an 'equal probability of selection'

16

(EPS) design. Such designs are also referred to as 'self-weighting' because all sampled units are given the same weight. Probability sampling includes: Simple Random Sampling, Systematic Sampling, Stratified Sampling, Probability Proportional to Size Sampling, and Cluster or Multistage Sampling. These various ways of probability sampling have two things in common: 1. Every element has a known nonzero probability of being sampled and 2. involves random selection at some point. Nonprobability sampling is any sampling method where some elements of the population have no chance of selection (these are sometimes referred to as 'out of coverage'/'undercovered'), or where the probability of selection can't be accurately determined. It involves the selection of elements based on assumptions regarding the population of interest, which forms the criteria for selection. Hence, because the selection of elements is nonrandom, nonprobability sampling does not allow the estimation of sampling errors. These conditions give rise to exclusion bias, placing limits on how much information a sample can provide about the population. Information about the relationship between sample and population is limited, making it difficult to extrapolate from the sample to the population. Example: We visit every household in a given street, and interview the first person to answer the door. In any household with more than one occupant, this is a nonprobability sample, because some people are more likely to answer the door (e.g. an unemployed person who spends most of their time at home is more likely to answer than an employed housemate who might be at work when the interviewer calls) and it's not practical to calculate these probabilities. Nonprobability sampling methods include accidental sampling, quota sampling and purposive sampling. In addition, nonresponse effects may turn any probability design into a nonprobability design if the characteristics of nonresponse are not well understood, since nonresponse effectively modifies each element's probability of being sampled. ************************************************* Prof.Dr.S.Sakthivel, Professor , Hallmark Business School, Trichy.

17