INTRODUCTION TO RESEARCH DESIGN

Research design provides the glue that holds the research project together. A design is used to structure the research, to show how all of the major parts of the research project -- the samples or groups, measures, treatments or programs, and methods of assignment -- work together to try to address the central research questions.

What is Research Design?

Research design can be thought of as the structure of research -- it is the "glue" that holds all of the elements in a research project together. We often describe a design using a concise notation that enables us to summarize a complex design structure efficiently. What are the "elements" that a design includes? They are:

Observations or Measures

These are symbolized by an 'O' in design notation. An O can refer to a single measure (e.g., a measure of body weight), a single instrument with multiple items (e.g., a 10-item self-esteem scale), a complex multi-part instrument (e.g., a survey), or a whole battery of tests or measures given out on one occasion. If you need to distinguish among specific measures, you can use subscripts with the O, as in O1, O2, and so on.

Treatments or Programs

These are symbolized with an 'X' in design notations. The X can refer to a simple intervention (e.g., a one-time surgical technique) or to a complex hodgepodge program (e.g., an employment training program). Usually, a no-treatment control or comparison group has no symbol for the treatment (some researchers use X+ and X- to indicate the treatment and
1

control respectively). As with observations, you can use subscripts to distinguish different programs or program variations.

Groups

Each group in a design is given its own line in the design structure. if the design notation has three lines, there are three groups in the design.

Assignment to Group

Assignment to group is designated by a letter at the beginning of each line (i.e., group) that describes how the group was assigned. The major types of assignment are:

R = random assignment N = nonequivalent groups C = assignment by cutoff

Time

Time moves from left to right. Elements that are listed on the left occur before elements that are listed on the right.

DESIGN NOTATION EXAMPLES

It's always easier to explain design notation through examples than it is to describe it in words. The figure shows the design notation for a pretestposttest (or before-after) treatment versus comparison group randomized experimental design. Let's go through each of the parts of the design. There are two lines in the notation, so you should realize that the study has two groups. There are four Os in the notation, two on each line and two for each group. When the Os are stacked vertically on top of each
2

other it means they are collected at the same time. In the notation you can see that we have two Os that are taken before (i.e., to the left of) any treatment is given -- the pretest -- and two Os taken after the treatment is given -- the posttest. The R at the beginning of each line signifies that the two groups are randomly

assigned (making it an experimental design). The design is a treatment

versus comparison group one because the top line (treatment group) has an X while the bottom line (control group) does not. You should be able to see why many of my students have called this type of notation the "tictac-toe" method of design notation -- there are lots of Xs and Os! Sometimes we have to be more specific in describing the Os or Xs than just using a single letter. In the second figure, we have the identical research design with some subscripting of the Os. What does this mean? Because all of the Os have a subscript of 1, there is some measure or set of measures that is collected for both groups on both occasions. But the design also has two Os with a subscript of 2, both taken at the posttest. This means that there was some measure or set of measures that were collected only at the posttest.

With this simple set of rules for describing a research design in notational form, you can concisely explain even complex design structures. And, using a notation helps to show common design sub-structures across different designs that we might not recognize as easily without the notation. The next step after stating the management problem, research purpose, and research hypotheses and questions, is to formulate a research design. The starting point for the research design is, in fact, the research questions and hypotheses that have been so carefully developed. In essence, the research design answers the question: How are we going to get answers to these research questions and test these hypotheses? The research design is a plan of action indicating the specific steps that are necessary to provide answers to those questions, test the hypotheses, and thereby achieve the research purpose that helps choose among the decision alternatives to solve the management problem or capitalize on the market opportunity

DEFINITIONS OF RESEARCH DESIGN:

(1) According to David J. Luck and Ronald S. Rubin, "A research design is the determination and statement of the general research approach or strategy adopted/or the particular project. It is the heart of planning. If the design adheres to the research objective, it will ensure that the client's needs will be served." (2) According to Kerlinger "Research design in the plan, structure and strategy of

investigation conceived so as to obtain answers to research questions and to control variance." (3) According to Green and Tull "A research design is the specification of methods and procedures for acquiring the information needed. It is the over-all operational pattern or framework of the project that stipulates what information is to be collected from which source by what procedures." The second definition includes three important terms - plan, structure and strategy. The plan is the outline of the research scheme on which the researcher is to work. The structure of the research work is a more specific scheme and the strategy suggests how the research will be carried out i.e. methods to be used for the collection and analysis of data. In brief, research design is the blueprint of research. It is the specification of methods and procedures for acquiring the information needed for solving the problem. Questionnaires, forms and samples for investigation are decided while framing research design. Finally, the research design enables the researcher to arrive at certain meaningful conclusions at the end of proposed study.

GENERAL DISTINCTION IN RESEARCH DESIGN

Research design can be divided into fixed and flexible research designs (Robson, 1993). Others have referred to this distinction with quantitative research designs and qualitative research designs. However, fixed designs need not be quantitative, and flexible design need not be qualitative. In fixed designs the design of the study is fixed before the main stage of data collection takes place. Fixed designs are normally theory-driven; otherwise its impossible to know in advance which
6

variables need to be controlled and measured. Often these variables are quantitative. Flexible designs allow for more freedom during the data collection. One reason for using a flexible research design can be that the variable of interest is not quantitatively measurable, such as culture. In other cases, theory might not be available before one starts the research. EXAMPLES OF FIXED (QUANTITATIVE) DESIGNS Experimental design In an experimental design, the researcher actively tries to change the situation, circumstances or experience of participants (manipulation), which leads to a change in behaviour of the participants of the study. The participants are assigned to different conditions, and variables of interest are measured. All other variables are controlled Experiments are normally highly fixed before the data collection starts. Non-experimental research designs Non-experimental research is almost the same as experimental research, the only difference is that non-experimental research does not involve a manipulation of the situation, circumstances or experience of the participants. Non-experimental research designs can be split up in three designs. First, relational designs, in which a range of variables is measured. These designs are also called correlational studies, since the correlation is most often used analysis. The second type is comparative designs. These design compare two natural groups. The third type of nonexperimental research is a longitudinal design.

Quasi experiment Quasi research designs are research design that follow the experimental procedure, but do not randomly assign people to (treatment and comparison) groups. EXAMPLES OF FLEXIBLE (QUALITATIVE) RESEARCH DESIGNS Case study In a case study, one single unit is extensively studied. This case can be a person, organization, group or situation. Famous case studies are for example the descriptions about the patients of Freud, who were thoroughly analysed and described. Read more on case study. Bell (1999) states a case study approach is particularly appropriate for individual researchers because it gives an opportunity for one aspect of a problem to be studied in some depth within a limited time scale. Ethnographic study This type of research is involved with a group, organization, culture, or community. Normally the researcher shares a lot of time with the group. Grounded theory study The aim of grounded theory studies is to make theories that can explain certain events.

DESIGN VERSUS METHOD

Research design is different from the method by which data are collected. Many research methods texts confuse research designs with
8

methods. It is not uncommon to see research design treated as a mode of data collection rather than as a logical structure of the inquiry. But there is nothing intrinsic about any research design that requires a particular method of data collection. Although cross-sectional surveys are frequently equated with questionnaires and case studies are often equated with participant observation (e.g. Whyte's Street Corner Society, 1943), data for any design can be collected with any data collection method How the data are collected is irrelevant to the logic of the design. Failing to distinguish between design and method leads to poor evaluation of designs. Equating cross-sectional designs with

questionnaires, or case studies with participant observation, means that the designs are often evaluated against the strengths and weaknesses of the method rather than their ability to draw relatively unambiguous conclusions or to select between rival plausible hypotheses.

TYPES OF RESEARCH DESIGN

The research designs are broadly classified into 2 general types, namely EX POST FACTO RESEARCH RESIGN EXPERIMENTAL RESEARCH DESIGN

EX-POST-FACTO RESEARCH DESIGNS ("AFTER THE FACT")

Ex Post Facto (also called Causal Comparative Research) is useful whenever: We have two groups which differ on an independent variable and we want to test hypotheses about differences on one or more dependent variables OR We have two groups which already differ on a dependent variable and
9

we want to test hypotheses about differences on one or more independent variables Much less able to determine causality than true experiments but these are necessary and important research designs. Necessity for:1. ethical reasons or 2. an interest in organismic variables Two main types: Prospective and Retrospective designs: find naturally occurring groups (thus, "after the fact") and follow them forward (prospective) or trace their histories (retrospective) Problems: 1. subjects are not randomly assigned to treatments, as a result there will be inherent confounds in the populations studied (this is the most serious problem) 2. sampling problems (often a convenient sample): 3. dropouts in prospective studies 4. detection bias (equally likely to detect in both groups?) Partial solutions: Matching: 1) subject for subject (preferable but more difficult) or 2) distribution by distribution in both cases can selectively drop individuals and bias the sample further Measuring: so will: 1) know if potential confounds (uncontrolled or extraneous

10

variables) are confounded, & 2) to statistically control for these variables Retrospective studies have additional problems in that they rely on memory so the partial solutions are more difficult to employ successfully Have the advantage (over prosepective designs) in that they are more efficient (cheaper and faster) May be necessary with very rare grouping variables of interest (e.g., rare diseases) Note that even with measurement and matching, internal validity is still questionable. [The additional problems of retrospective designs are well illustrated by McFarland's (1988) study of cyclical variability in moods)]. DVs used in Ex-post-facto studies

relative risk ratio (prospective studies) - illustrated by breast cancer data

relative odds ratio (approximates the relative risk) -- retrospective studies

Problem with both in that absolute risks are hidden, both (absolute and relative risks) should be reported. Causality and ex-post-facto designs. Although no one (or few) quasi-analytic experiment will unambiguously show a causal relationship, with converging evidence from many such studies (5, 10 or 100?) can make causal statements (like "smoking causes cancer").

11

EXPERIMENTAL RESEARCH DESIGN

The aim of the experimental research is to investigate the possible cause-and-effect relationship by manipulating one independent variable to influence the other variable(s) in the experimental group, and by controlling the other relevant variables, and measuring the effects of the manipulation by some statistical means. By manipulating the independent variable, the researcher can see if the treatment makes a difference on the subjects. If the average scores of two groups prove to be significantly different, and if there are not any explanations for this difference, then it can be concluded that the effect of the treatment caused this difference. This is where experimental research differs from correlational research, For instance, correlational studies only describe or predict the strong relationship between socioeconomic level and the academic achievement but cannot prove the direct cause-and-effect relationship between these two variables. It is the experimental research which can demonstrate that by changing the independent variable, a change is possible on the dependent variable. In educational research the most frequently studied dependent variables are achievement, motivation, attention, interest in learning, participation and attitudes. The common independent variables that are manipulated are teaching methods, types of assignments, types of teaching materials such as text books and visual aids, types of rewards, types of questions used by the teacher, and evaluation techniques. There are however, some independent variables such as age and gender that cannot be manipulated. When the independent variable that is chosen cannot be manipulated, either a comparative research is conducted, or a second independent variable is chosen for manipulation in order to conduct an experimental study.
12

The following are some examples to illustrate the experimental research (Isaac & Michael, 1977): - To investigate the effects of two methods of teaching a twelfthgrade history program as a function of class size (large and small) and levels of student intelligence (high, average, low), using random assignment of teachers and students-byintelligence-level to method and class size. - To investigate the effects of a new drug abuse prevention program on the attitudes of junior high school students using experimental and control groups who are either exposed or not exposed to the program respectively, and using a pretestposttest design in which only half of the students randomly receive the pretest to determine how much of an attitude change can be attributed to pretesting or the educational program. - To investigate the effects of two methods of pupil evaluation on the performance of children in the twenty-three elementary schools of a given sub urban school. 'N' in this study would be the number of classrooms, rather than children, and the method would be assigned by stratified random techniques such that there would be a balanced distribution of the two methods to classrooms across grade levels and socio-economic locations of schools.

STEPS IN RESEARCH DESIGN PROCESS

The steps in the design process interact and often occur simultaneously. For example, the design of a measurement instrument is influenced by the type of analysis that will be conducted. However, the type of analysis is also influenced by the specific characteristics of the measurement instrument.
13

Step 1: Define the Research Problem Problem definition is the most critical part of the research process. Research problem definition involves specifying the information needed by management. Unless the problem is properly defined, the information produced by the research process is unlikely to have any value. CocaCola Company researchers utilized a very sound research design to collect information on taste preferences. Unfortunately for Coca-Cola, taste preferences are only part of what drives the soft drink purchase decision. Research problem definition involves four interrelated steps: (1) management problem / opportunity clarification, (2) situation analysis, (3) model development, and (4) specification of information requirements. The basis goal of problem clarification is to ensure that the decision makers initial description of the management decision is accurate and reflects the appropriate area of concern for research. If the wrong management problem is translated into a research problem, the probability of providing management with useful information is low. Situation Analysis The situation analysis focuses on the variables that have produced the stated management problem or opportunity. The factors that have led to the problem/opportunity manifestations and the factors that have led to managements concern should be isolated. A situation analysis of the retail trade outflow problem revealed, among other things, that (1) the local population had grown 25 percent over the previous five years, (2) buying power per capita appeared to be growing at the national rate of 3 percent a year, and (3) local retail sales of
14

nongrocery items had increased approximately 20 percent over the past five years. Thus, the local retailers sales are clearly not keeping pace with the potential in the area. Step 2: Estimate the Value of the Information A decision maker normally approaches a problem with some information. If the problem is, say, whether a new product should be introduced, enough information will normally have been accumulated through past experience with other decisions concerning the introduction of new products and from various other sources to allow some preliminary judgments to be formed about the desirability of introducing the product in question. There will rarely be sufficient confidence in these judgments that additional information relevant to the decision would not be accepted if it were available without cost or delay. There might be enough confidence, however, that there would be an unwillingness to pay very much or wait very long for the added information. Step 3: Select the Data Collection Approach There are three basic data collection approaches in marketing research: (1) secondary data, (2) survey data, and (3) experimental data. Secondary data were collected for some purpose other than helping to solve the current problem, whereas primary data are collected expressly to help solve the problem at hand. Step 4: Select the Measurement Technique There are four basic measurement techniques used in marketing research: (1) questionnaires, (2) attitude scales, (3) observation, and (4) depth interviews and projective techniques.

15

Primary Measurement Techniques I. Questionnaire a formalized instrument for asking information directly from a respondent concerning behavior, demographic characteristics, level of knowledge, and/or attitudes, beliefs, and feelings. II. Attitude Scales a formalized instrument for eliciting self-reports of beliefs and feelings concerning an object(s). A. Rating Scales require the respondent to place the object being rated at some point along a numerically valued continuum or in one of a numerically ordered series of categories. B. Composite Scales require the respondents to express a degree of belief concerning various attributes of the object such that the attitude can be inferred from the pattern of responses. C. Perceptual maps derive the components or characteristics an individual uses in comparing similar objects and provide a score for each object on each characteristic. D. Conjoint analysis derive the value an individual assigns to various attributes of a product. I. Observation the direct examination of behavior, the results of behavior, or physiological changes. II. Projective Techniques and Depth Interview designed to gather information that respondents are either unable or unwilling to provide in response to direct questioning. A. Projective Techniques allow respondents to project or express their own feelings as a characteristic of someone or something else.

16

B. Depth Interviews allow individuals to express themselves without any fear of disapproval, dispute, or advice from the interviewer. Step 5: Select the Sample Most marketing studies involve a sample or subgroup of the total population relevant to the problem, rather than a census of the entire group. Step 6: Select the Model of Analysis It is imperative that the researcher select the analytic techniques prior to collecting the data. Once the analytic techniques are selected, the researcher should generate fictional responses (dummy data) to the measurement instrument. These dummy data are then analyzed by the analytic techniques selected to ensure that the results of this analysis will provide the information required by the problem at hand. Step 7: Evaluate the Ethics of the Research It is essential that marketing researchers restrict their research activities to practices that are ethically sound. Ethically sound research considers the interests of the general public, the respondents, the client and the research profession as well as those of the researcher. Step 8: Estimate Time and Financial Requirements The program evaluation review technique (PERT) coupled with the critical path method (CPM) offers a useful aid for estimating the resources needed for a project and clarifying the planning and control process. PERT involves dividing the total research project into its smallest component activities, determining the sequence in which these activities must be performed, and attaching a time estimate for each activity. These activities and time estimates are presented in the form of a flow chart that allow a visual inspection of the overall process.
17

The time estimates allow one to determine the critical path through the chart that series of activities whose delay will hold up the completion of the project. Step 9: Prepare the Research Proposal The research design process provides the researcher with a blueprint, or guide, for conducting and controlling the research project. The blueprint is written in the form of a research proposal. A written research proposal should precede any research project.

18