[MET900] Research Methodology and Design of Experiments

Introduction to
Research Methodology

Deepak Verma PhD, MBA (IT/Marketing), ADSE (DBMS)

MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY
JLN MARG, JAIPUR – 302017 (RAJASTHAN, INDIA)
(An Institute of National Importance fully funded by Ministry of HRD, Government of India)
www.mnit.ac.in
Research is
systematic and objective
process of gathering,
recording and analyzing data
that provides information
to guide decisions
Scientific method of enquiry: What it is not
Non-scientific methods
▪ Intuitions
▪ Beliefs
▪ Consensus
▪ Authority
▪ Casual observation
▪ Informal Logic
Scientific method of enquiry: What it is
Scientific methods
▪ Systematic observation
▪ Formal Logic
▪ Consistently applied

▪ Provide chances of valid explanations

▪ Evaluate plausibility of hypotheses
Principles of Scientific Methods
Six principles
1. Empirically testable hypotheses (observe)

2. Replicability (repeat)

3. Objectivity (independent)

4. Transparent (replicable by anyone; publicly shared)

5. Falsifiability (contradiction possible)

6. Logically consistent (internally consistent; logically coherent)

Scientific claims
▪ Observation – a representation of world around us

▪ Hypothesis – describe pattern/ relationship in observations

▪ Laws – very precise & substantiated hypothesis

▪ Theory – overarching explanation of many related phenomenon

In science there’s no certainty…

…there are only best provisional explanations

Research

▪ Research is “systematic inquiry”

▪ systematic inquiry
▪ increases the efficiency of learning
▪ faster, cheaper
▪ increases the accuracy of learning
▪ zeroes in on the real “causes” of a phenomenon
▪ reduces chances of superstitious learning
Basic Research and Applied Research

Two major forms of research:

▪ Basic research:
a focused, systematic study to discover new knowledge
or expand existing one

▪ Applied research:
used to answer a specific determine why something
failed or succeeded
Stages In The Research Process

1. Defining the 2. Planning the 3. Planning a 4. Gathering the

Problem Research Design Sample Data

5. Processing & Analyzing

the Data

7. Defining the 6. Formulating Conclusions

New Problem & Preparing the Report
Defining Research Problem

Ascertain Isolate
Understand Determine
objectives and Determine
Background the
for identify the unit
of the Relevant
Decision the of Analysis
problem Variables
Making problem

State the Research Question

And Research Objectives
Importance of Problem Statement

feedback

Specific
Objectives 1

Statement Broad Research

of problem objectives Specific Research Results
Objectives 2 Design

Specific
Exploratory Objectives n
Research
Research Methodology: An Introduction

▪ Most often used to represent drawing up a research plan

▪ Limited to
▪ writing a questionnaire,
▪ collecting a limited set of data and, then,
▪ learning to apply some rudimentary statistics

This interpretation is, obviously,

naïve and incorrect
Research methodology is
based on a set of premises,
(theoretical) considerations and
practical conditions, according to
which the researcher structures the
logic of his research given the
question he wants to answer
The field of research

The real world

RESEARCH

Researcher’s interpretation The scientific world

The Research Pyramid

Research how the researcher views

Paradigm ‘reality’

‘way’ to conduct the research that is

Research Methodology tailored to the research paradigm

Research Methods specific steps of action

‘instruments’ or ‘tools’
for generating, collecting
Research Techniques and analyzing data
The basic approach

▪ Knowing Through the Eyes of the Researcher

knowing based on individual experience and test results

▪ Knowing Through the Eyes of Someone Else

implies knowing by hypothesizing and discovering
‘Box-of-bricks’ of Research

Problem Problem
owners solution

Observed reality STEPS

Empirical reality IN RESEARCH

Research Research
question answer
The Research Onion

Research
Methodological Choice Approach

Research Strategy Research

Philosophy
Time Horizon
Techniques
&
Procedures
The Research Philosophy

Positivism
▪ believes in the possibility to observe and describe reality
from an objective viewpoint

▪ adheres to the view that only “factual” knowledge gained

through observation (the senses), including measurement,
is trustworthy
The Research Philosophy

Interpretivism
▪ believes that it is necessary to understand differences
between humans in our roles as social actors

▪ assume that access to reality (given or socially

constructed) is only through social constructions such as
language, consciousness, shared meanings, and
instruments
The Research Philosophy

Realism
▪ relies on the idea of independence of reality from the
human mind

▪ based on the belief that Reality is “real” but only

imperfectly and probabilistically apprehensible and so
triangulation from many sources is required to try to
know it
Triangulation: Meaning

▪ Data Triangulation
the use of a variety of data sources

▪ Investigator Triangulation
the use of several different researchers

▪ Theory Triangulation
the use of multiple perspectives to interpret the results

▪ Methodological Triangulation
the use of multiple methods to study a research problem
The Research Philosophy

Pragmatism
▪ recognize “multiple interpretations” of the world and undertaking
research, that no single point of view can ever give the entire
picture and that there may be multiple realities

▪ involves using the method which appears best suited to the

research problem and not getting caught up in philosophical
debates about which is the best approach
Research Philosophies

Pragmatism Positivism Realism Interpretivism

Mixed or multiple Highly structured Methods chosen Small samples,
method designs must fit the subject
Large samples matter In-depth
Quantitative and investigations,
Qualitative Quantitative Quantitative or
measurement, but Qualitative Qualitative
can use Qualitative
Research Approach

Deduction
formation of a conclusion based on generally accepted
statements or facts

General rule 🡪 Specific conclusions

(always true)
Deductive Approach

Theory

Hypothesis

Observation

Confirmation
Research Approach

Induction
a syllogism in which the major premise is evident but the
minor premise and therefore the conclusion only probable

Specific Observations 🡪 General conclusions

(maybe true)
Inductive Approach

Observations

Patterns

Hypothesis

Theory
Research Approach

Abduction
inference of a generalized conclusion from particular
instances

Incomplete observations 🡪 Best Prediction

(may be true)
Methodological Choices

Qualitative Methods

collects and works with non-numerical data and

that seeks to interpret meaning from these data
Methodological Choices

Quantitative Methods

collects and works with numerical data and that

seeks to interpret meaning from these data
Methodological Choices

Mixed Method

employs both Qualitative as well as Quantitative

methods to collect data
Research Choices

Research Choices

Mono Methods Multiple Methods

Multi-Methods Mixed Methods

Multi-Method Qualitative Mixed Methods

Multi-Method Quantitative Mixed Models

Research Strategy/Design

▪ Surveys
find patterns in data
▪ Experiments
test hypotheses
▪ Archival research
extract evidence from extant literature
▪ Case Studies
study of a real life event
▪ Action Research
iteratively solve a problem
▪ Grounded Theory
Time Horizon

Longitudinal

several observations of the same subjects

over a period of time, sometimes lasting many years
Time Horizon

Cross-sectional

Characterized by comparison of different population

groups at a single point in time
Techniques and Procedures

▪ Sampling
from where to collect the data

▪ Data Collection
how to collect the data

▪ Data analysis
how to process the data and make meaningful insights from it
Quality of Research (Shipman (1988), The Limitations of Social Research)

1. If the investigation had been carried out again by different

researchers using the same methods, would the same results have
been obtained? (RELIABILITY)

What can cause issues with Reliability:

▪ Subject error (different results on different days)
▪ Subject bias (try to please researcher; social desirability, etc.)
▪ Observer error and bias
Reliability (consistency in measurement)

▪ Test-Retest reliability (across instrument)

▪ Parallel forms reliability (different versions of the instrument)
▪ Internal Consistency (within instrument)
▪ Average Inter-item correlation
▪ Split-half

▪ Inter-rater reliability (multiple judges)

▪ How to measure
▪ Correlations
▪ Cronbach’s alpha (internal consistency)
Improving reliability
▪ Clear conceptualization
▪ Standardization
▪ Increase the number of items
▪ Use more precise measurement
▪ Use multiple indicators of same construct
▪ Pilot-testing and replication
Quality of Research (Shipman (1988), The Limitations of Social Research)

2. Does the evidence reflect the reality under investigation? Has the
researcher found out what he/she thinks or claims it is about?
(VALIDITY/INTERNAL VALIDITY)

What can cause issues with Validity:

▪ E.g. refer to Threats to Validity of (Quassi) Experimental Designs

(Cook , T. D. & Campbell, D. T. (1979). Quassi Experimentation: Design and Analysis for Field Settings.
Rand McNally, Chicago, Illinois.)
Quality of Research (Shipman (1988), The Limitations of Social Research)

3. What relevance do the results have beyond the situation

investigated? (GENERALIZABILITY/EXTERNAL VALIDITY)

What can cause issues with Generalizability:

▪ Selection (specific to group)
▪ Setting (specific to setting)
▪ History (particular to past experience)
▪ Construct effects (only these groups has these effects)
Validity (accuracy in measurement)

▪ Face validity
opinion of researcher if items appears to measure what they are supposed to
measure

▪ Content validity
extent to which a measure represents all facets of a given construct

▪ Construct validity
items in construct adequately represent the construct being studied

▪ Criterion-related validity
results from the instrument accurately relate/predict relationship with external
variable(s)
Quality of Research (Shipman (1988), The Limitations of Social Research)

4. Is there sufficient detail on the way the evidence was produced for
the credibility of the research to be assessed? (CREDIBILITY)
Reliability and Validity