Introduction to the

CARE International
Design, Monitoring &
Evaluation (DME)
Standards
by Jim Rugh
November, 2001

Introduction:
The CARE Impact Guidelines were developed in
1999 and have been widely accepted.
At the second Impact Evaluation Initiative (IEI)
conference (in Wood Norton, UK, August 2001) the DME
Standards and Guidelines were updated, based
on feedback from the DME Capacity
Assessments, and to be compatible with new
conceptual lenses such as Rights-Based
Approaches, Gender Equity & Diversity, Civil
Society, etc.

Introduction:

This updated presentation introduces the

proposed CI DME Standards and core
guidelines.
It begins with a perspective of where these fit
within CARE International (CI).

CI Vision and Mission

We seek a world of hope,

tolerance and social justice,
where poverty has been
overcome and people live in
dignity and security.

Previous list of CI principles

CI Vision and Mission

CI Programming Principles

Significant Scope
Fundamental Change
Working with Poor People
Participation
Replicability

If we adopted the RBA principles

CI Vision and Mission

CI Programming Principles

Affirm peoples right to participate in decision-making

processes that affect their lives.
Require identifying and seeking to address the root
causes of poverty and suffering.
Refuse to tolerate discrimination and inequities that
impede peace and development.
Hold all of us accountable for respecting and helping to
protect and fulfil human rights .

This presentation
focuses on the
Basic DME Standards

CI Vision and Mission

CI Programming Principles
CI DME Basic Standards
Core DME Guidelines
Array of other
detailed
guidelines

Fo
c

c
i
t
s
i
l
Ho lysis
a
n
A

THE

us
st
ed
ra
te
gy

DME
CYCLE
Re

fle
ct
pr
i ve
ac
tic
e

t
n
re
e
h
on
i
o
t
a
C
m
r
o
m
f
e
t
in
s
sy

We will now begin going over the CI DME Standards. But

before we do, lets look at the introductory paragraphs:

These CARE DME standards apply to all CARE

programming (including emergencies, rehabilitation
and development) and all forms of interventions
(direct service delivery, working with or through
partners, and policy advocacy).

These standards, as well as accompanying guidelines,

should be used to guide the work of project designers; as a
checklist for approval of project proposals; as a tool for
periodic project self-appraisal; and as a part of project
evaluation. The emphasis should not be only on
enforcement but also on the strengthening of capacity to be
able to meet these standards for program quality. At the
time of initial approval, if a project can not meet one or
more standards, allow for explanation of why, and what
will be done about it. More than a passed/failed
checklist, these call for a description of how well a project
meets each standard.

CARE International DME Standards

Each CARE project should:

1. be consistent with the CI Vision &

Mission, Programming Principles &
Values.
Projects should show how they will
contribute, ultimately, towards lasting
improvements in human well-being, hope,
tolerance, social justice, reduction in
poverty, and enhanced dignity and security
of people.

CARE International DME Standards

Each CARE project should:

2. be clearly linked to CO strategy

and/or long term programme goal.
Projects should not be isolated, but clearly
embedded in long term multi-project
programmes and strategic frameworks that
address the underlying conditions and root
causes of poverty and social injustice.

To achieve impact all of these need to address the same

target population.
Program Goal: Reduce
childhood malnutrition
Food
Production
Project
Goal:
Increase
availability
of food to
households

ASSUMPTION
(that others will do this)

Health
Project
Goal:
Decrease
diarrheal
disease
among
children

CARE project

Nutrition
Project
Goal:
Improve
quality of
food fed to
children

PARTNER will do this

Program and project goals at impact level

CARE International DME Standards

Each CARE project should:

3. Ensure the active participation and

influence of stakeholders in its
analysis, design, implementation, and
M&E processes.
Every project should be explicit about its
process of participation and consultation,
aiming for openness and transparency.

CAREs
experience
and
comparative
advantage
PARTNERS
DONORS
interests and
Interests
capabilities
Assessment of
Community
needs and
assets

VIABLE
PROJECT

CARE

DONORS

PARTNERS

COMMUNITY

CARE International DME Standards

Each CARE project should:

4. Have a design that is based on a

holistic analysis of the needs and
rights of the target population and
the underlying causes of their
conditions of poverty and social
justice. It should also examine the
opportunities and risks inherent in
the potential interventions.

Elements of Household Livelihood Security

Education
Health
Food

Economic
Security

Shelter

Whether these are

Community
Empowerment

described as needs or
rights, holistic diagnosis

Water

implies taking them all

Means:
Income, skills, time

into account.

Environment

Context:
Social, Political and
Cultural

WORLD
NATION
STATE

Bi- / Multi-lateral
Agencies
National / subregional / local
Government
Private Sector
NGOs

DISTRICT
CBOs

COMMUNITY
Social Networks

CAREs options include working with or through

partners, including policy advocacy Bi- / Multi-lateral
Agencies
National / subregional / local
Government
S
ct
re
Di

Through whatever
e
vic
er

mode, it needs to be
shown that ultimately
there will be impact
on Household
Livelihood Security.

Private Sector
NGOs
CBOs
Social Networks

Other lenses may be used as well

CIVIL SOCIETY
R
E
D
EN
G

HU
MA
N

RI
GH
TS

CARE International DME Standards

Each CARE project should:

5. Use a logical framework that

explains how the project will
contribute to an ultimate impact upon
the lives of members of a defined
target population.
Specify level of intervention (household,
community, institutional, societal); identify
key assumptions and provide validation for
its central hypothesis.

Converting the
diagnostic
assessment of
needs/opportunities
into a logical
problem/solution tree
is the essential step
in project design

Consequences

Consequences

Consequences

PROBLEM
PRIMARY
CAUSE 1

Secondary
cause 2.1

Tertiary
cause 2.2.1

PRIMARY
CAUSE 2

Secondary
cause 2.2

Tertiary
cause 2.2.2

PRIMARY
CAUSE 3

Secondary
cause 2.3

Tertiary
cause 2.2.3

High infant mortality rate

Children are malnourished

Insufficient
food

Need for strengthened

capacity of health
institutions
Flies and
rodents

Diarrheal
disease

Poor quality
of food

Unsanitary
practices

Need for
improved health
policies

Do not use
facilities
correctly

People do not
wash hands
before eating

Lower infant mortality rate

More Children are well nourished

Sufficient
food

Strengthened capacity
of health institutions

Fewer flies
and rodents

Less diarrheal
disease

Good quality
of food

Sanitary
practices

Improved health
policies

facilities
used
correctly

People wash
hands before
eating

Common terminology for project hierarchy

OUTCOMES
OUTPUTS

PROCESS
INPUTS

OUTCOMES
PROCESS

--- RESULTS ---

CAREs
CARE terminology for project hierarchy

IMPACT Equitable and durable

improvements in human well-being
and social justice
EFFECTS Changes in individual behavior or
systemic competence.

OUTPUTS
ACTIVITIES

Products of project activities

Interventions / processes implemented by project

INPUTS Resources needed by project (e.g. funds, staff, commodities)

Example for childhood malnutrition project

PROGRAM IMPACT Reduction in

malnutrition rate among children under five

PROJECT IMPACT Decrease in the incidence of

diarrhea

EFFECT Parents of children practice sanitary behaviors

OUTPUT Parents acquire knowledge about sanitary practices
ACTIVITIES Parents given training in sanitary practices
INPUTS Funding obtained, staff trained, training center set up

The quality of each level is

measured by the next higher level.

IMPACT !
EFFECTS which, if our hypothesis is valid,
should be shown to lead to ...
OUTPUTS, which, if our assumptions hold true, the
effectiveness can be measured by...
ACTIVITIES (interventions) which should lead to ...
If sufficient INPUTS are received , then we will be able to do ...

Comparison of logical framework terms used by different agencies

This has been called the Rosetta Stone of LogFrames

CARE International DME Standards

Each CARE project should:

6. Set a significant, yet achievable

and measurable final goal.
The final goal should be practical and doable, yet be at the outcome level
(intermediary impact or at least effect).
A project final goal should clearly contribute
to higher level program impact: equitable
and durable improvements in human wellbeing and social justice.

Significant
Long-term (ultimate) impact
(Strategic Plan or Program level)

PROJECT FINAL GOAL

Short-term (intermediate) impact
(Achievable & measurable during LOP)

Project Effects
Project Outputs

Achievable

CARE International DME Standards

Each CARE project should:

7. be technically, environmentally and

socially appropriate. Interventions
should be based upon best current
practice and on an understanding of
the social context and the needs,
rights and responsibilities of the
stakeholders.

CARE International DME Standards

Each CARE project should:

8. Indicate the appropriateness of

project costs in light of the selected
project strategies and expected
outputs and outcomes.
Project designers must be able to defend the
budget relative to its outputs, scale and
anticipated impact. M&E plan should include
methods for measuring cost effectiveness.

CARE International DME Standards

Each CARE project should:

9. Develop and implement an M&E plan

and system based on the logical
framework that ensures the collection
of baseline, monitoring, and final
evaluation data, and anticipates how the
information will be used for decision
making; with a budget that includes
adequate amounts for implementing the
M&E plan.

M&E Methodologies
Project M&E plans should provide sufficient detail
to clearly identify:
8evaluation design (based on key questions)
8sources of data (evidence to be collected)
8means of measurement (how to collect evidence)
8schedule for measurement (when)
8data processing and analysis
8dissemination of information to and utilization
by key stakeholders, and
8 responsibilities for each of these processes.

What should be
included in budgets
to assure
accountability?

ACCOUNTABILITY
Consider how many
resources we put
into accounting for
how funds have been
spent.

Then consider how

many resources we
put into monitoring
and evaluating
results achieved by
our projects.

FUNDS
ACCOUNTING
M&E of
RESULTS

CARE International DME Standards

Each CARE project should:

10. Establish a baseline for

measuring change in indicators of

impact and effect, by conducting a
study or survey prior to
implementation of project activities.
Distinguish between a diagnostic
assessment and baseline. The latter focuses
on measuring effect & impact indicators
required for before-and-after comparison
with evaluation.

We need to be
aware of the
relative focus and
purposes of
different forms of
assessment

Including
review of
secondary
data

Household
Livelihood
Security
Assessment

Long-Range
Strategic Plan

More
focused

CARE International DME Standards

Each CARE project should:

11. Use indicators that are relevant,

measurable, verifiable and reliable.
Indicators should be capable of yielding data
that can reveal vulnerabilities such as
gender, age and social class.
Both qualitative and quantitative measures
are acceptable as long as they can illustrate
discernible and significant change.

Indicators
8Indicators should be relevant to the
goals they represent, quantitatively
or qualitatively measurable,
objectively verifiable, reliable, meet
international professional
standards ...
8and yet be understandable and
appreciated by project participants
and other stakeholders.

Indicators
Be clear on appropriate levels of
indicators:
8Indicators that represent different
levels of goals and objectives in the
logframe hierarchy
8Different levels of precision from
broad issues to objectively
verifiable indicators and specific
raw data variables to be collected.

Example of levels of precision of indicators

Broad indicator:
8 Improve the well-being of children
Objectively verifiable indicator:
8 Level of childhood malnutrition measured as %
of children <5 below 2 SD of weight for height
Specific variables include:
8 numerator(s): # boys / girls measured whos
weight for height was <2 SD
8 denominator(s): # boys / girls between 12-60
months age measured in survey

CARE International DME Standards

Each CARE project should:

12. Employ a balance of evaluation

methodologies,assure an appropriate
level of rigor, and adhere to
recognized ethical standards.
The minimum is that there should be at least
a final evaluation that summarizes the
achievements and lessons learned by the
project. Evaluation events should utilize a
balance of methodological approaches to
assure triangulation, and a richness of data.

Appropriate mix of Methods

Employ appropriate combinations of
methodologies during
assessments, monitoring and
evaluation, including:
4Qualitative / quantitative
4Participatory / extractive
4Multi-disciplinary in focus
4Secondary / primary data sources

Your human
interest story
sounds nice, but
let me show you
the statistics.

QUALITATIVE

QUANTITATIVE

Your numbers
look impressive,
but let me tell you
the human
interest story.

DIFFERENT PARADIGMS
We need to be careful that we not get too
locked into the scientific, hypotheticodeductive paradigm (with a clear causeeffect chain attributable to a projects
interventions). It is not the only one that can
reveal reality in the world where CARE works.
Also, we need to beware of reductionism,
especially the attempt to reduce complex
realities to superficial, numerical scales -- to
quantify the qualitative.

DIFFERENT PARADIGMS
There are other, more culturally-sensitive
paradigms, including the perspectives of
participant communities themselves, on
whether or not they feel their lives have
improved.
We have to ask what evidence donors (and
the rest of us) would accept that impact
has been achieved.

Participatory approaches should be

used as much as possible

But even they should be used with appropriate

rigor: how many peoples perspectives
contributed to the story?

CARE International DME Standards

Each CARE project should:

13. be informed by and contribute to

ongoing learning within and outside
CARE.
Relevant research and previous evaluations
should inform the project design. Also,
lessons learned from a project should be
adequately documented for utilization in the
design of other projects. Learning should be
an organization-wide priority, supported by
frequent meta-evaluations.

Thats the end of the introduction to the CI DME Standards

If youve had enough you can end this presentation here.

Or proceed to learn more about basic DME guidelines.

A little more on basic

DME methodologies
consistent with the DME
Standards

An Overview of the DME Cycle in CARE

1) Long-Range Strategic Plan
2) Diagnosis, including Participatory Assessment of
Community Needs and Opportunities
3) Project Design (summarized in logical framework)
Submission of proposal to donors
4) Detailed implementation plan, including detailed M&E plan
based on life-of-project evaluation design
5) Baseline study
6) Monitoring of implementation
7) Evaluations: Mid-term + Final (+ Post-project)
8) Apply lessons learned to next planning cycle

An introduction to various evaluation designs

Illustrating the need for quasi-experimental
longitudinal time series evaluation design
Project participants

Comparison group

baseline
scale of major impact indicator

end of project
evaluation

post project
evaluation

Suggested criteria for determining

appropriate design for a projects
evaluation plan
For what reasons should longitudinal, quasiexperimental research/evaluation designs be
used?
To test hypothesis (i.e... correlation between interventions
outputs and impact) which has not been previously proven
through other evaluations or research
Pilot project to serve as model for wide multiplication
When donor insists that it is necessary to prove impact and
attribution
Donor willing to dedicate significant amount of budget (e.g.
10%-25% or even more) for research (M&E)

Suggested criteria for determining

appropriate design for a projects
evaluation plan
When might quasi-experimental impact
evaluation design not be necessary?
Proven intervention, correlation between interventions and
impact previously proven in acceptably similar conditions
Only need to verify that implementation and outputs comply
with standards
Reliable, valid and relevant secondary data available
When it would be unreasonable (or unethical) to use a
control group
Short-term (i.e. emergency); Questionable security situation
Only 3%-10% budget available for M&E.

Different levels of rigor

depends on source of evidence; level of confidence; use of information

Objective, High precision, More time & expense

Level 5: A thorough research project is undertaken to conduct in-depth

analysis of situation; P= +/- 1% Book published!
Level 4: Good sampling and data collection methods used to gather data
which is representative of target population; P= +/- 5% Decision maker reads
full report
Level 3: A rapid survey is conducted on a convenient sample of
participants; P= +/- 15% Decision maker reads 10-page summary of report
Level 2: A fairly good mix of people are asked their perspectives about
project; P= +/- 25% Decision maker reads at least executive summary of report
Level 1: A few people are asked their perspectives about project;
P= +/- 40% Decision made in a few minutes

Level 0: Decision-makers impressions based on anecdotes and sound

bytes heard during brief encounters (hallway gossip), mostly intuition;
Level of confidence +/- 50%; Decision made in a few seconds

Subjective , Sloppy, Quick & cheap

Determining appropriate precision and mix of multiple methods

Nutritional
measurements

HH
surveys

Focus
Groups

Nutritional
measurements

Focus
Groups

HH
surveys
Key
Informant
interviews

Large
group
Low rigor, questionable quality, quick and cheap

Participatory --- Qualitative

Extractive --- Quantitative

High rigor, high quality, more time & expense

Determining appropriate levels of precision for

events in a projects evaluation plan
High rigor

Same level of rigor

4

Baseline
study

Final
evaluation

Mid-term
evaluation

Needs
assessment

Special
Study
Annual
self-evaluation

2
Low rigor

Time during project life cycle

Project concept
Diagnosis Project Design Logframe M&E plan
The problem Project hypothesis: outputs + valid assumptions will

lead to outcomes
Target group

ASSUMPTIONS /
EXTERNAL FACTORS

Impact
INDICATORS

Effects
INDICATORS

INDICATORS

Outputs

Activities

Inputs

INDICATORS

INDICATORS

INDICATORS

Evaluation

Monitoring

Of Outcome / Purpose

Of Process / Operation

Upper Logframe

Lower Logframe

A good M&E plan will show all of these dimensions and how they relate

The Project Implementation Process

Was our hypothesis valid?

Impact

Effects

Are we doing well?

Project Efficiency
Outputs

Activities

Inputs

Project Effectiveness

Did we do the right thing in a worthwhile way?

DME-IS

Systematic

Plan

Detailed M&E

Framework

Logical

Three Major Components of

Good DME

Level of sophistication

Levels of DME-IS (from recent DME-IS conference)

4
3
2
1

Level 4: Fully integrated

software package
Level 3: Database & analysis software

Level 2: Common software programs,

e.g. Excel, Word
Level 1: paper-based, manual system

Information Requirements

We need to recognize that not all

projects are ready for level 4
DME-IS. Even at level 1
(manual systems) there is room
for improved systemization.
In fact, systemize before you
automate needs to be the guiding
principle behind all DME-IS.

The quality of DME in a project

can be greatly enhanced by using
a fully integrated DME-IS (like
MER), for it depends upon,
complements, and makes
functional a logical logframe and
well detailed M&E plan. Not only
for one project, but related
projects within a larger program.

Over-all Principles of good DME

8Holistic diagnosis of needs and
opportunities which includes
community participation
8Logical project designs
8Systematized monitoring systems
8Quality evaluations that measure
impact, are credible and useful
8Decision makers think evaluatively

Whats meant by
thinking evaluatively?
4Reality checks: We all need to seek
objective feedback, gain perspectives
on our work; learn lessons and apply
them.
4Rational decision-making: Before
making decisions be clear on what
evidence we have, from whom it was
obtained, and how reliable it is.

Whats meant by
thinking evaluatively?
4Challenge paradigms: -- Our own as
well as others. Be visionary. Think out
of the box. Ask what other perspectives
would be informative.

Whats meant by
thinking evaluatively?
4Be accountable: We owe it to our clients
(intended beneficiaries as well as donors)
to document not only our use of inputs
and production of outputs, but also what
outcomes are achieved; i.e. what
difference our projects have made in the
lives of real people.

In these ways we can assure that

all of us working together
will help make the world a
better place for all !!

Nagarkot, Nepal; photo by Richard Caldwell