Program evaluation

Sebastian Uijtdehaage
bas@mednet.ucla.edu
310.794.9009
Why do you evaluate?
 Five reasons of evaluating
1. FORMATIVE
informs program development
2. SUMMATIVE
demonstrate program’s success and impact
3. UNDERSTANDING
how and why did program work?
4. DISSEMINATION
sharing best practices
5. RECOGNITION
scholarship, academic credit, leadership

Stufflebeam 2003
What type of evaluation
should I conduct?
Outcome &
Impact

Did it work?
 Dan Stufflebeam:
CIPP Model of Program Evaluation
CONTEXT

Needs
Assessment

What should we
do?
PRODUCT INPUT

Program
Outcome & Planning
Impact
How shall we do
Does it work? it?

PROCESS

Implementation

Are we doing as
planned?
What type of evaluation
should I conduct?
Formative or Summative?
formative & summative evaluation
Formative Evaluation

Context What needs to be

done?
Input How should it be
done?
Process Is it being done?

Product Is it working?

Stufflebeam 2003
formative & summative evaluation
Formative Evaluation Summative Evaluation

Context What needs to be Were important

done? needs addressed?
Input How should it be Was program plan
done? defensible?
Process Is it being done? Was the plan well
executed?
Product Is it working? Did the effort make
a difference?

Stufflebeam 2003
What type of evaluation
should I conduct?
Quantitative or Qualitative?
Quantitative or qualitative measures?
Quantitative measures Qualitative measures
HOW MUCH…? WHY and HOW?

• How much did the • Why did the program

participant change? work?
• How much did the • How did participants
community change? change as result?
What makes good evaluation?
Evaluation must be planned ahead
• Evaluation should be part of program planning

• Planning and resource allocation makes

evaluation more successful
– Example: long-term tracking of participants
 Confidence being admitted in a
medical school
4.8

4.6

4.4

4.2 Start of program

End of program
4

3.8

3.6
Category 1
What else makes good evaluation?
• Scientific rigor
• Evaluation produces useful information that
helps make decisions
– Assessment should be done with a particular
purpose in mind
• Evaluation is tailored to your program
objectives
• Evaluation is done with conceptual framework
in mind
 LOGIC MODEL
CONDITIONS and DRIVERS

Resources Activities Output Outcomes Impact

The Planned Work The Intended Results

 LOGIC MODEL
CONDITIONS and DRIVERS

Current situation that precipitates the need for

the program
 LOGIC MODEL
CONDITIONS and DRIVERS

Resources Resources
Activities and inputs
Outputneeded toOutcomes
operate the program:
Impact
personnel, facilities, materials, equipment
 LOGIC MODEL
CONDITIONS and DRIVERS

Resources Activities Everything

Output the program does if it had Impact
Outcomes access to
the resources
 LOGIC MODEL
CONDITIONS and DRIVERS

The “deliverables”—the events,

products and services
Resources Activities Output Outcomes Impact
Number of participants, classes,
brochures, advisors, etc
 LOGIC MODEL
CONDITIONS and DRIVERS
If the program
delivers, the
participants
Resources Activities Output Outcomes will benefit
Impact in
certain ways:

Knowledge,
skills,
behavior,
attitudes

Immediate,
mid-term and
long-term
outcomes
 LOGIC MODEL
CONDITIONS and DRIVERS

Resources Activities Output Outcomes Impact

If these benefits to participants are achieved,

then certain changes in organizations and
communities might be expected to occur
 LOGIC MODEL
CONDITIONS and DRIVERS

Resources Activities Output Outcomes Impact

The Planned Work The Intended Results

 LOGIC MODEL
• Links processes and outcomes:
– Funders love this!
• Great for program planning
– Work backwards
• Evaluation is built in!
• Ain’t perfect
Your turn…