Consumer-Oriented Evaluation Approach

Beth Holz
EDF 5461 Introduction to Program Evaluation

By PresenterMedia.com
Agenda
History


Applications

Strengths, Limitations & Use

Summary

Sources

Item 1
Case Study

Item 2
Item 3
Item 4
Item 5
Item 6
History
One of the oldest evaluation approaches

Comprehensive judgments of the quality of a program or product

Judges the quality of products for potential customers

Used by government agencies and independent consumer advocates

Became important in educational evaluations in
mid 1960s
Not concerned with field expertise, but evaluators
expertise
Concerned with quality as MEASURED not
experienced
Based on Evaluation logic and quantitative
methods
Determine Criteria
Examine or measure performance
Synthesize information





Michael Scrivens Model

Determine criteria
Evidence of achievement of important
educational objectives
Evidence of achievement of important
non- educational objectives
Follow-up results
Secondary and unintended effects
Range of utility
Moral considerations
Costs


Scrivens Evaluation Logic - Step 1

Examine or measure performance using

Key Evaluation Checklist
Generic (products, personnel, programs, evaluation
Critical emphasis is merit and worth
Broad use, both consumers and evaluators
Fosters comprehensiveness in evaluations






Scrivens Evaluation - Step 2

Product Evaluation Checklist
Products
Critical emphasis is demission's of product soundness
Broad use, consumer and evaluators
Fosters comprehensiveness in product evaluations



Step 2 (cont.)

Step 3
Synthesize results
Provide results to consumer

The goal of the WWC
is to be a resource for
informed education
decision making.

Identifies studies that
provide credible and
reliable evidence of
the effectiveness of a
given practice,
program, or policy.




Expert, independent,
nonprofit organization
whose mission is to
work for a fair, just,
and safe marketplace
for all consumers and
to empower
consumers to protect
themselves.


Watch and study
customers reactions
Use consumer-oriented
evaluation for formative
purposes.


Applications
Public and Private Organizations and Industries
Consumers make informed choices
based on a comprehensive evaluation
of products or programs
Consumers can compare outcomes
between products or programs
Saves consumers time
Consumers are more aware of market
gimmicks and tricks

Strengths
Time required to complete such
comprehensive evaluations is passed on
to consumer in the form of higher
prices
Dependence on outside agencies to
perform evaluations
Stringent criteria might inhibit product
creation

Limitations
Summative evaluation
Consumer focused
Focused on assisting
consumers make purchasing
decisions


Formative evaluations
Organization & Consumer Focused
Evaluations occur throughout the
life-cycle of product development
Recognizes multiple stakeholders
Consumers
Organizations
Others involved in process
Use
Assists the PUBLIC understand the Quality, Value, Merit or Worth of
Product, Program, or Policy
Relies on transparent and quantitative methodologies made by an
evaluator, NOT an expert
Uses Scrivens Logic of evaluation
Although still used in government and industry, the consumer-oriented
approach is losing prominence in professional evaluator literature











Conclusion

Sources:
http://www.consumerreports.org/cro/about-us/index.htm
http://ies.ed.gov/ncee/wwc/aboutus.aspx
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2012).Program evaluation alternative
approaches and practical guidelines. (4th ed.). Upper Saddle River, NJ: Pearson
Education, Inc.
Picture from PresenterMedia.com