Training Busy Staff

to Succeed with Volunteers:

The 55-Minute Training Series
Copyright 2007
Betty Stallings

Energize, Inc.

Volunteer Program
Evaluation
2007, Betty Stallings - Energize, Inc.
S-1
Purposes of this Session
Identify the steps needed to establish a
volunteer program evaluation.

Explore 4 types of volunteer program
evaluation.

Learn to identify significant stakeholders who
need results of the evaluation communicated
to them.

2007, Betty Stallings - Energize, Inc.
S-2
Learning Objectives
Discuss the purpose and values of carrying out a
volunteer program evaluation.

Identify key steps and questions to be answered
prior to designing a volunteer program
evaluation.

Understand the difference in purpose and value
of 4 types of volunteer program evaluation
techniques.

Identify key stakeholders who must be engaged
in the creation/approval of the evaluation process
and given the results derived from it.
2007, Betty Stallings - Energize, Inc.
S-3
Definition
Volunteer program evaluation is a
process to determine the strengths,
weaknesses, and impact of your
volunteer program through use of
appropriate instruments or techniques
to gather this information.
2007, Betty Stallings - Energize, Inc.
S-4
Warm-up Exercise
What are some advantages to
evaluating the success of the volunteer
program in your organization, unit,
department, or for yourselves as
individuals?
2007, Betty Stallings - Energize, Inc.
S-5
Advantages to Evaluation
Can be used to set goals for future volunteer
involvement.

Creates buy-in from all who are involved in any aspect
of the volunteer program,

Can educate staff and volunteers about the elements of
a successful volunteer program.

If given to appropriate stakeholders, may
stimulate/support needed changes.

May anticipate future concerns that can be worked on
to avert significant future problems.

2007, Betty Stallings - Energize, Inc.
S-6
Advantages to Evaluation
Builds professionalism into the volunteer program.

Can help diffuse negative feelings.

Can analyze cost/benefits of the volunteer program.

Great tool for a new staff person.

Gathers various viewpoints from those involved in the
delivery of the volunteer program.

Gains a profile of the current program: the big picture.

Can be the basis for a staff retreat to enhance the
program.


2007, Betty Stallings - Energize, Inc.
S-7
4 Key Concepts of
Volunteer Program Evaluation
1. Assessment begins with clearly determining what
information you want, the best process to obtain
that information, and which stakeholders might be
interested in the results.

2. The most common methods of gathering data for
evaluating volunteer programs are:
Evaluating Output Statistics
Exploring Customer Satisfaction
Comparison to Outside Standards

2007, Betty Stallings - Energize, Inc.
S-8
4 Key Concepts of
Volunteer Program Evaluation

3. The ultimate information to derive from Volunteer
Program Evaluation is the impact the volunteer
program is having on the mission of the
organization.

4. After data and impact of volunteer activity is
measured, effectively communicate these to
significant stakeholders.
2007, Betty Stallings - Energize, Inc.
S-9
Key Concept 1

Assessment begins with:
clearly determining what information you
want,
the best process to obtain that
information, and
which stakeholders might be interested in
the results.

2007, Betty Stallings - Energize, Inc.
S-10
Preparation for Designing
Volunteer Program Evaluation
For what purpose is the evaluation being done? What
do you want to be able to decide as a result of the
evaluation?

Who are the audiences that will want this information?

Do we have the support of those who will be able to
carry out the suggestions arising from the evaluation?

2007, Betty Stallings - Energize, Inc.
S-11
Preparation for Designing
Volunteer Program Evaluation
What kinds of information are needed to make decision
and from whom do we need to gather that information?

What methods can be used to access that information?

What roles will be taken and by whom to:
Develop/pilot test/approve the questions.
Carry out the tabulation/analysis/interpretation of the results.
Present the material to appropriate audiences.

2007, Betty Stallings - Energize, Inc.
S-12
Discussion Question
Have you ever carried out an evaluation
and left out one of these 6 steps? What
impact was there on the
effectiveness/value of the evaluation?
2007, Betty Stallings - Energize, Inc.
S-13
Additional Issues to Consider
How confidentiality will be handled.

Do you have the necessary resources to carry out an
evaluation?

Who will lead the process?

How you obtain information will be impacted by the
size and culture of your organization.

Always involve a task force to carry out the evaluation.

Make certain that those who are in a position to carry
out the recommendations are in support of the
evaluation.
2007, Betty Stallings - Energize, Inc.
S-14
Key Concept 2

The most commonly used methods of
gathering data for evaluating volunteer
programs are:
Evaluating Output Statistics
Exploring Customer Satisfaction
Comparison to Outside Standards

2007, Betty Stallings - Energize, Inc.
S-15
Evaluating Output Statistics
This method measures quantitative
information about a volunteer program:
Numbers of volunteers, hours, clients served, etc.
Dollar value of donated time
Numbers of volunteers serving in different capacities
Specifics about volunteers, such as: gender, age,
ethnicity, education, etc.
Percentage of volunteers who also gave financial
donations
2007, Betty Stallings - Energize, Inc.
S-16
Methods of Computing the
Value of Volunteer Hours
Minimum Wage
Lowest equivalency

Imputed Wage System
Department of Labor figure calculated annually in
the US by Independent Sector

Equivalent Wage System
Calculating cost of equivalent paid job
classifications, plus fringe benefits.
2007, Betty Stallings - Energize, Inc.
S-17
Output Statistics
Potential Benefits
Change in numbers or kinds of volunteers in
your organization.

Information for volunteer insurance purposes.

Volunteers who are giving financial support to
the organization.

2007, Betty Stallings - Energize, Inc.
S-18
Output Statistics
Potential Shortcomings
Numbers do not deal with impact or quality
of the work donated.

There are many other quantity markers
beyond hours served that would be useful
such as:
Sources of new volunteers
Average length of service
Numbers of clients assisted
Range of jobs performed by volunteers
2007, Betty Stallings - Energize, Inc.
S-19
Exploring Customer Satisfaction
Asking for feedback from your customers
Paid staff
Volunteers
Executive management
Clients (if appropriate)
Funders
Community

Qualitative information elicited by:
Surveys
Interviews
Focus groups

2007, Betty Stallings - Energize, Inc.
S-20
Customer Satisfaction
Potential Benefits
Satisfaction from customers greatly impacts
the success of the volunteer program.

Determines major weaknesses and strengths
in the program from perspective of
customers.

Gains buy-in from staff for volunteer program

Great form of volunteer appreciation.

2007, Betty Stallings - Energize, Inc.
S-21
Customer Satisfaction
Potential Shortcomings
Answers not accurate if there are
organizational trust issues.

Must prepare those taking the survey
regarding reasons for survey and what you
will do with the results/ recommendations.
2007, Betty Stallings - Energize, Inc.
S-22
Activity
Look over the examples of customer
surveys in Handout #3.

What are the benefits and shortcomings of
each example?

Which questions would you keep or discard
for your organization and what others would
you add? Why?
2007, Betty Stallings - Energize, Inc.
S-23
Comparison to Outside Standards
Comparison to objective management standards
created by an outside body or authority.

Examples of such standards:
The Volunteer Management Audit (Energize, Inc.)
The Canadian Code for Volunteer Programs (Volunteer
Canada)
Points of Light Paradigm Project and Volunteer
Program Evaluation


2007, Betty Stallings - Energize, Inc.
S-24
Comparison to Standards
Potential Benefits
Education of excellence in standards of
volunteer management.

Identification of areas of weakness that may
be greatly impacting your program.

Gives you information that will help you
develop goals for improving the program.

2007, Betty Stallings - Energize, Inc.
S-25
Comparison to Standards
Potential Shortcomings
Takes dedication/time.

Not helpful if no action occurs after you
identify areas needing improvement.

May be challenging to compare your program
to a set of generic standards.
2007, Betty Stallings - Energize, Inc.
S-26
Key Concept 3
The ultimate information to derive from
Volunteer Program Evaluation is the impact
that the volunteer program is having on the
mission of the organization.
2007, Betty Stallings - Energize, Inc.
S-27
Impact or Outcome-Based
Evaluation
Focuses on results not just activities.

Question asked is: What has changed in the
lives of individuals, families, organizations, or
the community as a result of this program?

Keep asking: So what?
2007, Betty Stallings - Energize, Inc.
S-28
Impact or Outcome-Based
Evaluation
An outcome-based evaluation discovers:
What target groups will change?
What about the target will change and in what direction will it
change?
What will happen to create that change?
How will the change will be identified and tracked?
How will the change will be measured?
How will we know that our service is what impacted the change?
How long must the change remain in effect before being
considered successful?


2007, Betty Stallings - Energize, Inc.
S-29
Challenges of Impact Evaluation
Some attempts and research tools to demonstrate
the impact of volunteers.
None, to date, are without result flaws and all are
very demanding in terms of time.
We cant strictly evaluate the exact impact of a
volunteer, but in we can measure discrete planned
change in the recipients of volunteer service in
Knowledge gain
Values altered
Behavioral adjustment
Alteration in social or economic status

Attitude change
Improved skills
Client reaching set
goals
Mood, stress reduction

2007, Betty Stallings - Energize, Inc.
S-30
Example
The mission of the mentoring program is to
keep high-risk students out of gangs and
involved in further education.

To describe the impact of the mentoring
volunteers, begin to describe the program.
For each statement, ask: So what?
To whom might this matter?
Is this really evidence of something important?
2007, Betty Stallings - Energize, Inc.
S-31
Example. continued
We had 90 volunteers working on the mentor project for 3
months. (So what?)
The volunteers were mentoring high risk students at the
continuation high school. (So what?)
Most students reported liking their mentor. (So what?)
Most students began to turn in their homework more regularly.
(So what?)
The students began to develop more pride in their work and
began to get passing grades. (So what?)
As a result of receiving better grades, many were able to
transition back into regular high school. (So what?)

2007, Betty Stallings - Energize, Inc.
S-32
Example, continued
59% of high risk students being mentored for a minimum of 3
months by a caring adult tested higher on standardized tests,
got after-school jobs that they had kept for 2 months, and were
expressing interest in additional education to prepare them for
the work world. (So what?)
It has been statistically shown that students who are able to
keep an after school job and are motivated to get additional
training/education for future employment have a 70% less
chance of breaking the law and becoming dependent on drugs
and involved with street gangs.
OH, WOW, GREAT!!! So that is the impact! I would
love to be involved and/or support that program!
2007, Betty Stallings - Energize, Inc.
S-33
Activity
Find a partner.

One person explains the impact of some
volunteer work on the mission of the
organization.

The other person keeps asking, So what?
until both of you are satisfied that you
understand the impact of the volunteer work
being done.


2007, Betty Stallings - Energize, Inc.
S-34
Key Concept 4

After data and impact of volunteer
activity is measured, effectively
communicate these to significant
stakeholders.
2007, Betty Stallings - Energize, Inc.
S-35
Communicating to Stakeholders
Who needs the information?
Volunteers
Paid Staff
Executive Team or Executive Director
Board
Clients
Funders
The Community


2007, Betty Stallings - Energize, Inc.
S-36
Activity
What is your experience with volunteer
program evaluations?

How might you establish an evaluation
system for whatever involvement you have
with the volunteer program?

To whom would you give the results of that
evaluation to help improve the program in
your area and/or in the larger organization?
2007, Betty Stallings - Energize, Inc.
S-37
Summary
Answer key questions before designing your
evaluation.

Select the appropriate method(s) to access the
information needed:
Statistical Outputs
Customer Satisfaction
Standards-based evaluation
Impact/Outcome Evaluation

Share Information with key stakeholders.
Summary of key findings
Complete report on results and recommendations