Sunteți pe pagina 1din 114

Cavendish University

Zambia

BACHELOR OF ARTS IN PUBLIC RELATIONS


(BA PR)

PRINCIPLES OF MONITORING &


EVALUATION
(BBA213)
MODULE
i
Table of Contents

INTRODUCTION ............................................................................................................................................. 1

COURSE OBJECTIVES ..................................................................................................................................... 3

LEARNING OUTCOMES .................................................................................................................................. 3

ASSESSMENT CRITERIA ................................................................................................................................. 4

COURSE OUTLINE .......................................................................................................................................... 4

MODULE DESCRIPTOR................................................................................................................................... 5

1. MONITORING AND EVALUATION CONCEPTS........................................................................................ 6

1.1. Description .................................................................................................................................... 6

1.2. Learning Outcomes ....................................................................................................................... 6

1.3. Introduction .................................................................................................................................. 6

1.4. Project ........................................................................................................................................... 6

1.5. Programme ................................................................................................................................... 7

1.6. Monitoring .................................................................................................................................... 7

1.7. Evaluation ..................................................................................................................................... 9

1.8. Aims............................................................................................................................................. 11

1.9. Methods and Methodologies...................................................................................................... 12

1.10. Quality Assurance ................................................................................................................... 14

1.11. Revision Exercise ..................................................................................................................... 15

1.12. Suggested Answers to Module 1 Revision Exercise ................................................................ 16

2. LOGIC MODEL PROGRAMME DEVELOPMENT .................................................................................... 17

2.1. Description .................................................................................................................................. 17

2.2. Learning Outcomes ..................................................................................................................... 17

2.3. Brief History of Logic Models ...................................................................................................... 17

ii
2.4. Construction of A Logic Model .................................................................................................... 22

2.5. Logic Model Formats ................................................................................................................... 23

2.6. Benefits of Logic Models ............................................................................................................. 26

2.7. Situation ...................................................................................................................................... 28

2.8. Assumptions ................................................................................................................................ 30

2.9. A Community Support Initiative.................................................................................................. 32

2.10. Community Nutrition Initiative ............................................................................................... 33

2.11. Cause-Effect Relationships or “Causation” ............................................................................. 35

2.12. Revision Exercise ..................................................................................................................... 42

2.13. Suggested Answers to Module 2 Revision Exercise ................................................................ 43

3. MONITORING AND EVALUATION: SYSTEMS/WORK PLANS ................................................................ 46

3.1. Description .................................................................................................................................. 46

3.2. Learning Outcomes ..................................................................................................................... 46

3.3. Introduction ................................................................................................................................ 46

3.4. Revision Exercise ......................................................................................................................... 57

3.5. Suggested Answers to Module 3 Revision Exercise .................................................................... 58

4. MONITORING AND EVALUATION PROCESSES .................................................................................... 60

4.1. Description .................................................................................................................................. 60

4.2. Learning Outcomes ..................................................................................................................... 60

4.3. Introduction ................................................................................................................................ 60

4.4. WRITING AN EVALUATION REPORT ............................................................................................ 74

4.5. REVISION EXERCISE ..................................................................................................................... 76

4.6. SUGGESTED ANSWERS TO MODULE 4 REVISION EXERCISE ........................................................ 76

5. MONITORING AND EVALUATION OF INSTITUTIONAL CAPACITIES ..................................................... 79

5.1. Description .................................................................................................................................. 79

iii
5.2. Learning Outcomes ..................................................................................................................... 79

5.3. Study Notes ................................................................................................................................. 80

5.4. Physiological Needs..................................................................................................................... 88

5.5. Psychological Needs .................................................................................................................... 89

5.6. Revision Exercise ......................................................................................................................... 92

5.7. Suggested Answers to Module 5 Revision Exercise .................................................................... 92

6. DISSEMINATION OF INFORMATION AND DATA ANALYSIS TOOL (SPSS) ............................................ 95

6.1. Description .................................................................................................................................. 95

6.2. Learning Outcome ....................................................................................................................... 95

6.3. Study Notes ................................................................................................................................. 95

6.4. Revision Exercise ....................................................................................................................... 105

6.5. Suggested Answers to Module 6 Revision Exercise .................................................................. 105

7. BIBLIOGRAPHY AND REFERENCES ..................................................................................................... 107

iv
INTRODUCTION
The Principles of Monitoring and Evaluation (M&E) is designed in recognition to its growing
demand and importance in the implementation of projects/programme. M&E is an integral
management function and yet little emphasis is being done in the management of
projects/programmes. These elements can be used to improve programme delivery, increased
output at minimum cost.
Everyday we face new challenges and the need to improve on yesterday’s practice is becoming
increasingly enormous. But in order to have a breakthrough in performance, there is need to be
courageous, creative and flexible. So one has to be courageous enough to pass bold decisions
that may, sometimes make us step outside ‘our comfort zone’, creative enough to make change
and embrace new ideas and be flexible enough to take heroic actions which will enable the
organization to move forward.
Nowadays, change is inevitable. It is a situation where we can either ‘change or die’ (Wyzocki,
2003). It is in this light that rigid organizational structures of control which were introduced at
the turn of the century are rapidly disappearing. With these structural changes and worker
empowerment comes the need for all of us to have solid project management skills. ‘White
collar’ managers are no longer relevant. Modern managers are required to be vibrant enough to
anticipate changes that will affect their organizations in one way or the other; and must be skilled
enough to make those changes when they come.
Organizations must be aware that the wind of change is blowing and the window of opportunity
is narrowing and constantly moving. So it is time to take advantage of the situation than miss the
whole business opportunity. That is why even governments can no longer remain as referees,
creating an enabling environment for business houses to make money. Government departments
have been urged to remain afloat by contributing to the national treasury in whatever way they
can.
Because of this wind of change, the need for M & E skills can not be over-emphasized.
Organizations no longer look at ‘what a customer wants’ but ‘what a customer needs’. This is
true for all kinds of institutions, whether they are profit making or non profit making.
Therefore, the traditional world of project management belongs to yesterday.

1
In agreeing with this statement, Wyzocki (2003) wrote in his book
that it is seriously out of alignment with the needs of our
businesses. The problem is that project management is the
hammer and all projects are seen as nails. The nature and
characteristics of the project must dictate the type of management
approach to be taken. Anything short of that will fail. Projects
have fundamentally changed but our approaches to manage them
have not. We need a more robust approach to project
management – one that recognizes the project environment and
adapts accordingly.
This is a world of fast-paced, high-change and complex
projects… traditionists have tried unsuccessfully to adapt their
ideas to these types of projects. The failure rates are their
testimony to their inability to adapt traditional thinking to a non-
traditional environment.
Therefore change is not just about individuals, but equally the organizations for which those
individuals work. If one lags behind then the other suffers, and consequently leads to poor
products or services. In the end the organization has no choice but to give in to the pressure and
wind up. So there is only ONE WAY to go … ‘change’ … if you want to survive the troubled
waters … ‘or die’ … so that you are swept away in a rain storm.

2
COURSE OBJECTIVES
The course has been designed in modules of study, which follow the naturally occurring phases
of monitoring and evaluation processes. The concepts commonly used in M & E have been
explained to give you a ‘kick start’. It is expected that students will be equipped with knowledge
and skills that will enable them develop their ability and capability to monitor and evaluate all
aspects of any project/programme.
The course aims at providing students with an overview of basic concepts in monitoring and
evaluation, and offers an explanation to the fundamental similarities and differences between
them. It will introduce you to the logic models, their functioning and application to planning.
Students are expected to plan, develop monitoring & evaluation systems and work plans for their
own identified projects/programmes. In addition, students will be expected to carry out the actual
monitoring and evaluation using identified tools and methods, and present the findings in report
form.
The course will also provide students with an opportunity to carry out an institutional audit, in
order to determine their capacities and capabilities in the context of monitoring and evaluation.
Lastly, it will explain the process of information dissemination and how evaluation information
can be collected, processed and analysed using data analysis software called SPSS.

LEARNING OUTCOMES
At the end of this course, the student should be able to:
1) define and explain the concepts used in monitoring and evaluation
2) develop the basic logic model and use it in planning process
3) develop monitoring and evaluation systems and work plans
4) carry out monitoring and evaluation processes using identified tools and methods
5) carry out institutional audit to determine capacity and capability
6) explain information dissemination and its importance construct evaluation reports

3
ASSESSMENT CRITERIA
To achieve each outcome listed above, the student must demonstrate ability to:
1) adequately define and explain the concepts used in monitoring and evaluation
2) successfully develop the basic logic model and use it in planning process
3) adequately develop monitoring and evaluation systems and work plans
4) carry out monitoring and evaluation processes successfully using identified tools and
5) methods
6) adequately carry out institutional audit to determine capacity and capability
7) correctly explain information dissemination and its importance
8) correctly analyze M & E information and results using data analysis software (SPSS)
9) write evaluation reports satisfactorily

COURSE OUTLINE
The course has been designed around six modules, namely;
1) Concepts used in Monitoring and Evaluation
2) Logic Model Programme Development
3) Monitoring and Evaluation Systems/ Work plans
4) Monitoring and Evaluation Processes
5) Monitoring and Evaluation of Institutional Capacities
6) Dissemination of Information and Data Analysis Tool (SPSS)
7) Each module has been outlined in the following manner:

4
MODULE DESCRIPTOR
The Module (Principles of Monitoring and Evaluation) is designed to recognize the growing demand and
importance of M&E as a Management function. The module is aimed at imparting a solid theoretical
background as well as fostering essential practical skills; the module will play an important part in the
development of a variety of skills vital in the achievement of both individual and organizational goals.

LEARNING OUTCOMES
These are the skills and knowledge students are expected to master; against which they would be
examined.
STUDY NOTES
These are the notes of the module which you are expected to study very well. But as a self study
student, you are advised to have access to internet which will give you more information you
may need, including case studies, so as to supplement on these study notes.
In addition, you should take keen interest in reading widely, especially the current publications
of journals, hand books and reference books. You can also access project papers done by
international organizations like IFAD, USAID, ILO, UNDP, UNFPA, World Bank, including the
many papers which have been done by world renowned scientists and academicians.
REVISION EXERCISES
These are given at the end of the study notes. They are meant for self assessment and should help
you understand the study notes. You are expected to answer all the questions that follow and
send them to your tutor for marking.
You will be assessed on how well you answer the questions, including presentation, clarity, flow
of information and the wide readership exhibited in your write-up.
SUGGESTED ANSWERS TO THE REVISION EXERCISE
These are given after the questions. You are strongly urged NOT to read them BEFORE
attempting the questions given in the exercise.

5
1. MONITORING AND EVALUATION CONCEPTS
1.1. Description
This module aims at introducing basic concepts in monitoring and evaluation. It explains the
purposes of M & E and their similarities and differences. It ends by explaining what the result
based management is all about in the context of monitoring and evaluation.

1.2. Learning Outcomes


At the end of this module, the student should be able to:
1. Define and explain the basic concepts used in M & E.
2. State the purposes, similarities and differences between M & E.
3. Explain the differences between input, output, outcome and impact.
4. Explain the results – based management in the context of M & E.

1.3. Introduction
There are many concepts associated with monitoring and evaluation. The concepts given below
are in common use and are meant to help you understand the notes that follow.

1.4. Project
A project is sequence of unique, complex and connected activities having one goal or purpose
that must be completed by a specific time, within budget, and according to specification.
The activities in a project must be unique meaning such a project has never happened before and
will never happen again under the same conditions (Wyzocki, 2003). The activities are so
complex in nature and are not repetitive like painting a house or washing a car. For example,
designing intricate interior parts of a two storey shopping mall can be a complex activity. When
the activities are connected, they are regarded to be linked in some logical order or sequence.
They are considered connected because the output from one activity is the input to another.
Every project has one goal, eg constructing a school; but this can be broken down in many sub
projects like building classrooms, offices, library and teachers’ houses; all meant to be a
complete school. Projects have a specified completion time and this can be set by contractors or
other stakeholders. Projects have resource limits such as specified funds, equipment or machines
or materials to be used.
6
Projects specifications can be self imposed but are meant to involve public interest. For example
the hospital complex can be said to be modern in nature with the state of art equipment and to be
opened in six months time. But whether these will be a reality or not remains to be seen. In
industry, this can be referred to as scope.

1.5. Programme
Program is a set of organized activities supported by appropriate resources to achieve an
intended result. It may be narrow or broad in scope with the target being individual, group,
system, or community. But the methods used for service interventions include; education and
training; outreach; community mobilization; advocacy; research; and policy development.
Unlike projects, programmes can have many goals. There are five parameters that are so
important to the success or failure of the projects/programmes and these include;
• Scope is the functional specification of the functional specification of the project/programme.
It tells us not only what will be done but even what will not be done.
• Quality can be broken into two parts: product quality which refers to the quality of the output
from the project and process quality which refers to the quality of the process itself.
Product/service quality can be measured using identified quality control tool, but continuous
quality improvement can be effective tools to measure process quality.
• Cost of undertaking the project is another variable that defines the project.
• Time frame or the deadline by which the project must be completed is important. To a certain
extent, cost and time are inversely related to one another. The time a project takes to be
completed can be reduced, but cost increases as a result.
• Resources are assets of the project and these include people, equipment/machinery and
materials and all that is necessary in ensuring successful completion of the project. But all
these must be done during planning stage; estimates of time, cost, and identification of
resources needed to deliver the scope, and quality of the project needs to be done.

1.6. Monitoring
Simply monitoring keeps a finger on the ‘pulse’ of your programme. It provides a means of
finding out the manner in which in which the programme is being put in action. In short, this
process is concerned with collecting feed back on how the programme is being implemented.

7
Monitoring is therefore a routine and systematic collection and recording of information
concerning a programme, for the purpose of determining its progress against its preset plans. In
other words, it is a process of finding out whether a programme is heading in the right direction
or not. If it is not, then corrective measures ought to be taken soon than later before resources are
wasted.
The essence of monitoring is to control, and this can be undertaken by way of rate of
implementation.
This means controlling the pace at which implementation is being done. If it is too slow, then the
costs are likely to be high and if it is too fast, then quality is likely to be compromised. It is
important therefore to strike a balance between the two.
Utilization of resources is another element that can be used to control. Resources can create a
constraint if they are affected by availability and utilization. Control here would entail that
resources are utilized economically ie with minimum wastage. If wastage is not controlled, then
the overall cost of the project is likely to go up too.
Another element that can be used to control is checking of how the objectives are being
implemented as the programme is running. This can best be achieved through monitoring.
Monitoring does not end at ‘observing’ what is happening and generate reports. It should involve
discussing with the people on the ground why and how deviations are coming in and render
immediate suggestions as solutions. It is for this reason that monitors do not have to be
‘academicians’ (those who know the subject by the book) but they should be technocrats (those
who have the know-how) and with enough experience to offer professional advice. If there is no
person well versed in the project, then the services of a specialist can be sources.
Monitoring is therefore that element which keeps the project on the track, on time and within
budget limits. Each project, no matter how simple or complex, should be assessed with the
appropriate level of control; but it should be noted that too much control would be time
consuming and too little control would be too costly. The main purpose of monitoring is to
improve project implementation and management. It is a way of obtaining feedback about the
progress of the project. Feedback would also give an indication to whether the resources are
being well utilised.

8
Monitoring has many useful intentions. Data collected from the exercise can be used for
continuous improvement in the programme structure, content and process. In addition, through
monitoring people can become aware of the changes which could have taken place in
professional practice and technological advancements emerging on the market. It is also meant to
ensure that educational settings and methods of instruction lead to effective and efficient learning
experiences. It is therefore important that monitoring must focus on procedures that promote
integration, application and assessment of principles.
As an activity, monitoring would invoke critical thinking and problem solving techniques that
will bring about need for corrective action. And information gained from such an activity will
provide useful basis for strengthening or expanding the scope of the programme. In short,
monitoring can be a useful tool in corrective decision making process.

1.7. Evaluation
Evaluation is a process of judging the value on what the project/programme has achieved,
against the set objectives and the activities which were planned to take place. It is a process of
finding out how close you have come to achieving your goals, through outcomes and impact of a
programme. Evaluation is not only an important means of demonstrating the value or worth of
your project to the stakeholders or sponsors but it is an assurance that the programme is
achieving what it was intended to achieve, and through evaluation the matter can be proved right
or wrong. Many lessons can be learned when carrying out evaluation including constraints that
may hinder the project achieving its objectives. Besides, this is a good opportunity for planners,
evaluators and implementers to assess the benefits and costs that accrue to the target groups.
Lessons drawn from implementation and monitoring experience would be used to ensure future
programmes do not face similar hurdles.
In education circles, evaluation can help identify programme effects and assist staff to find out
whether their programmes have any impact on the knowledge and attitudes of their students. So
evaluation can be said to be a systematic method of collecting data and analyzing it so that
informed decisions can be made concerning a programme under review. Largely, evaluation can
be used to find out what knowledge, skills and attitudes a programme should address. It will also
bring out any evidence that will show whether the objectives have been met or not. So for a
programme to be successful, it should carry objectives which must be clear, realistic and defined
9
in measurable terms. Apart from selecting from amongst different approaches and decide which
ones can best achieve the goals, it may be necessary to try out new programme designs that will
show the extent to which the target group is benefiting from it.
The table below shows the similarities and differences between monitoring and evaluation:

MONITORING EVALUATION
1. A means to control the progression of a 1. A means to determine the need and worth of a
programme programme.
2. To determine how the set objectives are being 2. To determine if the set objectives were
achieved. achieved.
3. To obtain feedback whether the resources were 3. To determine if the programme was worth the
being utilized properly effort and resources used.
4. Concerned with process control 4. Concerned with success or failure or
programme,
5. Undertaken when the programme is running. 5. Undertaken at the end of the programme cycle.
6. Collect information and data through 6. Collect information and data through
monitoring instrument. evaluation instrument.
7. Can be carried out by all stakeholders at all 7. Can be done by trained personnel
levels
8. Information obtained can be used to strengthen 8. Information obtained can be used to decide if
or expand the scope of an existing programme the programme should be modified, continued
or cancelled altogether.
9. Determine whether current approaches are still 9. Determine whether the approaches must be
useful, time efficient and cost effective continued, modified or new ones designed.
10. Allows for efficient but effective mid-course 10. Allows for ultimate informed decisions to be
correction made on the future of a programme.

As can be seen above, both monitoring and evaluation are processes that compliment each other,
and both are meant to improve programme delivery. Therefore, they both need careful planning,
commitment from all concerned and some investment of time.
However, evaluation, just like monitoring has many useful intentions. It can help streamline
service delivery to target groups and cut down on the cost of that service delivery, and this can
10
lead to increased efficiency so that more people can benefit from it, and redirect the resources to
those who it them most. The areas of the programme which had been successful should be
identified and strengthened, but where there had been notable weaknesses, necessary
improvements should be made on them.
Organizations always want to show that they are working to high standards and that they are
achieving the objectives prescribed for them. This is done in response to increasing pressure
from donor agencies, including governments who want them to demonstrate how well they are
doing and to what extent they are achieving the objectives; in order to secure new funding.
Monitoring and evaluation both compliment each other in many ways. They are both means to
ensure smooth progression and determination of the worth of a programme.
Both monitoring and evaluation can be used for development. Through these activities, one
would have an insight of what the organization is all about and make use of the lessons learned
through the process.
Both monitoring and evaluation can be used for accountability purposes – to show the
stakeholders that you are capable of delivering the goods, at minimum cost and within
reasonable time. It is only through ‘proof of success’ that you will be assured of more funding;
hence continued sustenance of the project. So it is not enough to know whether you are
achieving your objectives at all, but it is equally important to know whether you are actually
bringing about any changes that you had hoped for. While it is important to monitor if the
outcomes you planned to happen have actually happened, it is equally important to know
whether there are any expected or unexpected results.

1.8. Aims
Aims describe the change (s) you want to see in your target group, in broad general terms. These
aims can be called the “mission statements.” But the overall aim is likely to be too broad a
statement to allow for meaningful and detailed planning or to provide guidance on what can be
monitored and evaluated. The other part can be called “specific aims.” These are more precise
statements about different aspects of the overall aim. An aim can also be said to be a goal; the
result or achievement toward which effort is directed.
OBJECTIVES

11
Objectives define the practical activities that are designed to bringing about a change in the
target group; they are the basis to achieve the aims. Objectives answer the questions like, “how
will you bring out the changes”? and “what do we need to do in order to achieve such changes”?.
There is a direct link between each aim and its objectives. In order to achieve some aim, it may
be necessary to carry out several different activities. Therefore, each aim may have more than
one objective and some objectives may relate to more than one aim. An objective can also be
said to be efforts or actions are intended to be attained or accomplished.
VISION
The vision consists of ideas shared by stakeholders who dream of a “better world” in general,
and their contribution to that better world. It is some kind of imaginative conception or
anticipation of wealth and glory. And it is this entrepreneurial vision that drives people to work.
IMMEDIATE EFFECT
These are the outcomes which occur as a result of the activities, usually during or after those
activities. They include those immediate outcomes which are intended or unintended.
RESOURCES
These are the things which are needed in order for the activities to take place. They include
human resources, infrastructure, materials, finances, machinery and equipment. The list can also
be extended to include time, skills, goodwill etc.

1.9. Methods and Methodologies


Although these terms are sometimes used interchangeably, methods should be distinguished
from methodologies. Methods are the specific tools and techniques used for data collection and
information exchange.
Methodologies define a particular approach. They provide a framework for selecting the means
to obtain, analyse and exchange information about a particular issue. They define what should be
learned or shared and how it should be presented.
CRITERIA
These are a set of broad guidelines for the selection of outcomes. Criteria can also be said to be
degrees to which outcomes must be assessed. A criterion is a standard of judgment; a rule or
principle for assessing or testing something.
INDICATORS
12
These are the means by which either improvement towards or achievement of a goal can be
assessed. These are signals which will show whether objectives are being achieved or not.
ACTIVITIES
These are the day-to-day actions undertaken by members of an organization or group. They are
specific deeds, actions, functions, or sphere of action. In academic circles, activities are tasks that
involve direct experience and participation of the students.
INPUTS
These are the resources which go into an organization or programme in order to produce the
results desired by the organization. Inputs allow us to achieve the desired outputs, which lead to
outcomes, hence, the ultimate benefits to the target group.
OUTPUTS
These are the works generated by the programme/project. The term output is used to describe
project activities, services and products. These are usually the tangible results of the major
processes of the organization. Outputs can be used as indicators of success. Unless outputs are
directly associated with achieving benefits, they may be end up to be poor indicators of success.
For example, a college may be producing many trainees annually but only a few may be fully
trained.
OUTCOMES
Outcomes are the expected (even unexpected) changes or impacts that may result from a
particular programme or intervention. They are the positive (even negative) impacts on those
people who are targeted to benefit from the programme. Outcomes are usually specified in terms
of short-terms, intermediate and long-term. They can also be referred to as benefits that will
come about as a result of activities.
IMPACT
Impact is the final or ultimate result ie the long-term outcome. Although the terms impact and
outcome are used interchangeably, impact refers to any outcomes that answer the question,
“what difference does it make?”
Information gathered from here can be used to:-
• Improve on the design and implementation strategies of an on – going programme.
• Make an overall judgement on the effectiveness of the programme which has come to an end.

13
• Act as a beacon of accountability.
• Generate better knowledge about good professional practice.
• Help identify more effective implementation strategies.
• Assist in decision making relating to effectiveness, impact and sustainability of the
programme.
DATA
Datum is a Latin word meaning, ‘a thing given’. Today the word is used in English to mean both
singular and plural form. It means facts or pieces of information.
INFORMATION
This is knowledge communicated or received concerning a particular fact or circumstance; news.
Knowledge gained through study, communication, research, instruction, etc.; data; facts.

1.10. Quality Assurance


A system for verifying and maintaining a desired level of quality in a product or process, as by
planning, continued inspection, and corrective action as required.
RESULTS BASED MANAGEMENT
In a results based situation, management is driven by a high expectation of 100% pass record and
they would wish to maintain the same record year in year out. All resources and effort put in are
geared towards achieving a 100%, consistently
The principles followed here include:
• That all outcomes are achievable, significant and clear so that everyone can see a “light at the
end of the tunnel” and follow the light.
• That the resources; human, financial and material are expanded for successful reaching of the
end by achieving the set exit outcomes.
• That there must be explicit relationships between any learning experiences and the ultimate
outcomes to which the experience is essential.
This can be achieved through specifying clear and achievable outcomes, developing performance
indicators which can “light up” each time an objective is achieved. The process should also
include designing learning experience that could determine appropriate instructional strategies
which could result in a 100% achievement.

14
Management should not be “blinded” by the 100% result, unless they are convinced that such
results are being achieved through hardworking and highly motivated staff, use of appropriate
resources, best strategies designed and best performance indicators were determined for each
objective/activity, and that the activities being run are directed towards achieving 100% result.
Otherwise, it can be reasonably concluded that “underhand methods” could also be used to
achieve a 100% result.
Therefore, management should not only focus on results but go beyond and assess the impact
which gives long-term effect. Impact is the ultimate consequence or effects of a programme or
intervention. For example, a result of 100% pass was achieved by an engineering class. We need
to ask ourselves, what change or effect is there on each of these participants? Was this change a
result of the programme? Is this a short-term (learning), medium-term (action) or long-term
(conditions) change?
It is important that management takes a closer interest to study and analyse their vision in
relation to objectives set, activities being undertaken, outputs and outcomes, and ultimately the
impact of the programmes on the former students. But outcomes (long-term results, impact) are
the most important of all.

1.11. Revision Exercise


1). Define
o Monitoring
o Evaluation
2) What is:
o Input?
o Output?
o Outcome?
3) What are the main purposes of:
o Monitoring?
o Evaluation?
4) What is the difference between methods and methodologies?
5) Name 3 basic principles of a results based management.

15
1.12. Suggested Answers to Module 1 Revision Exercise
• Monitoring is a routine and systematic method concerning a programme ,mainly for the sole
purpose of determining its progress against set plans.
• Evaluation is a process of judging the value of what a programme has achieved against the
set objectives and planned activities.
2
i) Inputs are the resources which go into a programme in order to produce the desired results.
ii) Outputs are the results generated by the programme.
iii) Outcomes are the expected or unexpected changes that may come about due to a particular
intervention.
3
i) The essence of monitoring is to control. The main purpose is to improve the implementation
and management of programmes. It is also meant to ensure that educational settings lead to
effective and efficient learning experiences. Monitoring invokes critical thinking and allows one
to learn problem solving techniques.
ii) The main purpose of evaluation is to determine the value or worth of a programme by
ensuring that it is achieving what was intended to achieve. Evaluation can help streamline
service delivery to target groups, leading to increased efficiency and providing an opportunity for
many people to benefit from it.
4
Methods are specific tools and techniques used for data collection and information exchange.
Methodologies are the means to obtain, analyse and exchange information about a particular
issue.
5.
The three basic principles of results management are
• That all outcomes are achievable, significant and clear for every one to follow
• That the resources; human, financial and material are expanded for successful reaching of the
end by achieving the set exit outcomes.
• That there must be explicit relationships between any learning experiences and the ultimate
outcomes to which the experience is essential.

16
2. LOGIC MODEL PROGRAMME DEVELOPMENT
2.1. Description
This module will introduce you to the logic model. It will explain its purpose and how it
functions in programme planning and development. The module will show the process of
constructing a logic model and how it can be used in evaluation process.

2.2. Learning Outcomes


At the end of this module, the student should be able to :
• Define and explain a logic model
• State the purpose and explain the functioning of a logic model
• Design a basic logic model
• Link inputs (investments) to results that represent a theory of change.
• Understand the concept of causal connections that underlie logic models through the use of
if-then relationships
• Explain the “theory of change” that underlies our programs and why logic models show the
assumed causal connections that link what we do with our desired results.
• Design a logic model that can be used in programme evaluation

2.3. Brief History of Logic Models


Logic models date back to the 1960s. They came on the scene through Bennett's (1976),
hierarchy, known as The Seven Levels of Evidence. This is an early fore-runner of today's logic
model. But the first publication that used the term “logic model” is usually cited as Evaluation:
Promise and Performance (Wholey, 1979). Nowadays logic models have spread to all sectors of
the economy, including: private sector, public sector, non-profit making sector, education sector,
international arena and evaluation field.
The private sector has experienced some transformation in what is called Total Quality
Management (TQM) and performance measurement. The public sector has not been spared; as
the emphasis now is longer on just activities but on results and linking inputs to results. The non-
profit sector is concerned with improving programs so that they produce valued impacts.
17
Players in the international arena have used many variations of logic models for a long time.
The United States Agency for International Development (USAID) Log Frame developed in the
1970s is a historical precedent to the current logic modeling discourse. Most international donor
agencies now use a form of program logic for planning and evaluation.
Since then, there have been many refinements and variations that have been added to the logic
models. Fine tuning has been necessary in order to improve performance in terms of design,
reduction in processing time and incorporating user friendly language. As a result, there have
been many management frame works that have been developed in recent years. Sometimes, it
becomes difficult to choose which one of them is ‘the right tool for the job’. The following are
some of the examples:
• Organizational assessment tools – these are the tools used to analyse the current situation in
an organization but without prescribing any ‘medicine’ to change it. The tools include SWOT
Analysis, Skills Audit and Customer Surveys.
• Strategic planning tools – these tools are used to identify and prioritize long term results in an
organization and prescribe strategies to achieve those results; the tools include Strategic
Vision and Portfolio Management.
• Programme planning and evaluation tools – these are used for developing details of identified
programmes. The tools used here include Programme logic model, Managing for Results
model and Earned Value Management.
• Performance measurement tools – these are used to measure, monitor the quality, efficiency,
speed, cost and other aspects of a programme. The tools used here include Key performance
indicators and programme assessment rating tool (PART).
• Process improvement tools – these are used to monitor and improve the quality or efficiency
of a work process. The tools used here include total quality management (TQM), Six Sigma
and Organisational Design.
• Process standardization tools – these are tools used for maintaining and reporting on the
process in order to keep them stable. The tools used here include ISO 9000 and Capacity
Maturity Model Integration (CMMI).
As can be seen, the list is not exhaustive. It is therefore important that you ask many questions to
the vendors before deciding on the best tool for your job.

18
One of the most important uses of a logic model is programme planning and evaluation. It makes
planning easier ‘with the end in mind’, that is, you can visualize how the desired outcomes will
be achieved.
Evaluators have played a prominent role in using and developing the logic model. This may be
why the logic model is often called an ‘evaluation framework’. “Planning a course of action,
such as managing a programme or charting a course of policy, generally implies some sort of
logic model” (Millar, Simeone, Carnevale, 2001).
Wholey’s evaluability techniques were developed to determine if a program was ready for
evaluation. This work was a result of evaluators being asked to evaluate impact and finding that
goals and objectives were vague; finding that programmes didn't exist or weren't being
implemented in a way that would achieve the expected results; and seeking new approaches for
measuring causality, (Bickman, 1987).
The word LOGIC means principles of reasoning; reasonable relationship of elements to each
other and forming a whole. The word MODEL means a small object representing another,
usually a larger object; a preliminary pattern serving as a plan; tentative description of a system
or theory that accounts for its own properties.
There are multiple benefits that can be derived from the process of participating in the
development and use of a programme logic model.
• By using a program logic model as part of the evaluation process, staff will be able to
understand better the outcomes; connect the interim outcomes to long-term outcomes; and
link activities and processes to the desired outcomes.
• In short, through the process of creating a programme logic model your thinking about the
programme will clarified, how it was originally intended to work, and what adaptations may
be needed to be made once the programme became operational.
• The programme logic model provides a powerful base from which to conduct ongoing
evaluation of the programme. It outlines how the programme would produce the desired
outcomes. In this way, you can decide more systematically which pieces of the programme to
study in determining whether or not your assumptions were correct.
• A programme logic model helps to focus the evaluation on measuring each set of events in
the model in order to see what happens, what part works, what doesn’t work, and for whom.

19
• You and your evaluation team will be able to discover where the model breaks down or
where it is failing to perform as originally planned.
• Logic model or theory-based evaluation is also an effective approach for evaluating complex
initiatives with intangible outcomes (such as increased community participation) or long-
term outcomes that will not be achieved for several years.
• A programme logic model lays out the interim outcomes and the more measurable outcomes
on the way to long-term and intangible outcomes. As a result, it provides an effective way to
chart the progress of more complex initiatives and make improvements along the way based
on new information.
• There is also an element of value adding in the process of developing a logic model. It helps
participants identify gaps in the activities, expected outcomes, and theoretical assumptions
resulting in changes being made based on consensus building and a logical process rather
than on personalities, politics, or ideology.
• The process of building the model invokes creative thinking and this becomes an important
part of the overall success of the programme. The model itself provides a focal point for
discussion from the lessons learned. It can be used to explain the program to others and to
create a sense of ownership among the stakeholders.
Logic models come in many shapes and sizes, but there are three types of models which are the
most useful.
An outcomes model type displays the inter-relationships of goals and objectives. The emphasis is
on short-term objectives as a way to achieve long-term goals. An outcomes logic model might be
appropriate for programme initiatives aimed at achieving longer-term or intangible, hard-to-
measure outcomes.
Using this type, staff will be able to understand better to evaluate progress and program
successes, and locate gaps and weaknesses in program operations.
An activities model type links the various activities together in a manner that indicates the
process of programme implementation. It means certain activities need to be put in place before
other activities can occur.

20
An activities logic model is appropriate for complex initiatives which involve many layers of
activities and inter-institutional partnerships. It also provides an effective means to document and
benchmark progress as part of the evaluation process.
The third type is the theory model which links theoretical constructions together to explain the
underlying assumptions of the programme. This model is also particularly appropriate for
complex, multi-faceted initiatives that are aimed at impacting multiple target populations (e.g.,
multiple members of a family, whole communities, multiple institutions or community
organizations within a community, etc.).
Often, programme staff will find that they will need to use a hybrid of two or three in order to
achieve intended results.
Logic models can be created in many different ways. The starting point should be the elements
of an existing programme which are then organized into their logical flow.
For a new program that is in the planning phase, the starting point should be the mission and
long-term goals of the programme. The intermediate objectives that lead to those long-term goals
are added to the model, followed by the short-term outcomes that will result from those
intermediate objectives.
An activity logic model can be built in the same way; long-range activities are linked to
intermediate and short-range activities.
The key to building any model is to prepare a working draft that can be refined as the program
develops.
Most of a logic model’s value is in the process of creating, validating, and then modifying the
model. In fact, an effective logic model will be refined and changed many times throughout the
evaluation process as staff and stakeholders learn more about the programme, how and why it
works, and how it will operate.
As you test different pieces of the model, you will discover which activities are working and
which are not. You may also discover that some of your initial assumptions were wrong,
resulting in necessary model revisions to adapt it to current realities.
You will learn from the model and change your programme accordingly; but you will also learn
a great deal from putting the programme into practice, which will inform the model and provide
you with new benchmarks to measure.

21
This iterative evaluation process will ultimately lead to continuous improvements in the program
and in improved understanding of the program and how and why it works (W.K. Kellogg
Foundation Evaluation Handbook, 1998).

2.4. Construction of A Logic Model


A logic model is a simplified programme or intervention which is designed to respond to a given
situation. It shows the logical relationships among the resources (input), the activities that take
place and the benefits or changes which may result. It can be used in programme training,
evaluation, management and communication. It portrays the underlying rationale of the
programme or initiative. (Ranger and Titcomb, 2002).
In its simplest form, a basic model looks like the one below:

A logic model serves as a bridge between where you are and where you want to be. It provides a
clear understanding of the need for an initiative that will link inputs to the activities which are
used to achieve the desired results.
But this can also be expanded to show other details as shown in the figure below:

During implementation, a logic model is used to explain, track and monitor operations,
processes, and can do other functions. It serves both as a management tool as well as a

22
mechanism to monitor compliance to the plan. During evaluation, a logic model is used to
determine when and what to evaluate so that evaluation resources are used effectively and
efficiently. Although the logic model is not an evaluation tool, it is widely used by evaluators in
order to assist them in the process.
The logic model is designed on the fundamental principles of a normal programme. A logic
model displays the relationships among these core elements and brings to attention the
underlying assumptions set within the program’s environment of external factors.
Although programmes are diverse in nature, they all share common elements, after all, they are
all designed in response to a situation. As can be seen in the diagram above, the common shared
elements are inputs, outputs and outcomes.
INPUTS are sometimes referred to as resources or investments. These are the “things” that go
into the programme in order to enable us achieve the desired outputs.
OUTPUTS are the activities which the programme undertakes and are meant to benefit the target
group. From outputs come the outcomes-impact, which are the benefits and the changes that
result.
OUTCOMES can be said to be shorter-term achievements while longer-term achievements are
the impact. Outcomes relate to changes in behaviour, norms, knowledge, attitudes, capacities,
motivations, skills, decision making or other expected results of a programme. In addition,
outcomes play a major role in the operations of a logic model.
We should therefore think of a logic model as a road map which gives you direction from the
beginning to the end.

2.5. Logic Model Formats


There are two formats commonly used in logic models; one is a Table Format and the other is
Flow Chart as shown in the diagrams below:
THE TABLE FORMAT

23
THE FLOW-CHART FORMAT

From the formats given above, the following variations, which are now in common use, have
been developed.

24
The above figure is a Table Format. In such a model, items are listed in the input, output and
outcome columns. In some cases, these may have directional arrows to show the connections and
relationships. The lists may be numbered to show order within a column or to indicate rows of
connections across the columns.

The figure above shows a Circle Format which also shows directional arrows to depict
connections and relationships. Some people use different shapes like trees, footprints etc. to
depict their own programmes.

Some logic models are simple; but others are complex. Some are vertical; others are horizontal
and the figure above has captured both the vertical and horizontal formats.
25
The figure above shows a Flow Chart Format. In such models boxes are used, with lines and
arrows connecting the boxes so as to illustrate the causal linkages; and in some cases, the boxes
may be numbered.

Some logic models use formats which are ‘abbreviated’ and only key components are shown.
Some formats don't include assumptions, situation, or external factors, while others may only
include outputs and outcomes.

2.6. Benefits of Logic Models


There are many benefits that can be derived from development and using a programme logic
model.
• Through working on a programme logic model as part of the evaluation process, staff will be
able to understand better the outcomes; connect interim outcomes to long-term outcomes.
They will be able to link activities and how they are processed to desired outcomes.
• In short, through the process of creating a programme logic model, your thinking about the
programme will be clarified. You will also understand how it was originally intended to
work, and what adaptations may need to be made once the programme becomes operational.
• The programme logic model provides a powerful base from which to conduct ongoing
evaluation of the programme. It will show you how the programme produces the desired
outcomes. In this way, you can decide more systematically which parts of the programme
you need to study in order to determine whether or not your assumptions were correct.
• A programme logic model helps focus the evaluation on measuring each set of events in the
model to see what happens, what works, what doesn’t work, and for whom. You and your
evaluation team will be able to discover where the model breaks down or where it is failing
to perform as originally planned.

26
• Logic model or theory-based evaluation is also an effective approach for evaluating complex
initiatives with insignificant outcomes (such as increased community participation) or long-
term outcomes that will not be achieved for several years.
• There is value in the process of developing a logic model. The process is an iterative one that
requires stakeholders to work together to clarify the underlying rationale for the programme
and the conditions under which success is most likely to be achieved.
• Gaps in activities, expected outcomes, and theoretical assumptions can be identified,
resulting in changes being made based on consensus-building and a logical process rather
than on personalities, politics, or ideology.
• The clarity of thinking that occurs from the process of building the model becomes an
important part of the overall success of the programme. The model itself provides a focal
point for discussion. It can be used to explain the programme to others and to create a sense
of ownership among the stakeholders.
As the name suggests, a logic model is a model of logical connection. It shows the relationships
and theory of action among various components of the programme. The power of logic modeling
lies in articulating the linkages which provide chain relationships in the programmes; all
designed to ensure that positive benefits are achieved.
Therefore, logic models are designed to help you understand how a programme works. Logic
models come in as many sizes and shapes as the programmes they represent (Kellogg
Foundation, 2001). Some models may show various arrows pointing different ways; some show
vertical, horizontal and circular directions. Since models make explicit the underlying theory of a
programme, they may not be easy to ‘grasp at first sight’.
There are many variations of logic models. But the most important thing is to show all logical
connections in the context of an originating situation, and to ‘make implicit understandings
explicit’ (MacLaughlin and Jordan, 1999).
Logic models are made up of six components, namely; situation, inputs, outputs, outcomes,
assumptions and external factors, and these are marked (1) to (6) in the diagram shown below:

27
2.7. Situation
This is the foundation of a logic model development. In order to understand the situation, it is
important to ask oneself the following questions: What is the problem we want to solve? What
are the causes of that problem? Whose problem is it? Who is to benefit once a solution has been
found? Is there someone around here with enough expertise and experience to give us a solution
to this problem?
Answers to these questions would give you a basis from which to start tackling the problem.
This basis is what is called a SITUATION.
From the situation you can create a statement, and that statement would be the foundation of
your logic model, usually this statement will be attached to your model. Since the situation is the
base or foundation of everything else that follows, it is important that we understand the problem
fully. This will make it easier when you start developing your own model.
The situation is sometimes referred to as the originating problem, or issue set within a complex
of socio-political, environmental and economic circumstances. And this makes the situation as
the starting point of logic model development.
Once you have dealt with the situation, you should now set your PRIORITIES. But before
setting your priorities, you need to formulate the criteria for setting them. You must remember
28
that priorities will enable you identify your desired outcomes. But in order to determine your
focus, you will need to consider several factors which will include: your aim, the purposes of
doing this, available resources and anything else which will help you carry out your task as
desired. Organizations that have a clear understanding of their mission, vision, values, goals, and
program strategies to attain their expected outcomes, have usually performed better.
INPUTS
These are the resources that go into the programme in order to bring about outputs. Basically,
these are what go into the programme: resources and contributions that are invested. When we
are talking about inputs we are including such elements as staff, money, time, equipment,
partnerships, and the research base.
OUTPUTS
These are the activities which a program undertakes. They are the activities we do, services we
offer, functions or events we organize, courses we run, products we make and the people we
target as customers or clients. Outputs are intended to bring about specific outcomes. In some
designs, activities and outputs are separated, and in such cases, it is assumed that the activities
will be delivered before the expected outcomes can occur. Outputs may include such activities as
workshops, conferences, counseling sessions, products we make and the individuals, clients,
groups, families, and organizations targeted to be reached by these activities.
OUTCOMES
These are the changes or benefits that result from our programme activities. They are also
referred to as direct results or benefits for the target group (s). Outcomes may be positive or
negative, intended or unintended. Apart from being short-term, as in learning, skills acquisition,
attitude change, outcomes can also be medium-term as in skill practice, behavioural change,
decision making. Outcomes include short-term benefits such as changes in awareness,
knowledge, skills, attitudes, opinions and intent. Outcomes include medium-term benefits such as
changes in behaviors, decision-making and actions. “What gets measured gets done” (Osborne
and Gaebler, 1992).
This assertion was supported by Osborne and Gaebler (1992) who stated
that ‘if you don’t measure results, you can’t tell success from failure. If
you can’t see success, you can’t reward it. If you can’t reward success,
you’re probably rewarding failure. If you can’t see success, you can’t
29
learn from it. If you can’t recognize failure, you can’t correct it. If you can
demonstrate results, you can win public support’.
Impact is the long-term outcome or the goal. It is the ultimate consequence or effect of the
programme. In the diagram above, it refers to the long-term changes in social, economic or
environmental conditions. Outcomes include long-term benefits (often called impact) such as
changes in social, economic, civic, and environmental conditions.

2.8. Assumptions
These are the beliefs we have about the program, the target group and how we think the program
will operate when it is implemented. These are also known as principles, beliefs and ideas we
have about a programme.
In short, it is the theory of our programme. Assumptions underline and influence the decisions
we make about a programme.
When making assumptions, we should consider the problems at hand, assess the available
resources (including human), how the programme will operate and what we expect to achieve,
including the examination of the internal and external environment.
Assumptions include the ideas we have about the problem or situation at hand; how the
programme will operate; what we expect to achieve from it; how and what the participants will
learn, their behaviour, their motivations, etc.; including the resources and staff; their knowledge
base and experience; the external and internal environment environment. Faulty assumptions are
often the reason for poor results.
If a programme fails to produce good results, it may be due to inaccuracy in stating the
assumptions or some assumptions were just overlooked. So, although it is important to analyse
and explore all assumptions relating to the problem or situation for which we are trying to
provide the solution, it is equally important to depend on what we know about the problem
before listing what we are assuming.
EXTERNAL FACTORS
There are many external factors that can influence or affect the success of your programme. Such
factors may include cultural beliefs, climatic, economic, political and environmental conditions.
Experience of participants may be a key factor when it comes to achieving desired results.

30
External factors should not be ignored, because they are likely to affect programme
implementation, especially when it comes to the flow of events and the long-term outcomes.
The external factors are aspects that are external to the programme but are able to influence the
way the program operates, and are also influenced by the program. Dynamic systems interactions
include the cultural milieu, biophysical environment, economic structure, housing patterns,
demographic makeup, family circumstances, values, political environment, background and
experiences of participants, media, policies and priorities, etc. There is little control over
elements that affect the programme.
IF-THEN RELATIONSHIPS
Many people who have used logic models talk about them as a series of “if-then” sequences. If
X, then Y and If Y, then Z.
This sequence can be illustrated in the following diagram. When you read from left to right, a
logic model portrays a series of if-then relationships.

Let us examine the figure above, starting from the left side and see how this works:
IF you invest certain resources in a programme, THEN you will be able to provide activities or
services or products for target groups. IF you educate or train or sell some products to those
groups, THEN they will benefit in some specified ways in the short term.
IF the short-term benefits are achieved to the extent expected, THEN the medium-term benefits
can be accomplished too.
IF the medium-term benefits for participants, organizations, and decision-makers are achieved as
expected, THEN you would expect the longer-term improvements and final impact in terms of
social, economic, environmental, or civic changes to occur.
This explanation is the foundation of logic models and the theory of causal association is based
on this logic. In many complex programmes, the ‘if – then’ relationships are too simple and
linear to be applied. In such cases, complicated relationships may be used.
31
However, the more we work with these sequences, the more we uncover gaps in logic, clarify
assumptions, and more importantly we do understand how investments are likely to lead to
expected results.
If we are involved in research work, then the ‘if-then’ relationships would become clear and
strong. Often, however, we work in situations, and with issues and people, whose research base
is not well developed.
This theory is intended to offer some logical explanation that links programme inputs with
activities leading to outcomes, and therefore creating a chain of response – that leads to ultimate,
end results. However, when developing a logic model, you need to bring out the underlying
assumptions. Are they realistic and sound? What evidence or research supports your
assumptions?
Now let us look at one example of the ‘if-then’ relationship. You should check the assumptions
given but there could be more. It is important that no assumptions are ignored as this may affect
the expected result.

2.9. A Community Support Initiative


This is an example to illustrate and apply the ‘if-then’ relationship to some common situation.
The example is a project of constructing a medical centre, for the sole use by the local
community. You need to pay particular attention to the assumptions made.

IF money, expertise and time are invested in the programme, THEN a medical centre can be
constructed.
IF a medical centre is constructed, THEN the community will be aware of that facility and hence
the services available.
32
IF community knows, THEN they will be able to access the appropriate services in order to meet
their medical needs.
IF community can access the appropriate services, THEN their needs will be met.
This initiative has made certain assumptions, as can be listed below:
• That the medical centre that will be constructed will be linked to improvement of the quality
of life of the community.
• That there will be enough money, expertise and time to invest in the construction of a
medical centre.
• That the community, especially the target group will have access to the medical centre once it
becomes operational.
• That the medical centre will in fact meet the needs of the community in general, and the
target group in particular, once they start accessing its service.
• That interagency coordination will make a difference relative to the needs of the community.

2.10. Community Nutrition Initiative


This second example is about an initiative that a local community decided to embark on in an
effort to improve the nutrition status of the people who live in the community. This is illustrated
in the following diagram below:

IF we have resources, expertise, time and access to the target group, THEN we can provide
culturally-appropriate nutrition information about available community services, the importance
of nutrition and prepare foods using locally available foodstuffs.

33
IF we provide culturally-appropriate nutrition information about community services, food
preparation and the relationship between nutrition and health, THEN community will better
understand the importance of good nutrition and they will have increased knowledge about how
to obtain and use food resources available in their community.
IF recipients better understand the relationship between nutrition and health and the need to have
better access to nutritious food through the use of community services, THEN communities
make use of available services and prepare more nutritious foods regularly.
IF recipients use available services and prepare more nutritious food more regularly, THEN
communities will have balanced meals and hence improved nutritional status.
We can also use the expression: “BUT WHY?______, “SO THAT …. something will occur,”
instead of the IF ______ THEN.
And this can be illustrated in the following example:
First, we must establish the NEED or a SITUATION.
“I want to get a job”.
“BUT WHY?”
“SO THAT I can buy a house”.
“BUT WHY?”
“SO THAT I can live in it”.
“BUT WHY?”
“SO THAT I can be assured of safety and security”.
So continuing with this will create a chain of connections that will link your programme
activities to your desired end results. But if your ultimate goal is to have safety and security
assured, then that will be your end, but if it is something else, then you will need to continue till
you reach your desired goal.
LINES AND ARROWS IN A LOGIC MODEL
In the logic model, the lines and arrows are called linkages; and these are what give the model its
power. Lines and directional arrows are meant to show these linkages in what is called theory of
action. All lines and arrows may be included or may be abbreviated and this will be applied as
the case may be.

34
The flows may be vertical (upward or downward) and horizontal, one-direction or
twodirectional, and show feedback loops The feedback arrows (like the one shown at the far
right of the figure below) often depict learning and modifications made, or envisaged, during the
course of program implementation.

The lines in the diagram above look messy but they are necessary and you need to take time to
draw them. It is the arrows and lines that would create logical connections and create direct links
between inputs, outputs and the desired outcomes. In the final display, we may only include the
primary linkages; otherwise, the logic model may become too difficult to read.
The final outcome theoretically links back to the beginning to make a difference, and create “an
impact,” on the originating situation. The large feedback arrow at the top right of our logic model
is an attempt to illustrate this connection and the dynamics of programming. Some people like to
depict a logic model as a circle that explicitly connects the end to the beginning.
In actuality, program environments are dynamic and situations change so the beginning rarely
stays the same (Taylor-Powell and Henert, 2008).

2.11. Cause-Effect Relationships or “Causation”


The idea of cause-effect or causation is central to the logic model. The logic model depicts many
assumed causal connections of a programme. Yet, cause-effect relationships usually cause
problems in the context of community programming, but experience has shown that:

35
• In most cases, programmes have only a partial influence over the results. External factors
which are beyond the program's control can influence the flow of events, and this particularly
applies to longer-term outcomes.
• Many factors that affect the development and implementation of community initiatives make
it difficult to make logical causal connections. The external environment affects and is
affected by the programme. Many factors may come into play before, during, and after
program implementation in an almost constant dynamic of influences.
• It is quite rare to find only “one” cause as there are many multiple cause-effect chains that
are likely to interact.
• Short project time lines make it difficult to document the assumed causal connections.
• Measuring causal relationships and controlling for contextual factors through experimental or
semi-experimental designs is often not feasible and very expensive.
• Since data is usually collected through various methods (quantitative and qualitative), they
often show different and sometimes contradictory causal associations. So can make it rather
difficult to “prove” whether a particular outcome is the result of a particular intervention.
• Causal relationships are not as simple and clear as it is in the ‘if-then’ relationships example
above suggests. However, there are multiple and interacting relationships that affect change,
often functioning as feedback loops with the possibility of delays (Rogers, 2000).
Systems theory suggests a dynamic and circular approach to understanding causal relationships
rather than a uni-dimensional linear approach. Logic models can be created to depict these more
iterative causal mechanisms and relationships by adding feedback loops and two-way arrows,
narrative explanations, or a matrix. Limitations are imposed by the necessity of communicating
on paper in a two-dimensional space.
Remember, the logic model is a “model” – not reality. It depicts assumed causal connections, not
true cause-effect relationships. However, even simple models are very useful. They can help
clarify expected linkages, bring out underlying assumptions, focus on principles to test, educate
funders and policy makers, and move a program into action and learning.
PROCEDURE FOR DEVELOPING A LOGIC MODEL
STEP 1: PURPOSE AND USE
o State the problem which you are trying to solve using a logic model.
o State the advantage of solving such a problem using a logic model.
36
o State how the model will be used.
o State who will use the model.
STEP 2: INVOLVE OTHERS.
o Identify who should participate in creating the logic model.
o Identify who should facilitate the logic model development process.
STEP 3: SET THE BOUNDARIES FOR THE LOGIC MODEL.
o State the boundaries of the model. Will it be
• a single model,
• focused endeavor;
• a comprehensive initiative;
• a group process;
• or organizational endeavor?
o Identify the level of detail needed.
STEP 4: UNDERSTAND THE SITUATION.
o What is the problem at hand? What situation has given rise to this programme?
o Is the logic model the best method to solve this problem?
STEP 5: EXPLORE THE RESEARCH, KNOWLEDGE BASE.
o What do we know about the problem or context?
o Identify the factors that can make the programme fail to succeed.
STEP 6: ASSUMPTIONS
o List all the possible assumptions you can think of that the programme may assume.
STEP 7: EXTERNAL FACTORS
o External factors should not be ignored, because they are likely to affect programme
implementation, especially when it comes to the flow of events and the long-term
outcomes.
STEP 8: THEORY OF ACTION
o You must theorise how the programme will work, from inputs through outputs to
outcomes.
o In order to illustrate the program theory, the logic model needs to show all the logical
linkages between and among elements.

37
o You should remember that deciding on a single image that displays the program
theory is often the most difficult part of developing and using a logic model.
STEP 9: DRAWING THE MODEL
o Draw your logic model, showing all the linkages logically. A logic model makes the
connections EXPLICIT. Logic models may be simple or detailed and complex
depending upon your purpose and use.
o Use a graphic presentation that best fits the use and user.
o A common problem is that activities and strategies often do not lead to the desired
outcomes. Check your ‘if-then’ statements and ensure that they make sense and lead
to the outcomes you want to achieve.
Remember that the logic model is just a MODEL. The programme should not be made too
simple, as by so doing you are likely to miss out important information. It is important that the
model is clear and understood by those who will use it, be it yourself, your workers, your
sponsors or other persons.
The second approach is to work BACKWARDS as illustrated in the following example:
Step 1. Start at the end and ask yourself a question; “what is my long-term desired outcome”?
Step 2. Move one step backwards and identify the chain of outcomes that lead to the final, long-
term result.
Step 3. Move one step again backwards, and ask yourself another question; “who is/are going to
participate? Who is/are expected to achieve the expected outcomes?
Step 4. Move backwards and ask yourself again; “What ACTIVITIES must be provided and
completed so that the identified target individuals (or groups) will achieve the desired outcomes?
If necessary, you may need to group your activities into strategies (activities that fit together
conceptually) such as training, media work, development activities etc.
Step 5. Move backwards once more and ask yourself a question; “What RESOURCES are needed
to be put in place so as to make sure they are accomplished by the target group (s)?
It is important that you read the following hints on order to help you understand the process very
well:
1. You are advised to work in small groups which are highly interactive, hard working and
committed to the cause.
2. It is better to use appropriate examples to introduce ideas.
38
3. Do not use terminology that may be difficult for you and others to understand, use simple
words that every one can understand.
4. You are encouraged to build the model on the premise of shared vision so as to maximize
on the value and use of the model.
5. Do not just produce a model on the first attempt. You may need to revise it until it is fully
refined, and reflects your description of the programme.
6. Sometimes you may need a mentor or expert to check your work in order to identify the
strengths and weaknesses of the underlying programme theory and assumptions.
7. It should be recognized that logic model is not easy to develop. A logic model is a
framework for describing the relationships between investments, activities, and results. It
provides a common approach for integrating planning, implementation, evaluation and
reporting.
Since logic models can also be used for program evaluation, it is important we dwell on this
important subject.
Therefore, programme evaluation is the systematic collection of information about the activities,
characteristics and outcomes of programs in order to make judgements about the program,
improve program effectiveness, and/or inform decisions about future programming (Patton,
1997).
Now, let us consider the logic model as a tool for evaluation and the common types of
evaluation. We can link the evaluation process to the usual procedure which is normally applied
during programme evaluation. Here, we should link the process to the stages that are considered
when carrying out an evaluation of a programme.
Now, we need to review the evaluation stages so as to be able to formulate the evaluation
questions. In the following diagram, we have linked the logic model progression of events to the
evaluation process:

39
The first stage is to assess the situation or the need as it was originally stated at the beginning.
So we can take the situation (mentioned above) as the NEED of a programme.
When it comes to evaluation, we must ask ourselves leading questions, as thus:

• What were the characteristics, needs, priorities of target population? What were potential
barriers/facilitators?
• What this programme the most appropriate thing to do in order to solve that problem?
Answers to these questions will help us establish the need which we are trying to achieve.
The second stage is to conduct the PROCESS EVALUATION of the programme. So the inputs
and outputs of the logic model will be considered as the process of programme implementation.
Here, we should ask ourselves leading questions, as thus:
• How was the programme being implemented?
• Were the activities delivered as intended?
• What strategies were used for programme implementation?
• Were the participants being reached as intended?
• What were participants’ reactions?
The third stage is to conduct an OUTCOME EVALUATION of the programme. Again we need
to ask ourselves questions, as thus:
• To what extent are the desired changes occurring?
• Were the programme goals met?
• Who was benefiting or not benefiting from this programme?
• In which way did they benefit?
• Which parts of the programme went as planned?
40
• In which ways did the programme face the hurdles?
• Did the programme bring out any intended outcomes?
• Were there any unintended outcomes? If yes, then state them.
The fourth stage is to carry out what is called IMPACT EVALUATION of the programme. In
the logic model, outcomes are stated as Outcomes – Impact, but these are the outcomes of a
programme.
Here again, we should ask ourselves some leading evaluation questions, as thus:
• To what extent can that the changes be a result of the programme?
• What are the net effects of the programme?
• What are final consequences of the programme?
• Was the programme worth resources invested in it?
Largely, answers to these questions will form a basis for your evaluation, and will help you
formulate an evaluation plan.

41
2.12. Revision Exercise
1) What is a logic model and what purpose does it serve?
2) Explain the following terms:
i. Assumption
ii. Extended factors
iii. Situation
3) A certain community had a vision of eradicating illiteracy among its people. It therefore
engaged literacy teachers to carry out the job.
Using this information, develop a logic model that will embrace this goal, and showing all the
connections using the ‘If – Then’ relationships, and write out a narrative of the same.
4) Study the table below and state which of the parameters each of the following programme
components relate to. Make a tick in the boxes provided.

42
5. Are there any limitations in using a logic model – anything we should be cautious about?

2.13. Suggested Answers to Module 2 Revision Exercise


1. A logic model is a depiction of a programme showing what the programme will do and what it
will accomplish. It is the core of programme planning and evaluation.
It is a framework for describing the relationships between investments, activities and results, and
it provides a common approach for integrating planning, implementation, evaluation and
reporting.
2.
a) ASSUMPTIONS are the beliefs we have about the programme, the people involved, and how
we think the programme will work. Assumptions include our ideas about the problem or
situation; the way the programme will operate; what the programme expects to achieve; how the

43
participants learn and behave, their motivations, including the internal and the external
environment.
b) EXTERNAL FACTORS are the aspects external to the programme that influence the way the
programme operates, and are influenced by the programme and over which there is little control.
c) SITUATION is the originating problem, or issue, set within a complex of sociopolitical,
environmental and economic circumstances. The situation is the beginning point of logic model
development.
3.

IF we have resources, expertise, time and access to the target group, THEN we can provide
literacy services.
IF we provide the service, THEN community will better understand the importance and the need
for that service.
IF recipients better understand the need and importance of literacy services, THEN they will
access the services
IF recipients use the services THEN illiteracy would be reduced and consequently the need
would have been met.
4. The answers are given in the following table”

44
5. Yes. The most common limitations include:
 A logic model represents intention, it is not reality;
 It focuses on expected outcomes so people may overlook unintended outcomes, whether
positive or negative
 It focuses on positive change; but change isn’t always positive;
 It may simplify the complex nature of causal attribution, and yet there are so many
factors which may influence process and outcomes.
 When creating a logic model, you are not sure whether the programme you are working
on is the right one to do.
 May stifle creativity and spontaneity.

45
3. MONITORING AND EVALUATION: SYSTEMS/WORK PLANS
3.1. Description
This module aims at providing students with an overview of monitoring and evaluation
systems/work plans. It will also define indicators, state their purpose and how to identify
effective indicators.

3.2. Learning Outcomes


At the end of this module, the student should be able to:
i. Explain and develop monitoring and evaluation systems
ii. Design monitoring and evaluation work plans
iii. Define, state the types and purpose of indicators
iv. Identify effective indicators

3.3. Introduction
A plan can be said to be dynamic because we expect it to change. A complete plan will clearly
state:
• The goal of the project.
• The objectives that must be met to accomplish the goal.
• The tasks that need to be done.
• Why those tasks are necessary.
• Who is responsible for what tasks?
• When the project will be completed.
• What resources will be needed.
• What criteria should be met for the project to be declared complete and successful.
• Are there any assumptions, risks or obstacles that may affect the successful completion of the
project?
All these questions need to be answered at the planning stage. Planning has its own benefits.
• It reduces uncertainty in that it gives us the opportunity to determine the likely outcomes and
therefore put the necessary corrective measures in place.
• It increases our understanding of the goals and objectives through taking part in the actual
exercise.
46
• It improves efficiency by maximizing the use of resources and completes the work in time.
• It provides a basis for measuring work planned against work done.
• It is not only a road map showing how work will be done but it is also a tool for decision
making.
Planning is a process of discovery – discovery about a project and those hidden perils that can
cause embarrassment to the team. So all assumptions, risks and obstacles to the successful
implementation of the programme/project must be listed and discussed in full and if possible
offer solutions. There are several factors that may inhibit project success, and these may include;
• The technological advancement which is advancing at a faster rate and so should be your
organization and your personnel too. With the business world changing and refining itself
continuously, opportunities for new or enhanced products and services present themselves
constantly. Organizations must be able to take advantage of them quickly because the
window of opportunity is not wide and is itself constantly moving.
• Working environment needs to be modernized to match the changing times.
• Change in management may bring failure of a project. A high priority project may end up to
be a low priority project overnight, if the new management does not see it that way.
• A personal relationship between project team members is critical to the project’s success.
You do not have to be friends but team mates. Interpersonal relationships should be dealt
with immediately before differences affect a good project.
• We all like to think that what we are proposing will correct the situation being addressed. So
assumptions about cause-and-effect relationship need to be addressed.
Traditional project management methods are very robust. They can be applied to a variety of
situations, regardless of the application, the same steps do apply. There are so many recognized
similarities between the traditional management methods life cycle and the systems development
life cycle. Many organizations that claim to be practicing project management have basically
adopted a pseudo-project methodology.
Traditionally, a life cycle of a project starts from an idea which is then consolidated into a
project/programme after a lengthy session of pros and cons. So for anyone to be able to carry out
formal monitoring or evaluation he must be able to understand the life cycle of a
project/programme.

47
The table below shows the project management life cycle and the typical systems development
life cycle. On the left is a project cycle and on the right is the systems cycle that must be
developed to be able to manage the cycle.
The idea of having a systems cycle is meant to create a system of checks and balances so that all
activities are taken care of and nothing is left to chance, and also to avoid significant deviations
that will escalate the overall costs.

48
Whichever way you look at it, the evaluator must prepare adequately before embarking on the
job of monitoring/evaluation. The monitor or the evaluator must have the following as a check
list:
• Decide on the purpose of monitoring/evaluation
• Determine what should be monitored/evaluated

49
• Clarify you target and standards
• Decide on how you will gather data
• Prepare resources needed for the task
• Design data collection instruments
• Collect and analyse data
• Identify areas for change or development
• Draw up an action plan
• Implement the action plan
• Review the action plan
• Write a report
Now and again, we hear people saying,” The event was a success, the wedding was a success,
the function was a success or indeed a programme has been successful.” But how do we measure
success? If some but not all objectives were achieved, would you call that event a success? Or
would you attribute success to one aspect without considering other things?
You can measure success through beacons called indicators –these are the benchmarks intended
to measure the performance of a programme. The indicators, which are usually expressed as a
percentage, index or ratio, must be monitored at regular intervals.
Now let us consider the following case study.
One community held a function under the theme: “Marketing and promotion.” This was a key
result area in their strategic plan. Since this was an important project, specific objectives were
laid, which included:-
• Income generation for the programme
• Raising the profile of the area
• Increasing returns for the local businesses
At the end, the function was declared a success because it attracted 1466 people as compared to
1387 last year – an increase of 79. But would you call this function a success; just by the increase
in number by 79? It was not until the event was evaluated that the true picture was known as
shown in the table below:

50
Now looking at the whole picture there is no doubt that the function was a partial success. Every
indicator showed some benefit in one way or another.
The case above illustrates and underscores the need and importance of indicators. It is therefore
cardinal that once indicators have been identified they should be embedded in the programme at
the planning stage. The indicators are the evidence for success; otherwise a programme can not
be declared a success all the beacons ‘light up’.
Now let us analyse each indicator, one by one. One objective was to increase income generation.
After the show, there was a little increase in both profits and revenue, and this represented K10m
and K5m respectively. These figures may have been outstripped by the expenses incurred during
the preparation of the event. A careful analysis of the two figures above showed that in terms of
income generation, the event was a disaster. But the organizers failed to visualise which
activities would have brought more revenue.
On the second objective of raising profile of the area, there was a noted awareness amongst the
visitors due to the increase in the numbers of those who attended the event.
Surely the number 79 was a negligible figure when we look at the number that attended the
function last year – 1387. Therefore, the marketing strategy which organisers put was ineffective
and most probably immeasurable and imaginary. On the last objective of increased returns for
51
the local businesses was also a disaster to say the least. Although there was an increase in the
turnover but that only translated to 10% increase in profit. Again the indicator does not show a
good picture. It seems even the businesses also failed to live up to the occasion!!!
The overall picture was a clear disaster although there were traces of success here and there.
An event can be adversely affected by weather and be declared a failure. But such failure could
not be attributed to anyone except to ‘some situation beyond the control of the organisers.’ Such
factors include:
• Political decisions and policies – pronouncements by politicians may derail a well organized
function. This is also true when there is a change in government policies.
• Poor economic climate – this can have an adverse effect on the targets set by the organisers.
Therefore there should always be a balance between the economic prowess of the community
and the charges being demanded by the event organisers.
• We can conclude that the organisers might have ‘priced themselves out of business’.
• Competitors and critics of your programme – there is need to analyse the potential of your
competitors. Competitors should not be given chance to turn your golden opportunities into
total misery. You should always strive to live above them. But if given chance, then learn
from them, but they should not be ignored.
• Adverse weather conditions – You should always follow the natural climatic patterns before
announcing the date of the function.
• Natural condition like rain could easily disrupt a well planned wedding. So before due date,
an announcement to postpone should be made rather than risk losing half the number of
invited guests.
• Collapse of a major industry in a town- This can not be ignored because it means workers
will now be turned into loafers. And this would spell doom on the town.
• Consequently, there will be no money for merry making as the little cash available could be
saved for “family use.”
When evaluating a programme, one needs to go to the original purpose i.e. its intended
objectives and find out whether the same were accomplished successfully. The indicators are the
evidence or signals for success or failure. If the indicators can not point to achieving of
objectives then something would have been wrong with the set objectives. Objectives are not set

52
in vacuum; they must bring out achievable indicators. An objective statement should contain four
parts:
• An outcome which is a statement of what is to be accomplished.
• A time frame which is the expected date of completion.
• A measure which is an indicator that will measure success.
• An action is how the objective will be met.
The purpose of objective statement is to clarify the exact boundaries of the goal statement and
define the boundaries of your project. So every objective must be one that leads to achieving that
goal. It is because of this, that every objective must have the following characteristics as stated
by Doran (1981):
• S – be specific in targeting an objective
• M – establish measurable indicators of progress
• A – assign the task to one person for completion
• R – state what can be realistically be done with available resources
• T – state when the objective can be achieved ie duration
Performance indicators are the signs you check for to see how well your project is doing.
Indicators are the signals that are used for simplifying, measuring and communicating important
information (New economic Foundation, 1997), and they reflect changes that occur as a result of
particular intervention.
Such indicators are usually referred to as performance indicators as they are meant to indicate the
way a programme or other interventions are performing. They are many types of indicators;
namely- impact, process, outcome (or impact) indicators.
• Input indicators – are concerned with resources that go into the project/programme including
the activities associated with it.
• Process indicators- are the signals that monitor achievements during implementation and
measure how resources are delivered.
• Output indicators- measure intermediate results, for example, at a point when donor
involvement in the project is close to complete.
• Outcome indicators- measure long-term results of the project and after donor involvement is
complete (Walters et al, 1995).

53
As can be seen, outcomes are the expected or unexpected changes or impacts that result from a
particular intervention. But outcome indicators will help you to measure whether you are really
bringing about the change you wanted. So when formulating activities, you need to breakdown
your aims into all the different changes or benefits that you hope should take place in order to
achieve the objectives. These changes are known as outcomes.
When you have identified the outcomes you want, you can then identify the indicators for each
outcome.
These are the ones which will signal to you if any changes have occurred. In order to achieve
this, you can always ask yourself the following:-
• What changes (outcomes) do we want to see in our target group?
• What signs (outcome indicators) will show us that the changes we planned to see have
actually happened?
But there is need to limit the number of indicators as too many of them could confuse you.
You are therefore advised to concentrate on those which are achievable and will help you in the
evaluation process. But how will these indicators be checked? By whom? These questions shall
be answered as we start looking at the work plan – a plan to ensure work is going on as was
planned.
As the name suggests, a work plan is a plan outlining the activities in the order that they will be
executed and individuals to carry out such tasks. It is a plan spelling out specific roles of each
individual and when such tasks would be carried out. Since all activities in the work plan are
necessary, they should all be monitored. Objectives and indicators can be used to identify major
activities.
For example, in the case of constructing a community school the following indicators can be
used:
• Mobilising the community for their labour.
• Mobilising tools and implements for use in the actual construction.
• Collecting sand from the source
• Fetching water for making bricks
*It is assumed that building cement is already available.

54
Each of the tasks above will be assigned to an individual to undertake. They are all important
functions but they should all be monitored by a group of “inspectors”. The monitors’ role is to
measure how far they have reached in achieving the objectives. Their role involves assessing the
quality of work to ensure it is high quality. It is advisable that the team should also hire the
services of a local artisan who can assess the quality of the project, as this will cut on costs.
For a community project, it is advisable that monitoring visits be carried out at least once a week,
in order to avoid big deviations from the work plan. Although monitors are like observers, they
should take time to talk to all those who are involved in the construction of the project. You will
be amazed at the amount of information you get from the labourers.
It is a known fact that organizers will not be able to give much information because of fear of
losing jobs or indeed loss of continued funding of their projects; so they may choose to reveal
what is considered favourable information. At every level there should be an implementation
committee that will be charged with the task of implementing the findings of the monitors.
A work plan is a critical document in the conduct of an evaluation. Like all plans, you need to
take time to review them before settling on the final workable plan. The following steps should
be taken when formulating a work plan:
• Step 1 is the review stage. All evaluation questions should be reviewed and grouped in some
logical order. This can be done by subject area, data needed to address those areas or using
other methods considered suitable for the exercise. In short, you will be formulating the
overall evaluation strategy and developing the necessary evaluation instruments.
• Step 2 is for data collection process. This can be done by way of questionnaires, interviews,
observations and scrutinising relevant existing documents. Since evaluations take more time
than anticipated, it will be necessary to formulate a realistic schedule accompanied by a
bloated budget that could take care of ‘unforeseen circumstances’. It is easier to take back
extra or unused funds than to ask for additional funds to cater for the under-estimated need. It
is not good to expect that everything will run as planned without any hitches. For example,
questionnaire responses may be incomplete, and therefore additional follow up may be
necessary. So contingencies are needed in every case than “put all information ‘eggs’ in one
data collection ‘basket’. Imagine a well planned survey that could be spoiled by a heavy

55
rainfall on the day of data collection!!! It is useful to plan an evaluation with some
contingencies or alternatives in mind.
• Step 3 is for data coding. The task is to ensure that data collected is accurate and relevant to
the task at hand. Data is then processed into measurable formats for analysis. This is
necessary to make it easy for interpreting.
• Step 4 is for data analysis. At this stage the major tasks are: conducting statistical analysis
related to evaluation hypotheses, preparing summary statistics, chart, tables and graphs.
By and large, the evaluators have the code of ethics, called evaluation standards at their disposal.
In some cases, they must engage tact and diplomacy in order to obtain any evidence, especially
when questions are asked concerning human behaviour or where parental consent must be
sought.
Sometimes, existing data may not be kept in usable format, so it will need to be “worked on” in
order to yield meaningful results. If this was known early then evaluators would plan for
extended time and funds for such extra tasks to be accomplished.
Evaluation is a very important activity that it should not be left to “amateurs” to undertake. It
needs professionals who can use their technical know- how and experience to identify the
constraints or bottlenecks that hinder the project in achieving it objectives. It is through
evaluation that costs and benefits that accrue to the intended beneficiaries of the project can be
accessed.
Therefore, evaluation must be done BEFORE, DURING and AFTER the project has been
implemented.
BEFORE project implementation, evaluation is needed so as to assess the possible consequences
and / or benefits that the planned project may bring to the affected people. This is also the time
when a decision can be made on the alternative project strategies that may be implemented in
case of failure due to unforeseen circumstances.
Before implementation, decision makers have an opportunity to discuss and decide how the
project will be implemented. Therefore, all the possible or foreseeable constraints must be
discussed and solutions found. It would be difficult to discuss anything once the project is set in
motion.

56
DURING project implementation, evaluation should take place in all the areas of the project,
especially where constraints are likely to occur. This can be done through regular but progressive
review of the set strategies and compare them to the changing circumstances. In this way, both
the planners and implementers would have an idea whether the project is still running on its
wheels and in the right direction or “the train has left its tracks and is heading for derailment!!!
AFTER the project has ended, it will now be time to launch a full “investigation” into the life of
the project- to cover all the phases of the cycle. The task will involve retracing all the stages:
from planning till the outcomes that result. The process will entail collecting evidence and
thereby identifying the constraints which attributed the implementation as was planned.
During this stage, the real benefits to the target group will be assessed including how many
people actually benefited from the project. Project strengths learnt from this can be used for
replication.
“The aim of evaluation phase is to produce evidence about nature, direction and extent of
behavioral changes which arise from educational endeavours, and to use this evidence as a guide
to modification in any phase of curriculum process “(Wheeler, 1980).
This is obviously a vital phase, because without some quantitative and qualitative comparisons of
actual and expected outcomes, it would be impossible to know whether objectives have been
realized, and if they have been realized, then to what extent? So; “Was the educational
experience suitable for the purpose?” This is an evaluative question because it involves making
judgments about it.

3.4. Revision Exercise


1. What is planning? State the three benefits of planning.
2. List and explain any three factors which may inhibit the success of a well planned function.
3. What is an objective statement? List and explain the four parts of an objective statement.
4. What is an indicator? Name the three types of indicators which are associated with programme
performance.
5. Explain the purpose of the evaluation phase.

57
3.5. Suggested Answers to Module 3 Revision Exercise
1. Planning is a process of discussing and analyzing all assumptions, risks and obstacles to the
successful implementation of the programme/project and offering possible solutions.
Benefits of planning are:
• It reduces uncertainty in that it gives us the opportunity to determine the likely outcomes and
therefore put the necessary corrective measures in place.
• It increases our understanding of the goals and objectives through participating in the actual
exercise.
• It improves efficiency by maximizing the use of resources and ensures work is completed on
time.
• It provides a basis for measuring work planned against work done.
• It is not only a road map showing how work will be done but it is also a tool for decision
making.
2. Three factors which may inhibit success of a function may include:
• Political decisions and policies – pronouncements by politicians may derail a well organized
function. This is also true when there is a change in government policies.
• Poor economic climate – this can have an adverse effect on the targets set by the organisers.
Therefore there should always be a balance between the economic prowess of the community
and the charges being demanded by the event organizers, otherwise the organisers may ‘price
themselves out of business’.
• Competitors and critics of your programme – there is need to analyse the potential of your
competitors. Competitors should not be given chance to turn your golden opportunities into
total misery. You should always strive to live above them. But if given chance, then learn
from them, but they should not be ignored.
• Adverse weather conditions – You should always follow the natural climatic patterns before
announcing the date of the function.
• Natural condition like rain could easily disrupt a well planned function. So before due date,
an announcement to postpone should be made rather than risk losing half the number of
invited guests.

58
• Collapse of a major industry in a town- This can not be ignored because it means workers
will now be turned into loafers. And this would spell doom on the town. Consequently, there
will be no money for merry making as the little cash available could be saved for “family
use.”
3. An objective statement is a statement used to clarify the exact boundaries of the goal
statement and define the boundaries of your project so that every objective set must be one
that leads to achieving that goal.
The four parts of an objective statement are:
• An outcome which is a statement of what is to be accomplished.
• A time frame which is the expected date of completion.
• A measure which is an indicator that will measure success.
• An action is how the objective will be met.
4. An indicator is a signal that is used for simplifying, measuring and communicating important
information in a programme/project and should reflect changes that occur as a result of
particular intervention.
The three types of indicators are:
• Input indicators – are concerned with resources that go into the project/programme including
the activities associated with it.
• Process indicators- are the signals that monitor achievements during implementation and
measure how resources are delivered.
• Output indicators- measure intermediate results, for example, at a point when donor
involvement in the project is close to complete.
• Outcome indicators- measure long-term results of the project and after donor involvement is
complete.
5. The purpose of evaluation phase is to produce evidence about nature, direction and extent of
behavioral changes which arise from educational endeavours, and to use this evidence as a
guide to modification in any phase of curriculum process.

59
4. MONITORING AND EVALUATION PROCESSES
4.1. Description
This module aims at equipping students with the knowledge and skills in monitoring and
evaluation processes. In addition, students will be introduced to the planning and monitoring
processes and explain the purpose of monitoring tools. Also to be introduced will be planning
techniques, types, methods and tools used when carrying out evaluation.
Lastly, students will be introduced to evaluation standards, which are meant to guide the
person(s) carrying out evaluation; but all these will culminate in writing of an evaluation report.

4.2. Learning Outcomes


At the end of this module, the student should be able to:
• Design plans for monitoring
• Carry out monitoring process
• Explain tools used in monitoring
• Design plans for evaluation
• Carry out evaluation process
• Explain the types, tools and methods for evaluation
• Explain the standards for evaluation
• Write an evaluation report

4.3. Introduction
Monitoring is a means of collecting information that will help answer the questions about a
project. It is about tracking the day-to-day activities during programme implementation. It is
therefore important the information is collected in a planned, organised and routine manner.
Monitoring does not focus on the impact or outcomes, but in the manner in which the
programme is put into action. Information obtained from such a process would provide useful
data that could be used for strengthening or expanding the scope of the existing programme.

60
When planning for monitoring or evaluation, it is important to consider the phases through which
a programme must go before it is implemented. The planning stage is the initial phase and
determines the nature and scope of development. If this stage is not handled very well, it is
unlikely that the project will be a success once completed.
The phases of a programme can be stated as shown in the diagram below:

61
62
Monitoring is a process that should be carried out by all stakeholders at all levels. However each
level has specific objectives, roles and methods they can use to carry out their work.
In order to be effective, there is need to give feedback to each and every level.

63
At community level, monitoring would focus on improving the implementation and management
of projects. The focus here is to ensure that projects are completed on time and resources are put
to good use. The role of the community is to identify a project that will reflect community
interest as well as identifying a group of people who will spearhead the process of monitoring.
For monitoring to be effective, a work plan needs to be developed that will guide the monitors
identify specific activities to be monitored. The indicators for each activity will need to be
identified and highlighted in the work plan so that the monitors can compare what is actually
taking place with what was planned.
The monitors should also agree on their meeting schedules to carry out the task. It is
recommended that monitoring be carried out once a week so as to avoid big deviations from the
work plan.
At district level, monitors should focus on project performance which relate to turning inputs
into outputs, achievement of project outcomes and creation of capacity for community to carry
out such a task. Objectives at this level may include giving support to project performance and
measuring applicability of the project to community needs.
Methods used here can include routine monitoring, giving supervisory support and conducting
qualitative enquiry. These enquiries would enable the district to verify information collected at
community level and get information that could have been captured during monitoring. But this
will give the monitors a chance to meet the people on the ground and have on-the-spot enquiries
with them and also learn from them.
At national and donor level, the emphasis is to find out if inputs are well used and outputs are
being realized. It will also focus on whether the designs meant for the project were appropriate
and were being used to collect valuable information. Objectives at this level will include
ensuring effective and efficient use of resources and that activities are going on as planned.
Equally important is to determine whether the project was empowering the community and
drawing lessons for future replication.
There are four research tools used for monitoring the process of a programme and these include:
• STANDARD OPERATING PROCEDURE
This tool is used to examine the efficiency and effectiveness of existing strategies for
implementing a programme. Information about the strategies can be collected through
interviews and surveys and would be valuable to determine whether current approaches are
64
time efficient and cost effective. Once on-the-spot evaluation has been done, it would be
necessary to consider whether any activities may need revision or change in the light of this
evaluation.
• NEW PROCEDURE DEVELOPMENT
This tool is used to develop new rules and guidelines for implementation. It is meant to
assess new procedures or issues in the on-going programme. Information can be collected
through new or innovative procedures set up or designed for implementation, and also
through rationale used for selecting those procedures. Lessons learned from this activity will
provide an insight into new procedures developed for the programme. Once identified, then
procedures can be changed in light of new information.
• STRUCTURES OF RESPONSIBILITY
This tool is designed to explore the roles, responsibilities and relationship of individuals and
various interest groups. The role and responsibilities for individuals and groups involved in
the project should be stated including the support that would be needed in order to carryout
such roles effectively. Information collected from here would be used to assess the
effectiveness of current structures of responsibilities and whether they meet the needs of the
target groups and the programme itself. This tool would provide a valuable experience that
would used to reduce work overlap, clarify opportunities and identify areas of improvement.
• INTERNAL DECISION MAKING PROCESS
The tool is designed to assist in evaluating the internal dynamics of the programme.
Information collected from here will help in assessing the effectiveness of the planning
process. Insight gained from here will be used to make decisions concerning programme
implementation and suggest areas for refinement, modification and change.
Nearly all projects, despite their size and complexity require formal detailed planning. Planning
can be said to be a means of selecting objectives, establishing activities and procedures necessary
for achieving them. It is a procedure for establishing a predetermined course of action within a
forecasted environment, and can be used in decision making process.
Planning must be systematic, flexible enough to handle unique activities and capable of
accepting multi-functional inputs. Planning must reorganize and accept group decisions, and it
ensures increased motivation and commitment to organisational goals. As a management
function it should be used to ensure vertical and lateral communication among the members. And
65
a project to be successful, all members associated with the project must be aware of the strategic
variables than affect the success of the plan. This is desirable so as to take care of the
unexpected, determine the restraints and override the limitations of the programme.
No matter how hard we try, planning is not usually perfect and may fail due to various reasons,
including poor selection of aims and objectives. Poor financial estimates, plans based on
insufficient data and not enough time given to planning may also make planning fail.
Before you design a programme, you need to plan for its evaluation. The programme should
include an evaluation plan, which should address how the goals and objectives would be
achieved and the desired outcomes to be met. The plan should describe the desired outcomes for
the purpose of certification, research and other scholarly activities, and state how outcomes will
be measured and evaluated, and at what frequency. The evaluation plan should be reviewed for
completeness, appropriateness and effectiveness by all stakeholders.
So many questions need to be asked and answered at the time of planning, as follows:
• “How shall we know the programme is effective?
• What indicators will be used to show the effectiveness of the programme?
• How will the programme be implemented and by whom?
• Who are the beneficiaries?
• How will the programme be monitored on a day-to-day basis?
• What activities will tell us the programme is moving in the right direction?
• How will the programme be evaluated and by whom?
But some of the answers to these questions can be found from:
• Existing data
• Programme records
• Attendance registers
• Pictures
• Maps
• Pictorial records
• Programme participants
• Non participants
• Proponents and critics of the programme

66
• Staff and other stakeholders
Therefore, the evaluation plan must describe the manner in which resultant changes will be
implemented, evaluated, documented and communicated to stakeholders. The decision makers
need evaluation findings related to effectiveness, impact and sustainability in order to make
informed decisions-whether to continue, modify or cancel the programme altogether. Although
evaluation is the end of the cycle, it is also the beginning of another cycle.
At the most basic level, there are three types of evaluation, namely process, outcome and impact
evaluations.
PROCESS EVALUATION is used to describe and assess programme materials and activities, and

also enable the evaluator to develop better understanding of the functioning of different elements
and how each one ultimately contributes to the long – term vision.
The evaluator will analyse which objectives contribute most to the vision and develop plans to
adjust objectives or vision or both in light of this analysis. By so doing the evaluator(s) and
stakeholders will develop better understanding of the links between resource usage, unit
activities and the overall goals, and how they all contribute to the vision of the organization.
According to Snyder, this process encourages and enables improvement from the onset. As eople
come to understand the process of their work better, they change their behaviour to reflect that
improved understanding. Questions asked here include; “Are we implementing the programme
as planned? What aspects of the programme are strong weak? What can be done to strengthen
the programme? Are there any unexpected effect? Are there any remedial actions developed? If
so are they being implemented?”
OUTCOME EVALUATION is used to assess achievements and effects of a programme. This can

also be used to study the immediate or indirect effect of the programme on the target groups. The
scope of outcome evaluation may go beyond knowledge and examine the immediate behavioral
effects of the programme. This method allows for the development of measurable performance
indicators; especially those that can be found in ‘resources’ and ‘effects’ elements. If embedded
in the programme, indicators would provide regular feedback about its performance and this can
be used for programme improvement.
Another form of evaluation is IMPACT EVALUATION. It focuses on the long-term results as well
as unintended programme effects. But these evaluations are rarely done because they are too

67
expensive to undertake, time consuming and may need many strategies. For example, when such
an evaluation is undertaken it will examine if a programme’s immediate positive effect on
behaviour has been sustained over the time, and how activities are affecting participants in a
positive manner. Strategies used here may include observation, survey and tracking of former
participants. Information used here may include determining whether there is any change in
behaviour, maintenance of desired behaviour and rate of recidivism.
Despite the methods used, all evaluations use data that is collected in a formal and systematic
manner. Data used here may be qualitative and quantitative; a mix of the two would lead to
successful evaluations.
In broader terms, there are two types of evaluation: formative and summative evaluations.
FORMATIVE EVALUATION is used to ‘test run’ a programme. It is sometimes referred to as

developmental evaluation or implementation evaluation. The techniques used in other processes


can be used in formative evaluation too. Since outcome evaluation is considered quite difficult to
undertake because of feasibility and cost, attention has now turned to techniques which will
maximize chances of a programme being successful.
Formative evaluation, therefore compliments outcome evaluation rather than being an alternative
to it.
SUMMATIVE EVALUATION is an after-the-act assessment, which has to determine whether the

programme/project met its goals or objectives. This kind of evaluation assesses the programme
effectiveness, impact and their cost benefit. Also to be evaluated is the user satisfaction and the
outcomes, intended or unintended.
When evaluating a programme you are going to use more than one method. So is important you
know about them, you may need to examine several possible approaches before deciding on the
best one. By so doing, you will be developing your own knowledge in research practice and
analytical skills. These skills will assist you develop systematic, explicit and reproducible
methods for monitoring and evaluating an existing programme. So when evaluating a
programme, it is important to realistically assess the extent to which the objectives are a result of
the programme and thereby creating a link between the activities and the objectives that are
achieved.
Various methods can be used to collected data, including:-

68
• AN INTERVIEW is a data collection method. This can be structured or non-structured in

nature. In this method, information is collected by asking questions to the respondent, from
whom responses are received immediately. It is a face to face interaction.
The table below shows a comparison between structural and non-structural interviews:

NB: in all cases, the interviewer must be informed of the topic to be discussed before hand.
• MONITORING OF A PROGRAMME
The process monitoring, formal or informal can be very useful in assessing whether the
programme had achieved the intended objectives or not. The report compiled during
monitoring can make useful basis for eventual evaluation of a programme.
• A SURVEY
A survey is an important method of collecting data that can be used in determining the extent
to which the programme has been successful. Surveys are particularly useful because they
take little time to complete, do not require extensive training to administer, can allow for

69
collection of information on a wide range of topics at once. Using this, findings can be
presented easily and clearly.
Other methods may include test, observation, case study, photography, document review and
expert review.
However, although measurement occurs at the end of a project, a continuous guard on the
progress of the project should be maintained through monitoring and evaluation.
Evaluation is such a huge but important task that there is not enough time and resources to
answer all the questions about the implementation and success of the programme, including its
effect on the target group. The best way is to establish priorities and limit the number of
questions.
Some questions may ask about the inputs into the programme, some may ask about the activities
undertaken and their relevance to the programme, and some questions may border on the
outcomes. But other questions may go further to ask about the impact of the programme and how
the same has positively (or negatively) affected the target groups.
Successful evaluation can only be done by professionals or consultants who are guided by
prescribed standards.
Standards are a set of rules and guidelines that provide a common framework for practice.
They are specifications of what should be done, why and how. Standards provide a basis for
determining consistent and acceptable minimum levels of quality performance. They are also
meant to regulate the standards of evaluators when they are carrying out evaluations. Standards
are a kind of criteria that has been established in order to protect the interest of and integrity of
evaluators, those involved in and affected by the evaluation.
Standards can be summarized as follows:-
• UTILITY STANDARDS
These standards have been formulated in order to protect the information needs of all
stakeholders, including those involved in or affected by the evaluation. In order to ensure that
the evaluation findings receive maximum credibility and acceptance, it is important that the
person(s) carrying out evaluation should be both trustworthy and competent in the job. As an
expert you are expected and use certain information that will address pertinent questions of
the programme including the rationale that can be used to interpret the findings. This
classification should be clearly stated in the evaluation report by describing the programme
70
being evaluated, the context in which the programme is being evaluated, the purpose of
evaluation, the procedures to be used and how the evaluation findings will be disseminated to
the stakeholders.
It is recommended that an interim report be produced before a final report; although most
stakeholders would require a final report to be produced within a specified time. For a report
to have serious impact and encourage ‘follow-through’ by stakeholders, it must be planned
thoroughly and be presented in a professional manner.
• FEASIBILITY STANDARDS
These standards are intended to ensure that an evaluation will be realistic, prudent,
diplomatic and frugal. It is recommended that disruptions during evaluation should be
minimal otherwise needed information will not be obtained, and the report may end up
distorted. Therefore, evaluators must use tact and diplomacy in order to get maximum
cooperation from individuals in various interest groups in order to obtain useful information.
It is recommended that the evaluators must be realistic and prudent in their work so as to
produce ‘credible and valuable information’ at reasonable cost.
• PROPRIETY STANDARDS
These standards are meant to ensure that an evaluation will be conducted legally, ethically
and with due regard to those involved and those affected by its results.
Evaluations are intended to serve the interests of the organizations and to effectively serve
the needs of the target groups. It is important that parties to an evaluation should formally
agree in writing as to what should be done, how it will be done, by whom, when it will be
done and how long it will take before a report is handed in. in this way, all parties will be
obliged to adhere to the conditionalities of the agreement.
Evaluations are not meant to threaten or harm participants or those associated with it: if
anything, evaluators should respect and protect the rights of other people. It is also
recommended that evaluations must be fair and non bias, so as to bring about the actual
strengths and weaknesses of the programme being evaluated. Cases of conflict of interest
should be dealt with in a transparent manner so that evaluation results and processes are not
compromised.
• ACCURACY STANDARDS

71
These standards are meant to ensure that an evaluation will reveal and convey technically
adequate information about the features that determine worth or merit of the programme
being evaluated.
It is recommended that the programme being evaluated, the context in which it exists and
sources of information should be described clearly and accurately. It is the role of the
evaluators to choose or design and implement effective information gathering procedures so
that the interpretation arrived at will be varied and sufficiently reliable for intended use. It is
also important that the information collected, processed and reported should be
systematically reviewed and errors, if any should all be corrected.
Information of qualitative and quantitative nature should be appropriately and systematically
analysed so that pertinent questions are effectively answered. Any conclusions made in the
evaluation should be explicitly justified.
The evaluation itself should be formatively and summative evaluated against these or other
pertinent standards so as to guide its direction and credibility. It will be against these standards
that its strengths and weaknesses can be examined.
Once these set standards have been adhered to before, during and after evaluation, it is expected
that the findings will be realistic and credible, professionally adequate and technically correct.
By so doing the work is likely to receive maximum acceptance by all stakeholders.
After evaluation has been done, the next stage is to put pieces together and produce a written
report. This is the report that will be tabled before the decision makers so that they can make
informed decisions. So an evaluation report is a crucial document that can make or break the
programme/project.
Many students usually submit their assignments on the first attempt. Very few usually bother to
correct their drafts in two or three attempts. Saunders et al (2006) as cited in Becker’s (1982)
urges that this is the way most of us learned to write at school. It is because that assignment
papers were only seen by one person – the teacher.
However, project or evaluation reports are different. They will probably be seen by a group of
people rather than one person. Sometimes, they may be lodged in the library for succeeding
students to see. In many cases, evaluation reports will be seen by all stakeholders concerned with
that project/programme.

72
You will be judged on the quality of work, and you are strongly advised to polish up your work
with successive drafts before settling on the best one. “After each successive draft, leave a space
of time for your thoughts to mature. You will be amazed to note that what you wrote few days
ago will now make no sense to you” Saunders et al (2006).
Since your readers will be assessing your work against the criteria that apply to your work, it is
important that you familiarize yourself with that criteria. Easterby – Smith et al (2002) cite
Bloom’s well known taxonomy of educational objectives to illustrate the levels that an
evaluation report must meet.
At lower levels, the reports must show knowledge and comprehension of the topic at hand.
At intermediate level, they should contain evidence of application and analysis. At higher levels,
evidence of synthesis and evaluation should come out. In short, each of the levels of educational
objectives needs to be demonstrated in your report.
Whenever you write, some clear purpose should guide your thoughts; this will give you the
reason to write. “If you do not know why you are writing, neither will your reader”, Reinking
(2003). So fulfilling an assignment does not qualify as a good purpose, although that is what will
compel you to write.
There are many reasons for writing, some of which are:
• TO INFORM
Some professional writers have curved careers out of writing. So informing is one reason
why people write. The evaluation reports we are writing fall in this category.
Readers must be given facts about what took place; it is up to the reader to make conclusions
about the story.
• TO PERSUADE
You may have strong views on some issue. It is such feeling that may compel you to write in
order to solicit for support from your readers.
• TO EXPRESS ONESELF
Creative writing includes personal essays, fiction, plays and poetry. But self expression has a
place in other kinds too; it will offer you an opportunity to be a ‘master of words’. Some
people who are good at tongue twisting have ended up being good public speakers. You can
also develop a good style to express yourself in writing and before long; you could end up
writing books too!!!
73
• TO ENTERTAIN
Some wrings merely entertain. Such writings are meant to lighten the hearts of the people,
especially those in sad and difficult moments. Satire writing is such an example that is used
to throw fun at people while making others laugh.
A good report should contain original ideas rather than copy from other people. So the report
must invoke fresh thinking and ideas, and these must be expressed in acceptable standard format.
Readers do not expect you to display stylistic flair that seasoned writers use; but they expect you
to write it in clear style, strengthened with forceful and emphatic words.
When writing a report, you should ensure it has a beginning, a middle part and an ending. The
introduction sparks off the reader’s interest and acquaints him with what is to follow. The body
delivers the main message and makes a clear connection between ideas so that the thoughts can
easily flow and be followed. The conclusion sums up the report.

4.4. WRITING AN EVALUATION REPORT


The purpose of writing a report is to provide information which serves as a basis for decision
making or further judgements by stakeholders. It also serves as a rationale behind the decisions
and judgements already made.
A good report must state clearly what is being reported and expressed in a manner that will be
most meaningful to the reader. It should include a guide for interpreting diagrams, charts and
statistical data given in the report.
Below is an example of an evaluation report written after evaluating an academic programme at
a college. Only key areas have been highlighted.
ABSTRACT
An abstract is a summary of the whole report. It describes the purpose of the report; highlighting
the main objectives of the programme on which the report is being made and briefly describing
what the report intends to do.
INTRODUCTION
“A good introduction acquaints and coaxes. It announces the essay’s topic and may directly state
the thesis. In addition, it sets the tone… of what will follow”, Reinking et al (2000, p 211).
The introduction should describe the institution on which the report is being made; in terms of
sub-headings. It should also state how each of these items relate or contribute to the running of
74
the programme. It should also state how the report has been organized, what is to be evaluated
and how it will be done, and highlighting the strengths, weaknesses, opportunities and threats to
the programme.
BENEFITS OF A PROGRAMME
Here you should state how the direct beneficiaries or indeed the local community has benefited
from the programme against the set aims.
CURRENT STATUS OF THE PROGRAMME
The current status of the programme may be broken down into the following sub-headings;
• PROGRAMME ORGANISATION
Here you should state how the programme is being organized and run.
• HUMAN RESOURCES
Here you need to state manning levels, benefit of the programme to the target group, institutional
capacity to undertake such tasks and the assistance being rendered by management.
• INFRASTRUCTURE
Here you should state how the infrastructure has positively or negatively contributed to the
running and ultimate success or failure of the programme.
• MACHINERY, EQUIPMENT AND MATERIALS
Here you must state the physical condition of the machinery and equipment being used in
training and how the same are being utilized. Constant provision of materials as training support
must be highlighted.
CHALLENGES – STREGHTHS AND WEAKNESSES
Here you must state the challenges that the programme has experienced. The strengths
(successes) and weaknesses (failures) need to be highlighted. You should also state any reasons
for success or causes for failure.
RECOMMENDATIONS
Here you should state what you feel are the notable strengths and weaknesses including the
opportunities and threats which the programme could have experienced during its
implementation.
CONCLUSION
Here you should wrap up your report by stating a reasoned deduction or inference as to what the
exercise was all about, and what you feel will help stakeholders to make informed decisions.

75
You can also state the problems that you had encountered during evaluation and report writing.
BIBLIOGRAPHY (REFERENCES)
Here you should acknowledge and state any materials that you could have used in compiling the
report.
APPENDICES
Here you should append all documents whose information or data you could have used and also
those you consider being useful in assisting the stakeholders making informed decision.

4.5. REVISION EXERCISE


1. What is monitoring? State the key issues that we should focus on during the MONITOR AND
CONTROL phase.

2. What would be focus of monitoring a project:


• At community level
• At district level
• At national level
3. What is evaluation? State the major difference between formative and summative evaluations.
4. Define a standard. Explain what propriety standard is all about.
5. What purpose does an evaluation report serve? State the six qualities of a good report.

4.6. SUGGESTED ANSWERS TO MODULE 4 REVISION EXERCISE


1. Monitoring is a process of collecting information in a planned, organised and routine manner
so that it can assist in the tracking of the day-to-day activities during programme
implementation.
The key issues during monitoring should be
o Establishing progress reporting system
o Installing change control tools/process
o Defining problem escalation process
o Monitoring project progress against set plan

76
2. i) The attention at community level will be to focus on improving the implementation and
management of projects so that projects are completed on time and resources invested were put
to good use.
ii) The attention at district level is to focus on project performance which relate to turning inputs
into outputs, achievement of project outcomes and creation of capacity for community to carry
out such a task and also to give support to project performance and measuring applicability of the
project to community needs.
iii) The focus at national and donor levels will be to find out if inputs are well used and outputs
are being realized and also to find out whether the designs meant for the project were appropriate
and were being used to collect valuable information. Objectives at this level will include
ensuring effective and efficient use of resources and that activities are going on as planned and to
determine whether the project was empowering the community and drawing lessons for future
replication.
3. Evaluation is the process of finding out whether the project or the programme has met its set
aims and objectives.
The major difference is that FORMATIVE EVALUATION is used to ‘test run’ a programme. It is
sometimes referred to as developmental evaluation or implementation evaluation. It is meant to
carry out a mini evaluation of a running programme.
SUMMATIVE EVALUATION is an after-the-act assessment, which has to determine whether the

programme/project met its goals or objectives. This kind of evaluation assesses the programme
effectiveness, impact and their cost benefit. Also to be evaluated is the user satisfaction and the
outcomes, intended or unintended.
4. A standard is a set of rules and guidelines that provide a common framework for practice.
They are specifications of what should be done, why and how. Standards provide a basis for
determining consistent and acceptable minimum levels of quality performance.
Standards are a kind of criteria that has been established in order to protect the interest of and
integrity of evaluators, those involved in and affected by the evaluation.
The propriety standards are meant to ensure that an evaluation will be conducted legally,
ethically and with due regard to those involved and those affected by its results.

77
Evaluations are intended to serve the interests of the organizations and to effectively serve the
needs of the target groups. It is important that parties to an evaluation should formally agree in
writing as to what should be done, how it will be done, by whom, when it will be done and how
long it will take before a report is handed in. In this way, all parties will be obliged to adhere to
the conditionalities of the agreement.
Evaluations are not meant to threaten or harm participants or those associated with it: if anything,
evaluators should respect and protect the rights of other people. It is also recommended that
evaluations must be fair and non bias, so as to bring about the actual strengths and weaknesses of
the programme being evaluated. Cases of conflict of interest should be dealt with in a transparent
manner so that evaluation results and processes are not compromised.
5. An evaluation report is meant to provide information which serves as a basis for decision
making or further judgements by stakeholders. It also serves as a rationale behind the decisions
and judgements already made.
A good report must include the following sub headings:
ABSTRACT
An abstract is a summary of the whole report. It describes the purpose of the report; highlighting
the main objectives of the programme on which the report is being made and briefly describing
what the report intends to do.
INTRODUCTION
“A good introduction acquaints and coaxes. It announces the essay’s topic and may directly state
the thesis. In addition, it sets the tone… of what will follow”, Reinking et al (2000, p 211).
THE BODY OF THE REPORT
This must state how the programme has been running, including the flow of support resources
put in place which will ensure the programme does not ground to a halt.
It is important to assess how the target groups or indeed the local community have benefited
from the programme against the set aims and objectives.
Here you should also carry out an institutional audit to determine if the host institution had any
capacity to carry on with the programme at hand. Issues of resources, human, financial,
infrastructure, equipment and materials need to be mentioned.

78
The challenges that the programme had faced need to be mentioned here in terms of strengths
(successes) and weaknesses (failures), including the opportunities and threats the programme
could have encountered.
CONCLUSION
Here you should wrap up your report by stating a reasoned deduction or inference as to what the
exercise was all about, and what you feel will help stakeholders to make informed decisions.
BIBLIOGRAPHY (REFERENCES)
Here you should acknowledge and state any materials that you could have used in compiling the
report.
APPENDICES
Here you should append all documents whose information or data you could have used and also
those you consider being useful in assisting the stakeholders making informed decision.

5. MONITORING AND EVALUATION OF INSTITUTIONAL


CAPACITIES
5.1. Description
This module aims at providing students with knowledge of carrying out an institutional audit in
order to determine institutional capacity, including employees’ capacity in carrying out their
roles and responsibilities within the organization. It will discuss employees’ capacity building
through motivation, training and participatory monitoring and evaluation (PM & E) processes. It
will also discuss the role of consultants in M & E processes.

5.2. Learning Outcomes


At the end of this module, the student should be able to:
• Carry out instructional audit
• Design plans to build employees’ capacity through motivation, training and participatory
monitoring and evaluation (PM & E).
• Appreciate the roles of both internal and external evaluators.
79
• Appreciate the need and roles of consultants in the context of M & E.

5.3. Study Notes


Every organization must have a mission and a vision. A mission is the reason for its existence.
It is important, therefore that the mission must be clear and simple. The vision is the forecast of
where the institution will be in foreseeable future the mission and the vision of an organization
must not be left to management alone. They must be shared by all employees. It is only when all
employees understand and share the vision of the organization that they will put in their best to
ensure that overall aim is realized. This is one form of encouraging employee participation in
running the affairs of their organization. Staff are an important asset the organization has.
The staff, if ignored can easily bring down a very big organization. But when motivated, they
can make the company treble its profits in a few months. Therefore, employees need to be
nurtured, rather than be threatened with dismissals. They should be coaxed to stay rather than to
quit the company.
Since staff are part and parcel of management process, they should be involved in all activities of
their organization. And this is what staff capacity building is all about. Capacity building of
employees does not happen spontaneously, management must take the lead to encourage and
train them. Capacity building is a process of giving an opportunity to employees to prove their
worth or their ability to perform. It is known fact that many managers think that workshop
workers like artisans; technicians etc can not perform in an office. And yet the opposite is true.
All they need is to build their capacity by exposing them to management principles and practices.
Therefore, one method of capacity is through what is called staff development programme.
Every organization worth its salt must draw up a programme for upgrading of employees’ skills
and education. Where the organization can not manage to carryout such a programme to suit all,
then others may be encouraged to take up part time or after work studies. But to those employees
with busy or tight work schedules, then distance education will be the answer.
It is, assumed that a highly qualified worker is a better performer and hence a high quality
producer. Since training is a capacity building initiative by the organization, it should not be
viewed as wastage of resources but investment meant to enhance employees’ performance.
Another strategy that manager must engage is what is called delegation of duty. This entails
passing the buck to another person, usually a junior person, to carryout the duty but without
80
necessarily delegating the authority. Appointing a junior person to act in the senior position is
another form of delegating. The idea is to give chance to the junior person to prove his worth.
It is an opportunity which is double edged. One edge is for the junior to prove that he is capable
of doing the work, and the other edge is for the manager to assess the ability and capability of the
junior person to perform. It is through such strategies that the manager can determine which
employees are capable of doing what tasks; it is a talent or ability tapping exercise. The manager
must be able to tell which employees can best be used for say planning or monitoring or
evaluation or other activities the management may be involved in. Managers must realize that
employees, apart from their basic qualifications, possess many talents and experiences. It is up to
managers to take advantage of these and capitalize on them.
Apart from being an investment in human resources, these capacity building strategies should
also be viewed as incentives to the workers since they would have obtained life long skills or
education, which they may use to improve their personal status.
Participatory management has many advantages. It ensures that all workers share a common
vision and mission of the organization. There will be enhanced transparency and accountability
and this will translate into improved performance of the organization. It will also mean that
better decisions are made, and there will be quicker information dissemination through the
established channels of communications.
Participatory monitoring and evaluation (PM & E) is becoming an approach being used for the
purpose of institutional accountability and organizational development, and this will ultimately
entail strengthening the democratic participation of the society. PM & E often raises sensitive (or
threatening) questions about responsibility, accountability and performance (Whitmore, 111
1998). However, approaches emerging from the private sector are showing that large scale
institutions can learn more quickly and effectively through the use of participatory evaluation
and accounting approaches, such as social auditing (Zadek et al, 1997).
Moreover, the policies, procedure and systems within government and large institutions tend to
be rigid, hierarchical and bureaucratic but these can also mitigate against the principles of PM &
E which are based on sharing, flexibility, negotiation and learning. For PM & E to be effective,
there is need to have new kinds of relationships in terms of accountability amongst and between
stakeholders, and this calls for new forms of inter-organisational collaboration.

81
This also means that new forms of learning are created within the institutions, large and small, in
order to enable them operate in a more participatory and flexible manner.
Many local projects which have been evaluated do think of it as a process meant to assess and
pass judgement on them, through a process in which they had little control. In some countries,
the word monitoring has a negative connotation and is associated with words like ‘supervision’,
‘surveillance’ or ‘control’. In those countries, therefore, the words ‘monitoring and evaluation’
are meant to ‘spy’ on marginalized groups.
Therefore, PM & E attempts to change these traditional way of understanding through a process
that seeks to share control amongst various stakeholders – the sharing is not usually equally. PM
& E is not just about accountability of the organization but also accountability to the
organization. That is why traditional way of top-down management and one way accountability
ought to be reversed. Many are using this method as indicators of citizens’ participation in
management and governance. In the United States of America, citizen monitoring has a long
history and it is used as a means by which citizens assess and hold government accountable for
its programmes.
It can therefore be concluded that PM & E attempts to contribute to new forms of governance,
involving greater transparency and more democratic involvement between citizens and the
broader institutions that affect their lives (Gaventa and Valderrama, 1999). However,
accountability is a contentious concept; as it changes, issues of how to deal with power and
conflict become critical concerns.
In broader terms, PM & E seeks to give a voice to the voiceless, power to the powerless and
sometimes, it can be used as a means for attempting to redress power imbalances that exist in
many organizations. But this process can also allow ‘hidden’ stakeholders to emerge. And when
new actors enter a social arena, they may try to instill their ideas and set new priorities, which
may lead to new conflicts and disagreements.
Therefore, PM & E can provide a framework and forum for settling scores with different interest
groups. In such cases, identification and use of indicators can offer a means for improved
communication and make negotiation among different actors. Therefore, participatory indicators
allow for ‘give and take’ situation rather than direct confrontation.

82
It is part of capacity building process when employees are given opportunities to articulate their
views, ideas and needs through discussion.
But for tangible change to occur, it should start from within the organizations. And for any
change to occur, organizations must draw lessons from their successes and failures in order to
improve on institutional behaviour and performance. “The key to achieving impact and
competence … was not pre-planning, but an organization with a capacity for embracing error,
learning with people, and building new knowledge and institutional capacity through action”
(Korten, 1980. p. 480).
In the 1990s, proponents for organizational development argued that a change in organizational
practice would best be achieved if individual change and behaviour was encouraged and
provided with incentives. So by creating best practice, it was hoped that pressure was the only
way organizations would change for the better and in line with ‘new cultural trends’. This
approach is viewed as the best benchmark in corporate social responsibility work.
It is argued that creative learning can best take place by responding to rather than fighting against
prevailing institutional culture. It can be concluded that institutions which have developed a will
to learn and embrace an organizational culture are taking steps towards greater public
accountability. However, lessons learned from this can create guarantees in the context of
flexibility, ownership, accountability, transparency and trustworthiness.
For institutions to change, employees need to be motivated in order to apply what they have
learned. But ownership can end up to be a participatory rhetoric if institutions do not recognize
and appreciate individual roles and responsibilities and the benefit which these roles can bring to
those individuals.
Accountability does not only relate to financial transparency but to the social and economic
impact of the organisation’s activities. This involves changing relationships amongst all
stakeholders – especially between the ‘rulers and the ruled’; through meaningful dialogue and
openness about successes and failures. Responding to demands for strategic accountability
(wider impact) over functional accountability (resource accounting) is still proving to be a
challenge for these and other organizations (Edwards and Hume, 1995).
It should be recognized that there are very few employees who would share their failures in their
work or their future plans unless they have trust in their positions as well as in the organizations

83
they work for. Yet trust requires more than ‘permission’ to give voice to opinions. In most cases,
those at the bottom usually offer their ideas or opinions based on experience but those in
management positions use their ‘book knowledge’ to argue their cases.
But a mix of the two would create an ideal situation for open institutional audit. In this way a
sense of responsibility sharing would be created by direct impact which in turn would lead to
further promotion of trust, particularly by weaker stakeholders.
Employees who are given chance to participate in the running of the affairs of their organization
tend to cultivate a sense of belonging and ownership of that organization. For example,
decentralization policies in Zambia have empowered citizens, not only to monitor the
performance of the councils in which they live but taking part in all civic activities.
Institutional capacity does not only refer to the readiness or ability of an institution to host an
activity or run a programme. Capacity also involves flexibility and creativity, not just efficiency.
At college level this may entail examining all necessary resources which would be used in
activity/programme implementation. PM & E process also caters for problems of staff changes.
It holds that an organization should not come to a standstill because of a departure of a senior
manager.
Nowadays, it is becoming increasingly important for an organization to develop closer links with
other institutions in the same sector of operation. This is meant for idea sharing and learning
from each other, and in this way an organization would assess its own performance against those
they perceive to be ‘role models’.
It can be argued that PM & E is nowadays being practiced across the globe. These concepts are
being applied in every sector of human endeavours; including education, health, agriculture etc.,
and in small and large organizations. But there is still more to be done in order to strengthen the
conceptual and methodological bases of PM & E. Therefore, more must be done so as to
negotiate conflicts towards building collaborative action and apply it on a larger scale to include
issues of governance of institutions/organizations.
Learning from change is not an end in itself, if anything it is a process of reflection that affects
how we think and act in order to change the future. It is said that learning from change is
learning to change. If we can not learn effectively from our action then little will be done to

84
improve our understanding of the world we live in and worse still, we can not act more
effectively on it. So learning from change means changing those who learn.
The stages of change models emerged from the theories by Prochaska (1994) based on the
empirical research on smokers and the mental health behaviours. He concluded that there were
five stages of change:
• The first stage is called pre-contemplation. At this stage the individual has made no intention
to change his behaviour, and is not yet aware of this problem at all.
• The second stage is called contemplation. At this stage, the individual is aware of the
problem and is seriously considering taking some action about it.
• The third stage is called preparation. At this stage, the individual is intending to take action
sooner than later. He may have tried to take action sometime back but did not succeed.
• The forth stage is called action. At this stage, the individual will take action by changing his
behaviour.
• The fifth and last stage is called maintenance. At this stage, action is taken towards
preventing the recurrence of the same problem by consolidating gains attained during action
stage.
However, behavioural change is not linear; a lapse can recur at any time. There have been many
theories on change which have been developed; including,
• Diffusion Theory (Rogers, 2000) which holds that change occurs when new ideas are
invented, diffused, and adopted or rejected, leading to certain consequences.
• Empowerment Model (Fetterman et al, 1996) is a process by which people gain control and
mastery over their own lives and are able to influence others.
• Ecological systems Model (Bronfenbrenner, 1979) emphasizes the explicit on the role of the
physical and external environment in behavioural change.
• Social Marketing Model (Kotler and Roberto, 1989) adapts commercial marketing and
advertising techniques to programmes in an effort to influence voluntary behavioural change
of a target population.
In supporting the theories of change, Wheeler explained that among the
most powerful influences on an individual’s life pattern are his
emotions, so any consideration of curriculum must refer to emotional
adjustment and control. These are crucial to personality development
85
and mental health. In general terms, the theory of behaviour is of great
relevance to curriculum because it purports to explain why and how the
individual behaves the way he does.
Although employees are said to be assets that an organization has, but it does NOT own them. It
is wrong to regard workers as tools of production; rather they should be regarded as partners of
production. If capacities of the workers are enhanced so will the capacities of the organization as
well. It is therefore, the responsibility of the organizations to create an enabling environment for
the workers to carry out their work.
Workers have certain expectations about the organizations and the jobs they do. These
expectations may be classified as motivators and hygiene factors. Motivators are the factors or
situations that have a positive impact on the workers – they entice the worker to perform better.
Hygiene factors are those factors which by their absence have a negative impact on performance,
but they do not necessarily motivate the workers if they are present.
Herzberg (1959) who carried extensive research in motivation theory identified the following as
motivators:
• Achievement
• Recognition
• Advancement and growth
• Responsibility
• Work itself
And he also identified the following as hygiene factors:
• Company policy
• Working conditions
• Interpersonal relationships
• Job security
• Salary
As can be noted, motivators are related to the job, especially to its intrinsic characteristics but the
hygiene factors are related to the environment in which the job is performed.
Professionals have always responded to their challenge. A challenging job would always keep
the professionals ‘on their toes, and on their job’. Every worker wishes to progress in their job
and recognition of their contribution is one such attribute they value greatly.
86
For a job to be rich and enjoyable, its design is of great importance. Workers prefer jobs that
have many task varieties or opportunities to learn and practice new skills. This is where
management should engage various strategies like job enrichment or job enlargement. But some
workers may view this as an extra work load for which they should be paid, but others may jump
on this opportunity and learn new methods. Another dimension that defines a job is its identity. It
is critical for a worker to master his job very well and how that job contributes to the overall
mission and vision of the organization the work for. If the job is on a critical path, it will affect
their attitude and hence quality of work.
Workers, especially professional prefer to work on their own or with little or no supervision.
This is the only way they can prove their worth and prove that they are doing what they know
best. Best performers should be commended but those who perform below par deserve to be
warned.
Another way to encourage performance is by way of incentives. An incentive is viewed as a
reward hard work or extra work put in, e.g. a bonus paid for high production. Incentives also
have a serious disadvantage. There is no guarantee that after getting a bonus, then production
will go up or will remain at post bonus levels. Some workers may feel that it was within their
right to get a reward for job well done.
Promotion may be viewed as a reward for hard work, which is also meant to entice others to
work hard. But it can only be viewed so if it is done on merit. But others have a different school
of thought; as they view it as a way to silence voices that usually speak for the voiceless. But
whatever the case, promotion should go to those who deserve it.
Some administrators usually think that incentives and motivators are the only factors that will
compel workers to work harder and double production. This is a mistaken notion. People look
for employment with intention to satisfy certain needs. Others want to own a house, a car, a
higher salary, or just experience etc. But these needs are explained in “Hierarchy of Needs”
(Maslow, 1943).
Maslow’s ‘hierarchy of needs’ is always depicted as a pyramid, consisting of five levels; the
lower four levels are called deficiency needs or D-needs, and are associated with physiological
needs. The top level is represented by being needs or growth needs or B-needs and these are
associated with psychological needs. Below is a table showing Maslow’s hierarchy of needs:

87
5.4. Physiological Needs
These are the needs a human being must have and longs to have. These needs include good food,
safe water to drink and proper sanitation and a safe shelter which can ensure your safety.
According to Maslow, once a person is hungry or thirsty, the body becomes chemically
unbalanced; that all his energies and mind are geared towards getting rid of these deficiencies;
and therefore other needs remain passive or inactive.
Once these are satisfied, the person then looks forward to security needs; security of himself,
family, job, property and health. Nobody would wish to live in a troubled world.
Every person looks forward to a normal, predictable and orderly world where safety and security
are guaranteed. And this is the more reason why companies must first be assured of security of
the place before they can invest their money in the business.
The third level of human needs is the social need, which brings out a sense of belonging and
acceptance. They need to love and be loved by others. This level involves emotionally based
relationships, such as friendship, intimacy and having a supportive family. In the absence of
these, many people would become lonely, develop social anxiety and may end up in depression.
The forth level in this class of physiological needs are the esteem needs. All humans need
respect, self respect and to respect others. They need to be recognized, accepted and valued to be
in that profession; and expect to get praise for a good job done. In the absence of these may lead
to low self esteem and inferiority complex. Physiological needs can control thoughts, behaviours
and can make people sick, have pain and discomfort.

88
5.5. Psychological Needs
These are the growth needs; they are also called enduring motivations or drivers of behaviour.
These are the final stage of Maslow’s pyramid of needs. According to Maslow, every person
desires to make the most of their abilities, working towards fulfilling their desire to be the best
and …’filled with a desire to realize all their potential for being an effective, creative and mature
human being. What a man can be, he must be’ (Maslow, 1943).
Self actualization is reaching one’s full potential. Self actualizing people are said to have attained
a high level of maturation, health and self-fulfillment that ‘sometimes they seem almost like a
different breed of human beings’. They possess an ability to be objective about their own
strengths, possibilities and limitations. Moreover, they are believed to be honest, open, genuine,
and without pose or façade. In addition, they are presumed to be self motivated, unselfish, and
are good at problem solving. Social auditing is a process by which the social and ethical impact
of the organization can be assessed. This is done from two perspectives: one from inside and the
other looks at performance from outside. From inside, the organization is assessed against its
mission statement, aims and objectives. This is to ensure that the organization is not only moving
forward but is seen to be accomplishing its mission too. In the final analysis, conclusions should
be made as to whether the resources invested were now bearing fruit as expected.
From outside, the organization is compared to other organizations in the same sector, in the
context of social norms and behaviour. It has been said that competition brings out improved
service delivery and quality products. By listening to and reporting the assessment of your
organization to its stakeholders, the social audit will provide feedback about areas in which the
organization is failing to meet the stakeholders’ expectations and also against its own stated
objectives; and account for its performance to a wider range of interested people than simply
those who have invested capital (e.g. funders) (Mayo, 1996).
But when it comes to evaluation, the landscape is rather different.
The original mission of programme evaluation in the human services and education fields was
meant to assist in improving the quality of social programmes. However, the evaluation
landscape has changed over the years. Programme evaluation has come to focus (both implicitly
and explicitly) much more on proving whether a program or initiative works, rather than on
improving programmes.

89
During this period, “systematic evaluation [was] increasingly sought to guide operations, to
assure legislators and planners that they [were] proceeding on sound lines and to make services
responsive to their public” (Cronbach et al., 1980). “Programme evaluation as a distinct field of
professional practice was born of two lessons…: First, the realization that there is not enough
money to do all the things that need doing; and second, even if there were enough money, it
takes more than money to solve complex human and social problems. As not everything can be
done, there must be a basis for deciding which things are worth doing. Enter evaluation” (Patton,
1997, p. 11).
Conducting an evaluation requires an organization to invest valuable resources, including time
and money. It is an expensive venture but the benefits of a well-planned, carefully conducted
evaluation outweigh its costs.
In general, there are three categories of evaluators:
• External evaluators,
• Internal evaluators, and
• Internal evaluators with a consultant.
You must determine which category would be most beneficial to your project/programme.
External evaluators are contracted from an outside organization to conduct the evaluation.
These evaluators often are found at universities and colleges. Because external evaluators
maintain their positions within the organizations they work for, they generally have access to
more resources than internal evaluators (i.e., computer equipment, support staff, library
materials, etc.).
In addition, they are likely to have broader evaluation expertise than internal evaluators,
particularly if they specialize in programme evaluation or have conducted extensive research
before.
This internal evaluator could serve as both an evaluator and is a staff at the institution where he
works. Because an internal evaluator works within the project, he is likely to be more familiar
with the project and its staff and the local community. He is also likely to have access to
organizational resources, and have more opportunities for informal feedback with project
stakeholders.

90
However, an internal evaluator may lack the outside perspective and technical skills of an
external evaluator. Although this is less expensive, its credibility may be questioned because the
internal evaluator is not only close to the project to notice any significant changes. Besides, there
is a likelihood of a bias in the evaluation.
A selected member of staff can conduct the evaluation as an internal evaluator, and an external
consultant can assist with the technical aspects of the evaluation and offer some help on
gathering specialized information.
With this combination, the evaluation can provide an external viewpoint without losing the
benefit of the internal evaluator’s first-hand knowledge of the project. So the deficit on one side
could create gains on the other side, that is what the internal evaluator is missing may be
complimented by the expertise of the experienced consultant.
In such cases, the job of the internal evaluator is to collect information, handle logistics and do
the day to day evaluation tasks. The task of the consultant is to audit the evaluation process,
review the data collected, do the data analysis and write the final report.
Consultants are technocrats in their own way. They know what data to collect which will be
viewed as credible. They also know which analyses to be conducted and who to contact for
valuable information, putting in mind the ethical standards prescribed for them. This is necessary
when drawing up the evaluation instrument and the evaluation plan.
Professional consultants know which people need what results and how such results will be
transmitted to them. Above all, they are capable of simplifying the results so that stakeholders
can interpret them before making informed decisions about the programme/project.
Whenever you decide to engage an external or internal evaluator or a combination of both, it is
important to think about their roles. The goals and practices in the field of programme evaluation
have diversified, so too have evaluators’ roles and relationships with the programmes they
evaluate.
If the evaluation purpose is to determine the worth or merit of a programme, you might look for
an evaluator with methodological expertise and experience.
If the evaluation is focused on facilitating programme improvements, you might look for
someone who has a good understanding of the program and is reflective.

91
If the primary goal of the evaluation is to design new programmes based on what works, an
effective evaluator would need to be a strong team player with analytical skills.
Experience tells us that the most important overall characteristic is to look for in an evaluator
who has the ability to remain flexible and to problem solving.
It is in this vein that the funders or sponsors usually prefer renowned consultants to carry out
such tasks. Despite the cost, their work is usually credible and reliable.
Sometimes, other consultants may be required to carry out what is called meta-evaluation.
This is an evaluation of an evaluation. It can be likened to having a second opinion over the
evaluation already conducted. This is done where the stakeholders have differing opinions on the
issue in the earlier evaluation. In this case, the evaluator’s main task is to assess the feasibility
and accuracy of an evaluation, competence of the earlier evaluators and integrity of the
evaluation findings given in the reports.

5.6. Revision Exercise


1. What is meant by employee capacity building? Mention three ways in which an organization
can encourage their participation in the running of the organization.
2. What is participatory management? State any three advantages of this principle.
3. On what principle is PM & E based? What purpose does this approach serve in the democratic
governance of organizations?
4.
(i) Define motivators and hygiene factors; according to Herzberg. State any three examples of
each
(ii) According to Maslow, what is the major difference between physiological and psychological
needs?
5. In a case where both an internal evaluator and a consultant are engaged to carry out a job
together, can you state the roles each one is likely to perform during the evaluation exercise?

5.7. Suggested Answers to Module 5 Revision Exercise


1. Employee capacity building can serve two purposes:
• To enhance their work performance by improving on their efficiency and effectiveness in
their work areas.

92
• To give an opportunity to employees to prove their worth or their ability to perform.
Since staff are part and parcel of management process, they should be involved in all activities of
their organization. And this is what staff capacity building is all about. Capacity building of
employees does not happen spontaneously, management must take the lead to encourage and
train them.
Staff can be encouraged to participate in the affairs of the organization by way of :
• Acting in the higher positions when the incumbent are away.
• Job delegation
• Participate in meetings and let your voice be heard and contribute positively
• Job enlargement
• Job enrichment
2. Participatory management is a principle based on giving workers more opportunity to
participate in the affairs of the organization.
Participatory management has many advantages.
• It ensures that all workers share a common vision and mission of the organization.
• There will be enhanced transparency and accountability and this will translate into improved
performance of the organization.
• It will also mean that better decisions are made, and
• There will be quicker information dissemination through the established channels of
communications.
3. Participatory monitoring and evaluation (PM & E) is based on the principle of sharing,
flexibility, negotiation and learning
It serves the purpose of contributing to new forms of governance, involving greater transparency
and more democratic involvement between citizens and the broader institutions that affect their
lives (Gaventa and Valderrama, 1999).
4 (i) Motivators are the factors or situations that have a positive impact on the workers – they
entice the worker to perform better.
For example,
• Achievement
• Recognition

93
• Advancement and growth
• Responsibility
• Work itself
Hygiene factors are those factors which by their absence have a negative impact on performance,
but they do not necessarily motivate the workers if they are present.
For example,
• Company policy
• Working conditions
• Interpersonal relationships
• Job security
• Salary
4 (ii) Physiological needs are the needs a human being must have and longs to have. These needs
include good food, safe water to drink and proper sanitation and a safe shelter which can ensure
your safety. Other needs include security needs; security of oneself, family, job, property and
health, and also the social need, which brings out a sense of belonging and acceptance, to love
and be loved by others. And the last group of needs are the esteem needs where all humans need
respect, self respect and to respect others. Physiological needs can control thoughts, behaviours
and can make people sick, have pain and discomfort.
Psychological needs are the growth needs; they are also called enduring motivations or drivers of
behaviour but they are commonly known as self actualization needs. According to Maslow, self
actualization is process of reaching one’s full potential. However, those who have reached this
stage are believed to be honest, open, genuine, and without pose or façade. In addition, they are
presumed to be self motivated, unselfish, and are good at problem solving.
5. In such a case, the job of the internal evaluator would be to collect information, handle
logistics and do the day to day evaluation tasks.
The task of the consultant is to audit the evaluation process, review the data collected, do the
data analysis and write the final report.

94
6. DISSEMINATION OF INFORMATION AND DATA ANALYSIS
TOOL (SPSS)
6.1. Description
This module is intended to highlight the importance of information dissemination and how this
can be managed by management. It will also discuss how data collected can be processed and
analysed using data analysis software called SPSS.

6.2. Learning Outcome


At the end of this module, the student should be able to :-
• Explain importance and purpose of information dissemination .
• Explain the process of data collection and processing.
• Carryout data collection, processing and analysis of results using the data analysis software,
called SPSS.

6.3. Study Notes


Communication is the activity or process of expressing ideas and feelings or of giving people
information (Hornby 2002). Information may take the form of spoken word, written form or
signs. Person to person interaction can be achieved through speaking (use of voice), by way of
signs (sign language) or by reading from the written word.
Verbal or oral communication is the commonest and the fastest way to disseminate information.
The advantages of oral communication are:
• It gives an opportunity for many people to participate in the activity eg a discussion.
• Verbal reporting is usually done after an event. This makes the information arising from the
process relatively valid, reliable and up to date.
• Oral communication gives the listener an opportunity to ask questions in order to clarify the
issue at hand. Therefore, feedback is immediate and this helps in decision making.
• During oral communication, information can be disseminated to as many people as possible
eg. A teacher lecturing to 100 students in a class.
• Verbal communication cuts down on costs. It is the cheapest form of information
dissemination. It cuts down on time and other resources spent on reporting.
The oral method of communication has some disadvantages too; including:-
95
• The one reporting can choose to give wrong information especially if the information
implicates him or if he has any vested interest.
• It is very rare that verbal reporting can be disrupted by those listening except where someone
else knows otherwise. But the truth can be revealed through questioning.
• Oral reporting does not give time to the speaker to think through the questions asked and the
responses given. Longer periods between responses may be attributed to falsification of
information.
• It is difficult to keep and retrieve the same information reported for future use. Therefore,
there is likelihood of distortion if the speaker was asked to repeat his report or speech after
few weeks.
• The information is only kept by few people, especially those who participated in the event or
those who witnessed the event. The rest can only depend on hearsays, especially if those who
know the truth are unwilling to share it.
• Verbal information is not likely to be consistent especially where past information is needed
to generate new data. Besides, verbal information can not be stored and retrieved in the same
form. Once the “owner” dies then information is instantly lost.
• Sadly one can easily deny that he ever said that. So this creates a problem as to whom to
believe.
Oral information can be communicated through speech from person to person. This will have
same effect as using the radio but a TV has more effect because it requires you to engage both
your hearing and sight.
Written communication is considered to be an ‘official’ method of disseminating information
because it is considered to be ‘indelible’.
There are some advantages; including:
• They are reliable means of communicating to as many people as can be reached at once.
• Written reports can be cross-checked to ascertain accuracy of information, including its
source.
• Written information can easily be managed and stored in a safe place. Nowadays information
can be stored on the computers CDs and flash discs, and can be kept in the same form for a
very long time.

96
• The report must be elaborate so that it can be understood and acceptable by the intended
audience.
• Once written, information can be accessed by many people unless if it is “classified” or
“confidential”.
• The writer can not deny having written the document except if it was forged. So it is
advisable that the writer must sign the document so as to make it authentic or official.
Written, communication has also some disadvantages:-
• The communication has a delayed feedback or response. Some response is not immediate; it
may take hours, days or weeks to come by.
• Documentation of reports is costly both in time and money.
• Written reports are not up to date because they tend to ignore the day to day recordings done
during monitoring. They are usually summarised in order to produce a report.
• Written communications are targeted at selected few, leaving many others who could be
affected by its message. It is therefore, the management function to ensure that the best
method is used in order for the message to reach all concerned.
• Writers of such messages find themselves vulnerable, as they could be sued for damages
their messages could have caused.
• Addressees have no chance to ask questions for clarification. It does not provide any chance
for question and answer session.
The other method used in communication is by the signs. These can be in the form of gestures,
use of hands facial expressions or body language to convey a message. But the communication
meant for the deaf need special training to be able to use it effectively.
To disseminate is to spread information, knowledge or message so that it reaches many people.
In many organizations including government departments, there is a culture through which
information is disseminated to employees. This culture is called channel of communication.
Information is disseminated through some hierarchical order from top to bottom. This culture is
part of bureaucratic system of governance.
A letter addressed to Mr. X informing him that he has been promoted to be a supervisor is
usually seen by all his superiors. This acts as a means to inform them too. A notice to inform all
employees about the tour to the organization by a Minister can best be pinned on the notice

97
board for all to read. A notice to delay salaries or abolish some positions in the company may be
disseminated to the employees by way of a meeting. This will afford an opportunity for all
workers to ask questions to clarify the matter. So it is up to the manager to decide which way is
most effective and efficient.
In the context of M & E, the same can be applied in the same way. Once the information is
processed it should be disseminated to the stakeholders. The evaluator must know which
information is required by whom. The information collected during evaluation does not need to
be processed manually, because some data may not easily be analysed. Manual processing of
information has a limitation and because of that it may not easily be seen to be credible as far as
the stakeholders are concerned.
While outcome evaluation is intended to assess the intended (or unintended) positive (or
negative) consequences of a programme, it is not always the best type of evaluation to undertake.
In many cases it is difficult (even impossible) to undertake this task because of feasibility and
cost. Even where it is feasible and affordable, it may be years before the results become
available.
It is in this light that special software called SPSS has been designed to help in data processing
and analysis. This programme has a capacity to reduce time from years to few months, before
results become available.
The SPSS is a data analysis tool which has been designed to solve the organization’s most
challenging issues. Such issues may be retaining old customers, attracting new customers or
minimising losses through frauds can be processed by this software.
The SPSS predictive analytics software can be used to detect fraud, non compliance with the
laws and regulations and to protect public safety and enhance security. Educational institutions
and NGOs can use the same software to manage resources efficiently by predicting demand for
the programmes they offer and embark on fundraising ventures using cost effective methods.
Health institutions can use the software to carryout research and improve on their health care
delivery.
For more than thirty years, many institutions have used the SPSS software to predict changes in
people’s attitudes and behaviour. This has enabled them to confidently change strategies and
tactics to meet their goals. In this era of increased competition for customers, organizations can

98
use this facility to woo customers, through transaction history and demographic information of
their customers. Therefore, information obtained from such will give a comprehensive picture
that would enable an organization act quickly and effectively; hence obtaining a decisive
advantage over other competitors.
The SPSS software is a comprehensive and predictive analytics software meant to improve
business performance by way of analysing the past, present and future outcomes and trends.
This software is robust and can be expanded to suit any organisational predictive needs. Using
the information gathered from interacting with customers, the SPSS technology can prescribe a
recommendation on the possible action to take with a customer and predict possible response a
customer would make, and this is called real time decision optimisation. This would result in
greater achievement if organisational goals; be they for revenue generation, cost reduction,
customer retention or fraud prevention.
SPSS software provides solutions that can easily integrate with other operational systems and
maximise on gains from predictive analytics based on daily business operations. This is made
possible by using predictive results embedded in the applications. Better still; the software can
easily integrate with other software vendors, including Siebel Systems, Oracle/PeopleSoft, SAP
and many others.
SPSS software has become quite popular due to their faster return on investments. Results can be
received within few months not years. For example, FBTO, an insurance company in the
Netherlands, implemented an SPSS software programme. In two months, FBTO achieved
measurable increases in marketing responses and lower overall marketing costs.
This software does not need additional investments in hardware and other peripherals in order to
effectively implement their solutions. The technology is designed to work with the existing
information and communications technology (ICT) infrastructure; thereby keeping total cost of
hardware ownership at its lowest.
Before you start analysis, you must get your data ready. After completing the analysis, you will
need to put results in a format that people can use. Using software with limited capability means
you have to deal with many types of software before completing the work. But SPSS has
‘everything under one roof’ software. The data can be processed from planning, management,
data analysis until results are obtained, all in one.

99
The figure below shows the seven steps that your data must go through before results are
obtained. The software types used at each stage are shown at every phase:

Because SPSS products are streamlined, they can work together. But using products from other
vendors will force you to duplicate the work, almost at every stage. For example, if you collect
data in SPSS Data Entry, everything will be carried through data management and preparation.
So you can only concentrate on analysis. With SPSS family of software, you can import data
from one programme to another without any problem. For example, you can collect data in
Dimensions and import it to SPSS to create multi-item indices for scoring questions. The new
variables can be appended to the original data and exported to Dimensions again for analysis.
The following are some of the products in the SPSS family:-
• For planning, you can use Sample Power, SPSS Complex Samples and SPSS Conjoint.

100
• For collecting and entering data, you can use Dimensions and SPSS Data Entry.
• For faster data analysis, you can use SPSS Base.
• For data management and preparation, you can use SPSS Base, SPSS Data Preparation,
SPSS Missing Value Analysis and Dimensions.
• For reports on your results, you can use SPSS Base, SPSS Tables and Dimensions
• For results shared with others, you can use SPSS Predictive Enterprise Services.
The SPSS Base provides you with a wide variety of statistical functions designed to give you
most accurate response for a specific data type. Below is a list of statistical functions:-
• Linear regression is for exploring the relationship between predictive indicators and what
you want to predict. For example, sales can be predicted using price or customer index.
• Factors analysis is for identifying variables or factors that will explain correlation within a
set of observed variables. For example, you can use this data reduction procedure to identify
a small number of factors that will explain most of variance observed in a much larger
number of manifest variables. Factor analysis also includes methods for factor extraction,
rotation and for computing the factor scores.
• Two – step cluster analysis is for dealing with large datasets using the sealable duster
analysis algorithm shown below. This function can handle continuous and categorical
variables. The procedure enables you to group data so that records within a group are similar.

101
This algorithm enables you to find clusters in large datasets and mixed datasets with
continuous level variables (e.g income) and categorical level variables (e.g type of policy).
• K- means cluster analysis is for group data from larger databases e.g customer mailing lists.
In this case, data will be grouped in identified number of clusters. The data will then be
posted to each cluster using that identified number. Here you can select one of two methods
classify cases: either by updating cluster centres iteratively or by classification only.
• Hierarchical cluster analysis is for merging cluster from single records to form groups. You
can choose from more than 40 measures of similarity or dissimilarity, standardize data using
several methods and cluster cases or variables. Using the proximities procedure, distance and
other similarities can be generated, and statistics at every stage can be displayed. This
procedure is recommended for small databases (e.g target group lists).
• Ordinal regression (plum) is for making predictions with ordinal responses. For example, by
identifying customer ratification level (e.g very satisfied, satisfied or very dissatisfied) you
can understand customer loyalty. By choosing different links, you have the flexibility to fit
into ordinal logistic regression, ordinal profit models and Cauchit models, and you can also
model both the location and scale of the underlying distribution.
102
SPSS base contains many features (and techniques) which are designed to easily prepare and
manage data before it is analysed, including those meant for file management. The following
are some of the features available in SPSS Base software.
• The Data editor is meant for defining, entering, editing and displaying data, using a spread
sheet like system. This feature has been upgraded to include such functions like finding and
replacing information, check the spellings, labeling variables and sorting them by name, type,
format and other details.
• Open multiple datasets within a single SPSS session is meant for merging data files. It helps
to maintain consistency when copying data dictionary information from one file to the other.
• Define variable properties is designed to set up data dictionary information, such as value,
valuable labels and types. Once information is set up in dictionary form, you can apply it
using another function called Copy data properties. The dictionary information can be
applied to other data files and variables within the same file.
The figure below shows a data dictionary to prepare your data for analysis using the Define
variable properties tool. A data pass would allow SPSS to present a list of values and
numbers of these values, so that you can intelligently add them.

• Visual Binner enables you to create bins or bands. A data pass would create a histogram that
allows you to specify cut points in an intelligent manner. You can automatically create value
labels based on your cut points.
103
This figure below shows the Visual Banner on occupational prestige where “age” was binned
as the new variable and set cut points by age groups:

• Identify duplicate copies tool is meant to clean the data when duplicate records are detected
through the user interface, and track multiple duplicates within the record.
• Export to database wizard is meant to write back to databases from SPSS in the interface.
The wizard can create a new table and export it to the database in order to keep data in both
SPSS and database consistent.
• Custom attributes tool is designed to create dictionary information for variables.
• Variable sets are designed to instantly reduce the variable view to a subset of selected
variables, while keeping the file loaded and available for analysis.
• Output management system (OMS) is designed to create own custom programme. For those
who have no experience using syntax in SPSS, they can create custom programmes using
OMS function.
• Data restructure wizard is meant to restructure a data file that has multiple records per
subject.
• Clone or duplicate datasets will enable you to carry out transformations on a duplicate set of
data while protecting the original data.

104
• Data transformations tool will enable you work with combined data more reliably. This is
necessary when you want to create multiple item indices especially when you are working
with questionnaires that ask for positive or negative worded responses.
SPSS has a wide range of capabilities for the entire analytical process, and it can work with
popular operating platforms including Windows, Mac and Linux. It has a wide range of
applications including survey and market research, academics, resource planning and
management, medical and scientific research, planning and forecasting, quality control and
decision making and reporting.
In academic circles, SPSS is designed as a modular, fully featured programme meant to manage
information from planning to processing of desired results.

6.4. Revision Exercise


1. State the three types of communication. Which one is the commonest and why?
2. List three advantages and disadvantages of a written communication.
3. In which way can the SPSS software assist organizations improve their business performance?
4. Mention the stages through which the SPSS data must pass before results are obtained.
5. Mention six applications which the SPSS range of software can be used for.

6.5. Suggested Answers to Module 6 Revision Exercise


1.
The three types of communication are verbal, written and signs.
The commonest one is verbal. People spend more time talking to one another than any other
means. It acts as proof that the person you are talking to is alive and listening and you can get
feedback almost immediately.
2.

105
3.
The SPSS software helps business organizations improve their performance by way of
analysing the past, present and future outcomes and trends. This software is robust and can be
expanded to suit any organisational predictive needs.
4.
• Planning
• Data collection
• Data access
• Data management and data preparation
• Data analysis
• Reporting
• Deployment
5.
• Market research,
• Academics,
• Resource planning and management,
• Medical and scientific research,
106
• Planning and forecasting,
• Quality control and
• Decision making and
• Reporting.

7. BIBLIOGRAPHY AND REFERENCES


1. Anderson, A. (2000). Using Theory of Change in Program Planning and Evaluation.
Aspen, CO: Aspen Institute. PowerPoint presentation at the annual meeting of the
American Evaluation Association, Honolulu, HI.

107
2. Bennett, C. (1976). Analyzing Impacts of Extension Programs, ESC-575. Washington,
D.C.: Extension Service-U.S. Department of Agriculture.
3. Bloom B. (ed) Taxonomy of Educational Objectives. New York: McKay Charities
Evaluation Services (2002) First Steps in Monitoring and Evaluation.
4. Available at:
www.ces.vol.org.uk
5. Chapel, T. (2006). Reducing Fear and Loathing of Evaluation: Making Good Evaluation
Choices for EPHTN efforts. CDC webinar slide presentation.
6. Available at:
http://www.cdc.gov/nceh/tracking/webinars/jan06/chapel.pdf
7. George T. Doran (1981) “There is a SMART Way to Write Management Goals and
Objectives”, Management Review (November, 1981): 35 – 36.
8. Grundy, S. (1987) Curriculum: Product or Praxis? Lewes: Falrner Press Janner, G.
(1984) Janner on Presentations. London: Business Books Limited
9. Hornby, A.S. (2000) Oxford advanced Learners’ Dictionary. Oxford: Oxford University
Press.
10. Kaplan, S. & Garrett, K. (2005). The Use of Logic Models by Community Based
Initiatives. Evaluation and Program Planning, 28(2), 167-172.
11. Kerzner, Harold (2004) Project Management - A Systems Approach to Planning
Scheduling and Controlling. New Delhi: CBS Publishers
12. Konayuma, S. Gabriel (2007) Evaluation Report of TVET Summit on e-Learning in
Africa. Lusaka: MSTVT.
13. Kotler, P. and Roberto, E. (1989) Social Marketing. Strategies for Changing Public
Behaviour. New York. McMillan, Inc.
14. McLaughlin, J., & Jordan, G. (1999). Logic Models: A Tool for Telling your Program’s
Performance Story. Evaluating and Program Planning, 22, 65-72.
15. Osborne, D. & Gaebler, T. (1992). Reinventing Government: How the Entrepreneurial
Spirit is Transforming the Public Sector. Reading, MA: Addison-Wesley.
16. Poister, T. (2003). Measuring Performance inPpublic and Nonprofit Organizations. San
Francisco, CA: Jossey-Bass.

108
17. Pawson, R, & Tilley, N. (1997). Realistic Evaluation. Thousand Oaks, CA: Sage
Publications.
18. Prochaska, J. et al (1994) Stages of Change and Decisional Balance for Twelve Problem
Behaviours. Health Psychology, 13, 39 – 46.
19. Rockwell and Bennett (n.d.). Targeting Outcomes of Programs (TOP).
20. Available at
http://citnews.unl.edu/TOP/english
21. Rogers, M. Everette (2000) Diffusion of Innovations. New York: Free Press.
22. Rogers, P. (2000). Causal Models in Program Theory Evaluation. In Rogers, P., Hacsi,
T.
23. Petrosino, A. & Huebner, T. (Eds), Program Theory in Evaluation: Challenges and
Opportunities, New Directions in Program Evaluation (87), 47-55. San Francisco, CA:
Jossey- Bass Publishers.
24. Saunders, Mark et al (2006) Research Methods for Business Students. Edinburgh:
Prentice Hall. Standards for Evaluation. E-mail: Mary.
25. Ramlow@wminch.edu
26. Strickland, Thompson (1996) Strategic Management – Concepts and Cases. Chicago:
Richard D. Irwin.
27. Taylor-Powell, E. (2008). Building Capacity in Evaluating Outcomes. Madison, WI:
University of Wisconsin-Extension, Cooperative Extension, Program Development and
Evaluation.
28. Available at:
http://www.uwex.edu/ces/pdande
29. Taylor-Powell, E., Jones, A.L., and Henert, E. (2002). Enhancing Program Performance
with Logic Models.
30. Available at:
http://www.uwex.edu/ces/lmcourse
31. TenBrink, D Terry (1974) Evaluation – A Practical Guide for Teachers. New York:
McGraw- Hill Book Company.
32. Wheeler, D.K. (1980) Curriculum Process. Hodder and Stoughton, Kent.

109
33. Weiss, C.H. (1972). Evaluation Research: Methods for Assessing Program Effectiveness.
Englewood Cliffs: Prentice Hall.
34. Wikipedia (2008) Maslow’s Hierarchy of Needs On line
35. Available at:
http://en.wikipedia.org/wiki/Maslo’s_hierarchy_of_needs
36. Wholey, J. (1979). Evaluation: Promise and Performance. Washington, D.C.: Urban
Institute Press.
37. W. K. Kellogg Foundation (1998). W. K .Kellogg Foundation Logic Model Development
Guide.
38. Available at:
http://www.wkkf.org
39. Wyzocki, K. Robert (2003) Effective Project Management. New Delhi: Wiley Publishing
Inc.

110

S-ar putea să vă placă și