CAFOD – Accompanier Indonesia – Quarterly Review – 071220

1. Purpose and method of report

The purpose of this review is to carry out a review of CAFOD’s local partners’
programmes, to assess factors affecting performance during programme partners
implementation or accomplishment to date, to assess the role and responsibility of
Accompanier as the programmes of partners roll out, and by these provide
recommendations to CAFOD for future discussions and arrangements. I have been
given lots of spaces to be creative and independent to monitor, evaluate, and provide
assistance to develop a sound and logic processes with all partners to define steps
towards sustainability for future. The scope of this review is only limited to this
purpose and therefore does not reflect what have been achieved in performing other
duties as described in my job description (please refer to my Terms of Reference for
the Accompanier position). In doing so, I simplify this into 4 major components:
 Identify lesson learned to date;
 Provide guidance for further support for enhancing the program’s
effectiveness, new initiatives and achieving objectives for the remainder of
the program;
 Determine the needs of capacity building for partners; and
 Suggest next steps.

This document addresses these broad aims of review. The review should be per 3
months starting from 20 August to 20 November. However, given the time to in-
country induction and changes of plan and strategy to meet CAFOD and partners
needs have caused for delays. During this phase I was able to conduct many
comprehensive discussions with partners’ major staffs and stakeholders. I also
undertook a range of visits to project sites in which some of them are with potential
co-funding agencies and interview a range of beneficiaries and local government. I
have attempted to be collaborative, consultative and constructive in its approach to the
review task.

I acknowledge the constraints under which the programmes have been implemented,
notably the requirement to be flexible and adaptive as needs emerge to respond, and to
as quickly as possible develop the capacity of local partners to roll out their activities.
I thank the London team from the Programme Manager, Program Support Officer and
Program Administrator for the frank and open way in which progress and issues were
discussed.

2. Description of activity performance

For he purpose of this review the baseline documentation for CAFOD’s local partners
are as follows:
 Partners summary
 Program preparation document as originated and approved by CAFOD such
as grant agreements which formed the basis for commencement of the
programmes.
 Proposals (some) to be the basis of monitoring and measuring whether the
programmes have reached their objectives.
All CAFOD’s partners have largely followed the intention of what was expected to
perform. This needs qualification because some activities have been late and some
have not yet been entirely completed to plan. It is also noted that the original
approved documents were unclear to define strategies, processes and practice. These
have been lacking and as implications the effectiveness of CAFOD’s mechanism has
suffered accordingly.

Furthermore, with no monitoring and evaluation framework (or logical framework)

produced and finally approved for partners programmes, there are simply no
verifiable performance indicators to measure achievements. There was a draft of
monitoring framework for each programme prepared in MSGA. But this was not
followed up with revision and refinement at later stages and does not now fit the
current components of some partners programme.

3. Factors affecting performance

3.1. General Comments
Discuss any issues or problems which are expected to affect programme objectives as
well as any particular strength of partners including organisation and management
structure of their programme. Recommend ways to build on identified strengths and
address identified constraints in order to enhance overall effectiveness. This section
summaries the strengths and weaknesses of partners to date.

Strengths
Most programmes have clearly been able to adapt as it has progressed and given
appropriate technical, human resources and institutional support. They can certainly
do so again. They had good and established capacity to perform and while it took
some time to establish this, the activities now are in the full swing.

With revisions of strategy – and I recommend for additional expertise – the

programmes have a management structure and personnel at management level who
are able to carry out revised strategy if they are prepared to do so. There is already a
basis of organisational system which can be built quickly and used as a basis of
continuing the works towards sustainability. There are also managers and directors
who are making positive impacts and if there are additional staffing support, this will
provide a major opportunity to develop organisations in terms of programs and
fundraising strategy.

Most programmes now have a preliminary M&E framework which needs final
revisions based on changes recommended through accompaniment and their approval
as appropriate. This will be used as a management tool for ongoing assessment of
progress by CAFOD and the partners themselves although generally it has not been
able to capture qualitative data, for example standard of gender disaggregation.

Weaknesses
To date there has been no coherent implementation strategy in place and operational.
This includes a lack of strategy to ensure social inclusion and promote gender equity
and its manifest in an unresponsive approach. The programmes would also benefit
from a more flexible approach to what they offer through their core activities. There
has been insufficient attention to principles of ‘do no harm’ in working with
beneficiaries i.e. communities and in general a lack of understanding of and therefore
commitment to community participation.

A coherent M&E framework bringing together a monitoring capacity at a strategic

level has only just emerged for the program. This is an indication of a program that
has been developed with –to date – no higher level of monitoring capacity.
Most programmes are suffering from a slow start and associated difficulties to move
from emergency type intervention to longer-term developmental approach.
Insufficient attention has been paid to date to capacity building.

The setting up for contractual agreements between CAFOD and partners take time
and it meant that there is some time before the programmes are ready to be
implemented. In resolving this issue, many partners had used other party funds to
implement their program. As implication there are inconsistencies in partners
timeframe for programme implementation.

3.2. Management
Assess the efficiency and effectiveness of management processes, procedures,
strategies;

Administration and general management - Partners

It was hard to classify administrative procedures and processes for the programmes
i.e. it is likely to be on-off reasonably efficient as do financial management
procedures and processes. In regards to this matter, it is considered significant at this
stage for both CAFOD and local partners to conduct outsourcing evaluation and audit
to each programme. This should be followed by a more strict requirements by
CAFOD and continuously capacity building for partners.

It is noted that reallocation of funds and/or fraud have been the problem, but I
understand that procedures within CAFOD have been tightened up. Systems have
now been developed to report and act on financial requirements (as discussed to be
using MANGO guidelines).

The division of program staff into the program activities are practically management
approach. However, when executing complex program related activities it sometimes
either confusing or inappropriate. Perhaps this is because of the lack of understanding
that the program could adapt necessarily to the needs and therefore be amended, even
it has not been mentioned at the approved contractual agreement.

The effectiveness of management processes, procedures, and strategies can only be

measured by the effectiveness of program outputs and outcomes. To the extent that the
programmes have achieved what are outlined in their reports, procedures and
processes are reasonably effective. However I noted the general lack of clarity with
the strategy and recommended to start defining necessary capacity building through
trainings, cross visits, and adoption of civil society i.e. community development
strategy. This will significantly boost overall effectiveness.

Most programmes have lacked coherent strategy overall – as indicated by the lack of a
logical framework. While it is understood that the partners are young in emergency
and development works and have always been intended to be responsive to needs, the
reality is that programmes coherence between one defining document and the next has
been hard to track. This is not good for Accompanier and partners in trying to
maintain clear objectives for our work especially in program that has been intended to
adapt and change.

The program Executive Committee, which usually consisted of the Director, the PM,
and Finance Department has been the group who made final decisions on program
funding for individual activities (unfortunately this has been lack in gender balance as
well). It met regularly and invited staffs of activities to attend along with the manager
responsible for the programme. It has not to date included representation from outside
the management structures i.e. local government, major local partner network. This
implicates in exclusive decision making and lays the program open to criticism or at
least debatable. One option to consider is to devolve major responsibilities in decision
making to directors and managers and ensuring that outside independent expertise is
obtained for appraisal and evaluation of proposals. Relevant team managers should be
able to assess from technical perspectives based on clear and transparent criteria. The
PEC would then be simply required to provide final sign-off to ensure that proposals
accepted are within overall program objectives/parameters.

Recommendation 1: It is noted that financial management appear to be sound

but for sure not high quality as basic requirement for financial system such as
activity coding has not been met properly. It is suggested that consistency of
documentation i.e. procedures, operating manuals and contracts will assist.

Recommendation 2: There is a need to have an audit for partners who had not
done so. There is an urgent need to conduct financial training at the time
disbursement of funds is made.

Recommendation 3: Internal processes including management information

systems need to be reviewed to improve efficiency. For example appraisal of
proposals should be undertaken by staff with relevant expertise along with
external/independent inputs with final sign off by PEC

Contractual issues – CAFOD, Accompanier and Partners

In light of current contractual agreements, identify issues and critical factors that have
influenced or likely to influence effective management

The split contract model used in Indonesia programs has had its problem. There are
other agencies/instances where variants of this model had problems and required
modification and change.

The CAFOD-Accompanier-Partners model has created a three-way relationship

between CAFOD and two different contractors which has been at times difficult. Staff
of program partners and other people from outside who are associated with the
program are confused as to who is responsible for what. There needs to be more
clarity in the definition of the roles.

CAFOD intended that the Accompanier should act as its hand for related paperwork
activities, its mouth for bridging communication gaps and its eye to develop future
programming needs, primarily because of option to be unregistered in Indonesia but at
the same time need to fully keep track of partners program. It is understood that this
constraint still exists, but the role of Accompanier is sometimes not clear. The reality
is that Accompanier has been more aligned to work on a daily basis with CAFOD
partners and communications with CAFOD have suffered. The Accompanier has been
involved in strategy development to a considerable extent which at times not
consistent with the development of monitoring capacity. Under these circumstances
impartially is difficult.

The Accompanier provides CAFOD with weekly reports, but these are not structured
along the same lines as the programme partners’ reports to CAFOD. Therefore these
reports are more on day to day observations, documenting issues as they arise as seen
by Accompanier, but do not provide a systematic point or cross-check for CAFOD
that program activities are on track. This will lead to question of the effectiveness of
the weekly reports by this nature. Another option to consider is to utilise a reporting
format that is in line with monitoring the program according to agreed activities and
indicators. This underlines the monitoring role.

The Accompanier is required to act as a conduit of CAFOD in occasions to discuss

issues and assist partners in developing a sound programmes through discussions,
observations, monitoring and capacity building assessments over which the
Accompanier has no financial delegation. These requirements are by no means
relatively unrealistic as situation reports and all contractual decisions still require
negotiation and final decision between CAFOD London team and partners.

The Accompanier takes part in the program partners meeting, providing CAFOD
vision and strategy for implementation (including funding strategy). However,
Accompanier is not required to provide any formal verification that activities have
been completed. This is likely not appropriate if Accompanier is to monitor on behalf
of CAFOD. Strategic development in the program should become the major
responsibility of the partners that will implement. And Accompanier should in
correspondence take a more active role in ensuring from CAFOD’s perspective that
activities are being achieved as planned by partners and agreed by CAFOD and in
identifying progresses or constraints.

Recommendation 4: CAFOD and Accompanier may need to review and agree

the expected role and value of Accompanier and its relationships with partners.

Recommendation 5: Strategy development should now be the responsible and

role of partners with CAFOD providing quality assistance through capacity
building

Recommendation 6: Accompanier role should be to provide clear and accurate

monitoring information for CAFOD and to maintain close and frequent
communication to ensure that CAFOD has the information of the programmes,
their progress and problems they require.

Recommendation 7: Accompanier can continue to provide a second opinion on

activity approval to both CAFOD and partners, but only if this also means
monitoring activities during implementation.
3.3. Quality relationship and communication
Assess the effectiveness of communication mechanism among CAFOD, partners, and
different stakeholders. Evaluate the quality of communication and suggest ways to
improve performance.

There appears to be general lack of responsiveness to communicate effectively from

partners to CAFOD on issues affecting programmes. In many cases major important
documents were needed in a short time limit by CAFOD while in the other hand most
partners were slow in responding this. This might be an indication of lack of
understanding standard procedure/operation/guideline. Even I tried to avoid a position
game i.e. I am donor, you are partner and develop facilitative role communication
style but sometimes the condition was not as what was expected.

The programmes need processes where inputs that are provided from various
stakeholders are noted and deliberated. Reasons for acted and not acted need to be
documented and timely meetings held to discuss these issues. It is clear that in some
of meetings with local government and non government partners, input has been
provided but not always acted upon by the programme.

This lack of responsiveness to input from partners and stakeholders has been a
problem which has impacted negatively to partners performance. I was made aware of
other donor funded projects to address the degrees unaware by our partners,
networking partners (mostly in districts) declining to work with core partners. Perhaps
this illustrates a management strategies developed in isolation and in some cases not
incorporating other inputs. It is understood that in this regard, partners has taken
recent steps with programme management to improve relationships and this is
welcomed.

One key to improvement in the area of partner communication is a wider sharing of

lesson learned through the program. Increased sharing of major planning documents
and reports would also be beneficial, as also regular meetings of partners to share
information on a round table basis.

Recommendation 8: There is a need to develop a Standard of Operation which

will be discussed, approved and socialized to partners to anticipate timely
manner implementation.

Recommendation 9: There is a need to develop processes for improved

communication and sharing lesson learned with and among partners. Meetings
should be held to bring partners together to share lessons and to discuss and
agree on necessary amendments to activities accordingly. This should at a
minimum include partners which have contractual agreements with CAFOD and
government agencies with which the programmes most closely work.

Strategic Linkages – District and Provincial governments in Aceh and other

donors

Most programme partners have played a useful part in local coordination efforts by
government, and have taken considerable efforts to develop contact with thye primary
government and BRR counterparts. I was able to confirm this through some meetings
with regional government and evidence of activities which involved the government.

Partners have participated in working groups conducted regularly in Aceh. However,

contacts with other donors have been largely at the local level, with CAFOD initiated
co-funding agencies primarily undertaking the efforts. I did not find a particularly
major effort at donor harmonisation from most partners beyond the above mentioned
earlier contribution to the cluster group.

Some partners are benefited with the condition where many donors are in the
preparation of exit strategy and forced them to find local partners to spend the
remaining tsunami funds.

Of major importance is the need to proactively interact with other donors and have
donor liaison officer to provide and communicate quality proposal/objectives/reports
and maintain the interaction. In this regard, it would be interesting to provide an
independent donor liaison to work on behalf of CAFOD to assist potential partners to
interact and negotiate with other donors rather than have only one dedicated donor
liaison for one partner.

Recommendation 10: Programme partners need greater contact and

coordination with other donor programmes both locally and more widely. This
could be through existing mechanism like cluster working groups as well as
individual meetings with key donors.

Recommendation 11: Providing and utilising donor liaison officer would be

highly useful. Locally many partners would benefit significantly from a more
consultative approach to coordination with governments and other donors.

3.4. Gender and social equity issues

Assess and evaluate partners program to identify components of programs which
concerned gender balance and social inclusion

There is no strategy to promote active involvement of the broader community, let

alone traditionally marginalised, in positions other than beneficiaries. No strategies
for social inclusion, particularly to woman, have been developed, learned and
adopted. The focus of livelihood and gender programmes for example, has been
limited to the poor or women being identified (generally by community leaders) to be
beneficiaries, rather than engaging them in shaping the program and determining their
own priority needs.

Only one of the project partners (Beujroh – note that this partner works for gender
mainstreaming) has developed approaches to ensure women’s participation. Other
partners have not taken advantage of the opportunity to develop and resource a
strategy to ensure women’s participation in decision making, and provide training to
staffs in strategies to actively promote equity. This may be because of a lack of
understanding of gender issues in partners. Similarly, there has been no coherent
strategy for involvement of marginalised people. This is most apparent in cases where
social jealousy have occurred in targeted areas ( and this is not only to the CAFOD’s
partners program alone).
There appears to be a growing awareness and interest in partners for a strategy for
social inclusion. Given sufficient resources and management commitment, the
program will be more pro-active in this regard in the future, especially if steps can be
taken to provide some practical mentoring in this area.

Recommendation 12: There is a need to develop a coherent strategy bringing

together programmes based on community development approach. This should
be understood and internalized by the whole management team and quality
capacity building provided for all staff and relevant partners. It should be
discussed and disseminated among the programme team and major stakeholder
i.e. government and other donors.

3.5. Capacity Building

Assess what has been the ‘why’ of not having adequate capacity building and what
type of capacity building is needed the most.

It is clear that many programmes have suffered from the lack of capacity building
strategy and as a result insufficient attention has been provided to capacity building
across all aspects of programmes. There are 3 obvious areas where capacity building
applies: Organisational management including financial, fundraising in terms of
proposal and report writing and business plan, advocacy and communication
techniques.

Capacity building strategy developed for partners should ensure that the needs are
identified and prioritized as an integral part of the process, it is important in this
regard to assess available local capacity building service providers. In relation to this,
it may be important to identify different approaches for men and women.

Training will be a vital aspect of capacity building strategy, but it also hard to assess
whether what has been learned in class had benefited and practiced in field. The other
conservative ways to consider are consultancy, cross visits, short course, in which the
ideas are to learn and then practice. A more extreme method which is largely utilised
especially for micro finance project is an approach of practice into learning. It is
based on the trust to include some outsource professional to manage a programme
along with 2 layers of internal dedicated staffs which will be their shadows and
involved in all day to day management, operational and field implementation. This
process is assumed to be creating an enabling environment towards sustainability.

Recommendation 13: The upcoming approved programme should ensure that

capacity building needs for partners are identified and prioritized as an integral
part of the process. It is important to assess available local capacity building
service providers which may already available through information from sister
agencies and other agencies like UNDP, GTZ, etc.

Recommendation 14: There is a need to identify the most efficient techniques of

capacity building for partners. A vital component is of course training and
workshop. However, given the nature of difficulties to assess how applicable
lessons in class against field implementation, it would be highly useful to combine
training with other techniques such as cross visits, consultancies, short courses or
outsourcing professionals managing programmes.

3.6. Monitoring and evaluation

Assess the adequacy and usefulness of programmes monitoring and evaluation
framework for data collection and analyses.

M&E arrangements for the program have to date been poor, at least at the strategic
level. While it is noted that activity monitoring level are largely in place and
operational, the M&E system has not provided any real measurement at the goal,
outcome and component objective level. It has been hard for Accompanier to easily
understand from regular reports where progress has been made. There obviously a
need for the recent work on the M&E system. This also includes the urge to ensure all
monitoring data is gender disaggregated and the key social inclusion and capacity
building data is required.

The M&E framework as a management/implementing/reporting tool at least at the

strategic level has just been introduced to some partners and still needs to be finalised.
I noted that the activity level monitoring has been established since early in the
program implementation and is reasonable quality. It is anticipated that bringing
together all of this into the M&E performance framework will be highly beneficial for
program management in monitoring further progress and in providing a baseline for
evaluation.

Recommendation 15: There should be the same guideline of M&E system

discussed and utilised in all programmes based on CAFOD requirements. This
will include the creation of framework depending on decisions made as a result
of strategy changes.

Recommendation 16: If qualitative reporting is necessary, there is a need to

ensure all data is gender disaggregated and to pay attention to monitoring and
recording key social inclusion and capacity building activities.

3.7. Reporting
Assess the quality and discipline in providing reports to CAFOD

It is clear that reporting up to date has been neither consistent nor clear and not
particularly high quality. We found it generally hard to track issues through the reports
submitted and discovered inconsistencies in terminology within the documents. We
had expressed some concerns about these aspects and also tardiness of the reports. It
should be clear from the very beginning of formats and requirements in reports and
hope that these will resolve former problem.

Recommendation 17: This review commends on the steps already been taken to
rectify known problems of poor reporting. The key will be to ensure that
CAFOD and program staff have a clear picture of progress and issues from
regular reports. It is further recommended that program partners (again
partners who had the contractual agreements and government agencies with
which the program works closely) receive both working plan and
quarterly/semester reports once they are finalised as this will help them to put
their work into context. They should also be given the results of surveys and
evaluations as they become available and there should be opportunities for
documentation to be discussed.

3.8. Risk management

Assess risk management factors which should have been included in program
documentation

I am aware that there should be a document comprising the steps of risk management
within the value of CAFOD principle. However, I have not found this in all discussion
with partners. If this is a consideration then I would assume that we need to make an
appropriate mechanism to it. In a broader sense the programmes has not been
enthusiastic in stipulating the risk management. In any case, more thoughts should be
given to understand a more holistic approach to understand strategies and objectives
of the programmes and then further developing the risk framework. This means
tracking of risks, removing ones that are no longer appropriate and incorporate new
risks as they emerged. And this should be related to existing efforts at continuous
improvements.

Recommendation 18: The format and methodology for risk management need
urgent consideration or appropriate revision and should be utilised in overall
partners program. Risk management methods should utilise a more holistic
approach to understanding strategies/ objectives of the programmes. This should
result in a revised risk framework that accommodates tracking of risks,
removing ones that are no longer appropriate and incorporating new risks as
they emerged.

Recommendation 19: CAFOD and partners should jointly develop a risk

management matrix to identify, mitigate, and determine who is responsible for
managing the liability risk for each partner program. CAFOD needs to make a
decision on this for future activities.

4. Lesson Learned
The observations and recommendations given above all clearly reflect lesson from
this year operation of CAFOD’s implementing partners in Aceh. However, in a wider
sense there are lessons (largely relating to design and granting) that can be considered
by CAFOD in making responses from emergencies to development phase such as
occurred in Aceh province.

1. Approaches to grant making under emergency circumstances should by nature

be very different to approaches for more normal longer term developmental
circumstances. While being flexible in the early focus on emergency response,
we would need to develop a mechanism of developmental circumstances
which will be more complex and demanding in creating and analyzing success
factor to reach our CAFOD 2010 vision.
2. A strict logical frame approach to design may not always be desirable, but in
the case where overall objectives are reasonably tightly focused and where a
facility type of program is not being proposed, a logical framework and
ensuring monitoring or performance matrix is highly useful. This will allow
more consistent reporting and monitoring to be established early in
 implementation. It will also assist to establish a stable program design at an
earlier stage.
3. Partners often have form alliances and joint ventures to undertake complex
developmental activities. There are 2 implications of this; First, at the
technical assessment stage, prior to awarding grants, the exact relationships,
proposed roles and responsibilities, and contributions to achieving goals,
objectives and outputs need to be thoroughly examined. Second, In a program
of the nature of such as APF, if a livelihood dominated partner is appointed to
be the leader of federation, it is to be expected that the strengths will be in
livelihood. The community participation, civil society consolidation or
constructions are likely to receive less priority.
4. Community participation, social inclusion and gender are vital consideration
in the program and strategy development. It should not be left to a later stage.
It may be better to at least firmly outlined in original scoping document and
appraisal process should examine these aspects carefully. Partners should also
be required to report on these aspects during implementation.