Terms of Reference Consultant to Carry out Independent Evaluation for USDRP

A. INTRODUCTION The Government of Indonesia (GOI) has entered into a loan agreement with the World Bank for the execution of the Urban Sector Development Reform Project (USDRP). The project became effective on January 18, 2006. The objective of USDRP is to improve urban services in the participating Urban Local Governments (ULGs) as part of a first step towards achieving the GOIs long-term goal to develop self reliant cities in the wake of recent decentralization laws. The projects development objective is to improve urban services in at least 13 reform-oriented ULGs through governance reforms, capacity building and priority public investments. This is the first step toward achieving GOIs long-term goal of developing self-reliant cities. The project is funded by loan of US$45 million and PHRD grants of US$5 million as co-financing to support the reform components. The project aims to achieve its objective by: 1. Improving management municipal governance through: (i) enhanced civic participation in key municipal decisions and monitoring/supervision of their implementation; (ii) adoption of extensive public information disclosure policies; and (iii) reform of procurement and financial management practices; 2. Strengthening of municipal institutional capacity to formulate long-term urban development strategies and plans, including local economic development and urban poverty reduction strategies; staff; 3. Building municipal institutional capacity and professionalizing municipal managers 4. Enhancing fiscal capacity by rationalizing expenditures and increasing revenues; 5. Financing priority urban investments that support of improvement urban services, promote LED, accelerate poverty reduction. 6. Implementing, at the central level, the Urban Institutional Development Program (UIDP) for supporting urban reforms and institutional capacity building.

B.

THE URBAN SECTOR DEVELOPMENT REFORM PROJECT (USDRP)

The Project has two components: The Urban Reform and Institutional Support Component and the Urban Investment Component. The objective of the USDRP is to assist ULGs to enhance the quality of life of their citizens through improvements in priority urban services. Fulfilling this objective requires a synergistic three-pronged approach that brings together governance reforms, institutional development and investments in priority urban services. The project adopts this integrated approach as its key design feature, wherein these three arms of sustainable urban development form a virtuous cycle which provides momentum to the program. In line with the project objective, the project envisages reforms in core areas of municipal governancetransparency, participation and accountability (TPA), financial management, procurement and urban service delivery. Participating ULGs will undertake reforms in each of these areas, phase over the period starting from project preparation to completion. Beyond these core reforms, the project also supports other priority reforms at the local level, particularly in the areas of urban poverty reduction and local economic development (LED), and it also strengthens the capacity of the GOI to be an effective policy

maker and overseer in these areas. Lending reinforces the reform process. The project will also finance comprehensive institutional development programs and priority investments for improving urban services in the participating ULGs. Institutional development will focus on strengthening local government capacities in two aspects: 1. Capacity building related to core governance reform areas subproject; and; 2. Generic capacity building based on the unique needs of each ULG, thus laying down the institutional foundation for effective service delivery. Priority investments will cover key urban services identified by the ULG through a participatory process. Apart from their public benefits, it is expected that these investments will provide an impetus to the reform process subproject. The investments will also provide concrete occasions where core urban reforms are applied in the areas of service delivery improvement, selection of priority investments through participatory planning exercise, transparent competitive procurements, and improved financial management and reporting. Municipal Governance Reforms Transparency, Public Participation and Accountability. The ULGs will be required to improve their current practices on transparency and public participation. It is expected that the ULGs will improve the access o f people to public documents, handle public complaints in a more systematic way, and eventually prepare regulations (PERDA) for improved transparency. The ULGs are also expected to promote public participation in decision making, such as in the preparation of the RPJM and the annual local budget, improve the roles o f the Urban Forums by making them more inclusive, and by integrating them further into municipal decision processes and mechanisms, and ultimately issue a local regulation (PERDA) on participation in public decision making. The transparency and participation reforms will be also implemented in the activities related to the safeguard, financial management and procurement on the subproject implementation (the investment component of USDRP). Procurement Reforms. The activities of procurement reforms to be implemented by the participating ULGs have been developed based on the Country Procurement Assessment Report (CPAR) recommendations. It is also based on the assessment made on the selected participating ULGs by the Bank and GOI. The activities of procurement reforms cover the following: 1. Institutional reform by developing a procurement anchor unit; 2. Regulatory reform affecting procurement, including streamlining all existing local procurement regulations; 3. Improving information systems on procurement; 4. Improving controlling, auditing, feedback system at the district level; and 5. Training and capacity building of staff involved in procurement. Financial Management (FM) Reforms. Based on a detailed diagnostic study of existing financial management practices in selected participating ULGs, a comprehensive financial management reform program has been developed. This program addresses: 1. Institutional and legal framework issues; 2. Planning and budgeting process; 3. Budget execution and monitoring mechanisms;

4. Accounting and financial reporting systems; and 5. Capacity building. Other Urban Sector Reforms Over and above the core governance reforms identified above, it is expected that reform-oriented and proactive local governments will undertake further reforms in areas such as Local Economic Development (LED), poverty reduction, environmental management, heritage conservation, local civil service reforms, land use planning, and zoning and other development controls. In particular, urban poverty reduction and LED are two priority areas under the project. Urban Poverty Reduction. The project intends to strengthen ULGs capacity to design and implement sustainable and long-term urban poverty reduction strategies and programs. At the national level, the GOI is developing a long term strategy for poverty and the central government now expects all local governments to prepare Regional Poverty Reduction Strategy (RPRSP). The project will support this initiative as an integral part of cities long term-development strategy. The project supports procurement of national and international experts in urban poverty to support GOI and ULGs in this regard. Local Economic Development (LED). Several laws and regulations give explicit direction on economic development, and the local planning process (the RPJMD-mid term local development plan) creates opportunities for economic strategies at the local level. On their tables of organization, local governments have agencies and institutions with specific economic development functions with line items that can support economic projects. Cities need to develop LED strategies as integral parts of their long-term urban development strategies, based on their comparative advantages and economic and human endowment, and eventually associate urban investment subprojects with this LED strategy. They also need to address a range of regulatory shortcomings, as well as issues of pervasive corruption, rent seeking and other market-unfriendly practices. The project intends to support, through the UIDP (Urban Institutional Development Program), participating ULGs to undertake reforms that will enable them to become efficient catalysts and promoters of LED. Overall, the Urban Reform and Institutional Support Component (UIDP) will be supported by six (6) packages of consultant services as the following: (i) UIDP Management, (ii) UIDP Urban Strategy and Policy Development; (iii) Generic Capacity Building for Local Government, (iv) Transparency, Public Participation and Accountability Reforms, (v) Procurement Reform, and (vi) Financial Management Reform. C. STATUS AND ISSUES OF THE USDRP IMPLEMENTATION TO DATE

Participating ULGs and Subproject/investment Component In its original plan (as stated in Loan Agreement and Project Appraisal Document), the participants of USDRP were 13 ULGs. However, after two and half years of implementation, there are five ULGs left. This high turnover of ULGs was caused by various reasons, among others: (i) non-conducive political environment due to local election (Depok, Sleman); (ii) disapproval from the DPRD at the last minute (Semarang, Yogyakarta, Blitar); (iii) prolonged settlement of arrears (Blitar, Palu, Musi Banyuasin). In addition, Manokwari and Kabupaten Sorong withdrew its participation due to the disagreement with the sub-loan channeling mechanisms.

As of September 2008, there are five participating ULGs and three new ULGs (Palangka Raya, Barru and Sawahlunto). At present, seven subprojects in four ULGs are in the stage of construction process, except a subproject in Kota Palopo. Detailed loan allocation for the subproject component is presented in Table 1. As the Project adopts an onlending scheme, all sub-project investments have to be revenue generating investments, and consequently, the ULGs have proposed markets and terminal.

Table 1: Status of Loan Allocation for Subprojects as of September 2008

NO. KOTA/KABUPATEN SUBPROJECT Original Participating ULGs (with signed SLAs) 1. Kota Cimahi Construction of New Pasar Atas 2. Kota Palopo Construction of Palopo Big Market 3. Kota Pare-pare Construction of Lakessi Central Market 4. Kab. Sidenreng Rappang Rehabilitation of Pangkajene Central Market Construction of Rappang Central Market Rehabilitation of Tanrutedong Market 5. Kab. Parigi Moutong SUBLOAN (IDR 000) 14,086 44,120 41,200 16,220 17,450 9,640 US$ (million)

Construction of Parigi Central Market 21,500 Construction of Toboli Terminal 6,500 Subtotal 170,716 New Participating ULGs (SLAs nearly finalized and signed) 6. Kota Palangkaraya Redevelopment of Kahayan Market 23,370 7. Kab. Barru Reconstruction of Pekkae Market 43,155 Mangkoso Market development 9,612 Palanro Market development 12,970 Subtotal 89,107 Potential New ULGs (SLA not yet prepared) 8. Kota Sawahlunto SLA not prepared Total of signed, unsigned SLA, new ULGs TOTAL BANK FINANCING FOR SUBPROJECTS 388,884 (not including unallocated funds)

18.56

9.68 5.6 33.84 40.27

Remarks: exchange rate is US$1 equivalent to IDR 9,200; Eight of the original ULGs withdrew from the project; Remaining loan (not including the unallocated) will be allocated for new potential ULGs: Kab. Morowali, Kab. Pesisir Selatan and Kab. Solok Selatan

Reforms Component The ULGs implemented reform agenda (SK Bupati or Walikota) with their own resources and capacity. Achievement of the reform agenda varies across ULGs. Almost all ULGs are quite advanced in implementing the reforms in the transparency, participation, and accountability (TPA) area, particularly in handling public complaints and disseminating information. In terms of financial management reform, all ULGs have followed the national regulations (Permendagri 13/2006 and 59/2007) consistently, which basically are in line with the financial management reform agenda as agreed in the USDRP. Similar to financial management reform, the ULGs follow the national regulation (Keppres 80/2003) which is in line with their procurement reform agenda to be supported by USDRP. Nevertheless, overall reform progress in all ULGs is still behind the intended schedule. A systematic and detailed documentation of the reforms for each ULG is weak. Annexes 4, 5 and 6 in AM#6 presents implementation progress of reform components as of to date (available upon request).

Disbursement As of September 2008, the disbursement rate of USDRP to the special account is USD 10.95 million, lagged behind if compared to the estimated disbursement of USD 16.1 million of the PAD. The disbursement level may continue to be low in the future because the project now has only 8 participating ULGs (three of them joined only recently) rather than the 13 ULGs as planned in the PAD.

D.

OBJECTIVES OF THE CONSULTANCY SERVICES: INDEPENDENT EVALUATION

The main objectives of the Consultants assignment are to: 1. Review the overall project preparation at the national and ULG level to assess its contribution to the current situation of project implementation, including quality at entry, screening process, institutional capacity, project socialization, subproject preparation, costs and time for project and subproject preparation, etc. 2. Assess the overall project performance, at least but not limited to the : a) Achievement of project performance indicators and project development objective b) Project management (organization, quality and deliverables of project management manuals, etc) c) Implementation quality of the reform and subproject components d) Subproject procurement and consultants selection e) Project financial management f) Financing scheme and disbursement mechanisms g) Social and environmental safeguard planning and implementation h) Management information system i) Quality of monitoring and evaluation mechanism j) Quality of project supervision In more specific, the evaluation for the above aspects should be looked from cross cutting perspectives, at least from the following angles: a) Key factors affecting the project implementation and achievement of outcomes b) Main issues, hinderance of project/subproject preparation and implementation c) Lesson learned d) Impact of the project e) Sustainability f) Strategy to improve project implementation and corrective inputs or actions to enhance project capacity in achieving its development objective 3. Based on (2), assess to what extent the project needs to be restructured a) Justification (rationale) to project restructuring, e.g.: assessment of risks to achievement of project performance indicators b) Area of project restructuring c) Recommendation (workable step) for actions The results of the evaluation will be used as inputs to the Banks mid-term review mission and as measures to assess to what extent the project has achieved their intended objectives as outlined in the Project Appraisal Documents and Loan Agreements as well as Grant Agreement, particularly the Results Framework and Performance Indicators, respectively. It will also be used for inputs and give future perspective to prepare USDRP-2.

E.

SCOPE OF WORKS

Scope of works of the Consultant is the following, but not limited to: 1. ASSESSMENT OF THE OVERALL PROJECT PERFORMANCE a) Desk review to overall documentations of USDRP, including: Loan Agreement (LA), Grant Agreement (GA), PAD, Aide Memoires, Reports (progress, project management report, monitoring and evaluation report, and LARAP), project management manuals (PMM), reform agenda and its update, procurement and consultant selection documents, audit report, No Objection Letters (NOL), and other relevant documents. b) Assess performance and functioning of the USDRP organizations: IMSC, the Executing Agency, the Bank, CPMU, PPMU, MTAS, PMU, PIU, PAU, focal points for the Financial Management (FM) reform and TPA, urban forums, safeguard monitoring team (SMT), traders relocation team, and other relevant stakeholders c) Review effectiveness of the selection process of participation in USDRP, i.e: documents preparation, SLA processing, application of selection criteria, etc. d) Assess quality and deliverables as well as the usefulness of the project management manuals (clarity, completeness, timing, etc) e) Assess effectiveness and timeliness of the subproject procurement and consultant selection process and results f) Assess performance of the project financial management (disbursement, organization, budgeting process and procedure, reporting, etc) g) Assess quality and implementation of social and environmental safeguards h) Assess quality of construction work and results i) Assess quality of the implementation and achievement of reform components j) Review the capacity building efforts for project management and reform components implementation k) Assess the quality and effectiveness of management information system l) Assess quality and implementation of supervision, monitoring and evaluation mechanism m) Assess the management plan for the investment financed through USDRP n) Assess sustainability of the project: impact to the project beneficiaries, management of subproject, management of post-construction of subprojects (operation and maintenance adequacy and capacity, financial management, professionalization of the management), etc o) Identify lesson learned for all aspects within the project implementation aspects 2. ASSESSMENT TO THE FUTURE PROJECT RESTRUCTURING a) Review the overall performance of USDRP against its performance indicator and achievement of development objectives b) Assess area or risks to achievement of performance indicators and development objectives c) Assess key factors affect the project in achieving performance indicator and development objectives timely d) Provide recommendations to the project adjustment or restructuring, including area of adjustment and its rationale e) Provide recommendation (workable steps) for further actions F. IMPLEMENTATION OF THE ASSIGNMENTS AND DELIVERABLES

The Consultant will carry out the assignments for about 3 months, starting December (the latest). Week 1-2 of Assignment During the first two weeks of assignment, the Consultant will carry out:

1. 2. 3. 4.

Desk review to all documentations of USDRP Work plan development and field-visit preparation Presentation of Work Plan and draft outline of Report Interview with the central-level organizations of USDRP (IMSC members, the Bank, the Executing Agency, CPMU, MTAS, BPKP, etc)

Week 3-6 of Assignment The Consultant will carry out field visit to the participating ULGs and former participating ULGs (Kabupaten Sleman, Kota Yogyakarta and Kabupaten Manokwari) and perform the following: 1. Desk review to all documentations of USDRP at local levels 2. Interview with the provincial and local levels organizations of USDRP that related with project management: PPMU, PMU and PIU 3. Interview with the local level organizations of USDRP that related with the subproject components: contractors, supervision consultants, tender committee, observers, beneficiaries and future management body for subprojects. 4. Interview with the local level organizations of USDRP that related with the reform components: PAU, focal points for FM reform and TPA, related departments (BPKD/Dinas Keuangan, Bawasda, etc) 5. Collect relevant secondary data related to subproject and reform components implementation and achievement Week 7-8 of Assignment 1. Prepare draft report and presentation materials 2. Carry out presentation of the findings and recommendations to the relevant stakeholders (IMSC, the Executing Agency, CPMU and the Bank) for inputs and feedback Week 9-10 of Assignment 1. Finalization of report based on inputs and feedbacks gained during the presentation 2. Submission of final report to the Executing Agency 3. Submission of all documentation collected during performing of assignment

G.

REPORTING

The Consultant team has to prepare: 1. A work plan, to be submitted 7 days after contract signing. The work plan mainly consists of evaluation framework, evaluation methodology, clear division of tasks and responsibilities among evaluation team; time-frame and schedule; detail planned activities, output indicators; questionnaires, and outline of draft report. The Work plan will be presented to the Executing Agency, CPMU and the Bank for feedback. 2. A draft Report, to be submitted at week 7 after contract signing to the Executing Agency, CPMU and the Bank. Presentation of the draft report is expected to be held two weeks after the field visit. The Consultant will be required to present the draft report to the Executing Agency, the CPMU and the Bank for feedback and improvement. 3. A final report is expected to be received by the Executing Agency, CPMU and the Bank a week after presentation of the draft report. Final report should be completed with all relevant attachments. The Consultant team submits all reports to CPMU with copy to the Bank, both in hardcopy and softcopy. The report should be completed with all evidences found during the consultants assignment and outputs of the consultant.

H.

PAYMENT

Payment will be divided in three tranches: first payment, which is 10% of the contract amount will be given by the contract signing; second payment, which is 60% of the contract amount will be transferred after the submission of the work plan; and the final payment 30% will be given after the submission of the final report acceptable to the CPMU and the Bank.

G.

STAFFING

In order to implement the assignment, the Consultant Team composition should be the following: Team Leader (Local Institutional Development Specialist) 3 Man Month 1. Minimum Master S2 degree, with background in program (project) planning and management, having at least 10 years experience in project management, monitoring and evaluation for the government and loan funded projects, and at least 7 years as Team Leader. 2. Fluent in English and Indonesian, both oral and written, having capacity to produce reports in English with excellent quality. 3. Having ability to organize and implement an evaluation study: designing evaluation study framework to field visit and reporting. 4. Having inter-coordination skills to work with the government agencies, and consultants within the project, for implementing the assignment. Co-Team Leader (Urban Planner) 3 Man Month 1. Minimum S2 degree, with background in urban planning or development studies, having at least 10 years experience in monitoring and evaluation of the government and loan funded projects, and 5 years minimum experience in statistical and data management tasks. 2. Fluent in English and Indonesian language, both oral and written. 3. Having ability to organize and implement an evaluation study starting from designing evaluation study framework to field visit and reporting. Procurement Specialist 3 Man Month 1. Minimum S-2 in Civil Engineering or relevant education, having experience in government and loan funded project with specific task in public procurement for at least 5 years. 2. Familiar with national and the Bank procurement regulations. Project and Public Financial Management Expert 3 Man Month 1. Minimum S-2 degree in public administration and/or urban planning, having at least 8 years experience in monitoring and evaluation the local government and loan-funded projects, having at least 5 years experience with specific tasks in local financial management. 2. Familiar with the regulations related to the local public financial management. Municipal Engineer 3 Man Month Minimum S-1 in Architectural or Civil Engineering, having experience in at least 10 years in construction, construction management and supervision projects.