Timor-Leste Quality Education for All – 3rd Cycle Mathematics and

Science Curriculum

TERMS OF REFERENCE

To conduct and report an evaluation of the Project entitled “Timor-Leste

Service Contract Quality Education for All – 3rd Cycle Mathematics and Science
Curriculum”
Contract Period 13 October – 10 December, 2017
Targeted Locations Timor-Leste

1. BACKGROUND
UNESCO Jakarta Office Jakarta been implemented a four-year project (2014-2017) entitled “Timor-
Leste Quality Education for All – 3rd Cycle Mathematics and Science Curriculum” with a close working
partnership with the Ministry of Education through the National Commission for UNESCO in Timor-
Leste.

The ultimate vision of this project is to promote sustainable development in Timor-Leste by

strengthening the education, environment and economic sectors. Under this vision, the goal of this
project is to improve the quality of education in Timor-Leste through the development of a national
school curriculum that is both indigenous and in line with international standards. In order to achieve
this goal, the major objectives are:
1) Consolidate the mathematics and science educational national curriculum for Third Cycle
(Grades 7-9) in cooperation with the Ministry of Education, Timor-Leste;
2) Revise and finalize the syllabus for mathematics and science subjects based on the current
curriculum and textbooks to provide a clear roadmap to teachers at Third Cycle level on what
to teach and how to teach;
3) Develop the Third Cycle mathematics and science teachers guides linked to the revised syllabus
that contain hands-on activities that are possible to carry out with simple available materials
and teaching methods;
4) Develop workbooks for students that adhere to the revised syllabi and incorporate practical
activities;
5) Develop student assessments based on the revised syllabus and practical activities for Grades
7-9, and;
6) Train mathematics and science teachers at all Third Cycle schools on the revised syllabus and
the practical teaching manual.

2. PURPOSE
The overall purpose of this final evaluation is to determine the relevance and fulfilment of objectives,
development efficiency, effectiveness, and sustainability of the four-year project. The evaluation also
aims to inform programme stakeholders on the status of implementation, output delivery and
progress towards its objectives and to generate recommendations for improvement for potential
future projects in the framework of a new framework agreement. More specifically, the objective of
the evaluation are:

Page 1 of 4
 1) To assess the efficiency of output delivery and related achievements and challenges in
implementation of the project to meet its objectives under the ultimate vision;
2) To assess the effectiveness in achieving the defined objectives and to what extent conditions
are put in place that facilitate sustainability or a longer term impact, and;
3) To review the project’s strategies and to identify opportunities, challenges, good practices, and
lesson learnt that will be useful for strengthening and enhancing similar interventions for the
design of the potential next phase of the projects.

The evaluation shall provide recommendations for improving the programme’s working methods and
processes; and in relation to project continuation or closure and the criteria for selection of new
projects.

3. SCOPE
The evaluation of the project requires a structure consisting of certain criteria depending on the
resources of evaluation such as relevance, efficiency, effectiveness, impact and sustainability. The
following questions are indicative and shall be refined during the inception phase of the evaluation:

1) Relevance
 To what extent are the project in Timor-Leste and outputs aligned with UNESCO’s
organizational mandate (as defined in UNESCO’s Medium-Term Strategy for 2014-2021
(document 37 C/4) and the associated UNESCO quadrennial plans and biennial budgets
(Programme and Budget (C/5) as well as its global priority Gender Equality’?
 To what extent are the project’s activities and outputs aligned with the needs and
demands of key stakeholders and target groups in Timor-Leste?
 How relevant have the project been as instruments in the framework of education reform
initiatives and development policies in Timor-Leste?
2) Efficiency
 What have been the key outputs of IFIT projects in terms of research, awareness raising
and capacity development?
 Is output delivery on schedule? Are there any (unforeseen) delays? What have been the
main challenges in the delivery of these outputs?
 What has been the nature and quality of interaction and collaboration at the national and
regional levels?
3) Impact
 What have been the key outcomes resulting from the project?
 What are the main factors that have facilitated or obstructed the manifestation of
outcomes?
 Which activities and use of the produced output were the most/least effective in
contributing to the project’s objectives and why? What are their common quality aspects
and challenges?
 Are adequate monitoring tools and mechanisms in place, and functional? Do they allow
communication and exchange of lessons learned between the different projects?
 Are the established quality assurance mechanisms adequate?
4) Sustainability
 What are the main factors that facilitate or threaten the financial, political and
institutional of the outcomes of the project?
 More specifically, what are the main challenges in safeguarding the sustainability of
regional institutional partnerships through the project?

Page 2 of 4
  What was the potential for mobilizing further partners and donors and synergies/
complementarities with initiatives undertaken by other development agencies and
partners?

4. METHODOLOGY
To build blocks of evaluation of this project, the employed methodology includes:
1) A self-assessment of the project completed by the relevant project team, and the responsible
programme specialist will provide additional data as necessary;
2) A necessary additional desk study consisting of reviews of multi-dimensional documents; for
instance, strategy documents, operational procedures, meeting minutes and presentations,
mapping of activities, project documents, periodic progress reports; mission reports, UNDAF
in Timor-Leste, UNESCO’s Medium-Term Strategy for 2014-2021 (document 37 C/4), etc.;
3) Reconstruction of an Overall Intervention logic/Theory of Change for the project and how the
project contribute to it;
4) Questionnaires and surveys addressed to various stakeholders (e.g. Ministries, National
Commissions, universities, researchers’ networks, working partners, etc.).
5) Interviews, discussions and meetings with project responsible officers, as possible in –operons
or via teleconference (Skype or telephone). Interviews with representatives from the Donor
(i.e. Korea International Cooperation Agency),the UNESCO Office Jakarta, project partners
(such as Ministry of Education National Commission for UNESCO in Timor-Leste) and
beneficiaries (school teachers and students in Timor-Leste) to be arranged either on-site at
the UNESCO Jakarta Office or via teleconference;
6) Field visits to meet with the local stakeholders, beneficiaries and partners involved in the
implementation of the project, and;
7) Participatory workshops to steer the evaluation and to discuss findings and recommendations.
(in person or via skype).

5. ROLES AND RESPONSIBILITIES

UNESCO Jakarta is responsible for the overall management of the evaluation and quality assurance of
the deliverables in coordination by the Director of the UNESCO Office in Jakarta. The established a
reference group of the project will include the Director of UNESCO Jakarta, the responsible
Programme Specialist of the project, and staff involved in the project and a representative from the
UNESCO Internal Oversight Service (IOS). UNESCO Jakarta in consultation with UNESCO BSP and IOS
will:
1) Discuss the final evaluation approach and methodology with the evaluation consultant/
evaluation team;
2) Ensure that the evaluation team has access to and has consulted all relevant information
sources and documents related to the project;
3) Discuss and comment on notes and reports delivered by the evaluation team, and;
4) Assist in providing feedbacks of the findings, conclusions, lessons and recommendations from
the evaluation.

An external evaluation team will be responsible for the overall conduct and delivery of the evaluation.
The methodology of the evaluation and detailed work-plan will be specified in a succinct inception
report. Finally, the evaluation team will be responsible for drafting and finalizing the final evaluation
report.

Page 3 of 4
6. QUALIFICATION AND EVALUATION TEAM
The contracted individual/institution or organization shall have the following profile:
 Expertise in evaluation and development assistance projects.
 At least 10 years or more experience and engagement in programme and project
evaluation of relevant to policy making, to include international experiences. He/She team
will also have a strong record in leading and/or conducting evlaution.
 Advanced university degree in Ph.D. in education/natural/social sciences/evaluation-
related fields.
 Proven experience in working with United Nations agencies, international non-
governmental organizations, or similar entities. Previous engagement with UNESCO will be
an advantage.
 Proficiency in written and spoken English language. Knowledge of Tetum and/or
Portuguese languages are desirable.

7. DELIVERABLES AND SCHEDULE

The contracted individual/institution or organization shall deliver the following three key deliverables
in English:
1) Inception report - containing the evaluation framework, detailed evaluation methodology,
project/programme sample, work plan and logistical arrangements;
2) Draft Evaluation report of max. 50 pages (excluding annexes) to be structured as follows:
 Executive Summary (usually not more 3 pages)
 Programme description
 Evaluation purpose
 Evaluation methodology
 Findings
 Lessons learned
 Recommendations
 Annexes (including the list of stakeholders consulted during the evaluation, key
documents reviewed, ToR, survey forms and aggregate findings, budgetary analysis, etc.)
3) Final Evaluation report of max. 50 pages (excluding annexes) to consider as appropriate
comments and suggestions received by the reference group on the draft final report.

The evaluator will provide the deliverables according to the following indicative timetable from 01
October to 20 November, 2017):

Deliverables Installment Date

Draft of Evaluation inception report 01 October, 2017
Draft of Evaluation Report of the Project (draft final 30 October, 2017
report)
Final Evaluation Report 10 December, 2017

8. ANNEXES
1) Letter of Agreement- Signed by Korea International Cooperation Agency (KOICA) and
UNESCO Jakarta Office.
2) Plan of Operation – Signed by Ministry of Education in Timor-Leste and UNESCO
Jakarta Office.

END

Page 4 of 4