(2023) Rapport sur l'achèvement et les résultats de la mise en œuvre : Fournir une éducation de qualité en Haïti (PEQH)

Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00006211 IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA D1230 ON A GRANT IN THE AMOUNT OF SDR21.1 MILLION (US$30 MILLION EQUIVALENT) AND IDA D4700 ON A GRANT IN THE AMOUNT OF SDR28.1 MILLION (US$39 MILLION EQUIVALENT) AND TF B0083 ON A GRANT IN THE AMOUNT OF US$ 18 MILLION TO THE Republic of Haiti FOR THE Providing an Education of Quality in Haiti (PEQH) July 29, 2023 Education Global Practice Latin America And Caribbean Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized CURRENCY EQUIVALENTS (Exchange Rate Effective July 13, 2023) Currency Unit = Haitian Gourde (HTG) HTG 156 = US$1 US$ 1.35 = SDR 1 FISCAL YEAR October 1 - September 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Lilia Burunciuc Regional Director: Jaime Saavedra Country Manager: Laurent Msellati Practice Manager: Emanuela Di Gropello Task Team Leader: Quynh Thu Nguyen ICR Main Contributor: Axelle Latortue ABBREVIATIONS AND ACRONYMS CDB Caribbean Development Bank CETN EMIS Coordinating National Technical Team (Coordination de l’Équipe Technique Nationale du SIGE) CPF Country Partnership Framework DDE Departmental Directorate of Education (Direction Départementale de l’Éducation) DPCE Directorate of Planning and External Cooperation (Direction de la Planification et de la Coopération Externe) EFA Education for All EMIS Education Management Information System EPGC Community-Managed Public Primary School (École Publique à Gestion Communautaire) GA Grant Agreement GAC Global Affairs Canada GDP Gross Domestic Product GoH Government of Haiti IDA International Development Association IDB Inter-American Development Bank IRI Intermediate Results Indicator IRR Internal Rate of Return ISR Implementation Status and Results Report NDPV Net Discounted Present Value NPV Net Present Value MENFP Ministry of National Education and Vocational Training (Ministère de l’Éducation Nationale et de la Formation Professionnelle) ONAPE National Education Partnership Office (Office National de Partenariat en Éducation) PAD Project Appraisal Document PDEF MENFP Ten-Year Education Sector Plan (Plan décennal d’éducation et de formation) PDO Project Development Objective PDOI PDO-level Indicator PEQH Providing an Education of Quality in Haiti Project PIU Project Implementation Unit PLR Performance and Learning Review QAS Quality Assurance System RF Results Framework SABER Systems Approach for Better Education Results SY School Year ToC Theory of Change TWP Tuition Waiver Program UEP Studies and Programming Unit (Unité d’Études et de Programmation) UNOPS United Nations Office for Project Services USI Information Systems Unit (Unité de Systèmes d’Information) DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 11 II. OUTCOME .................................................................................................................... 15 A. RELEVANCE OF PDOs ............................................................................................................ 15 B. ACHIEVEMENT OF PDO (EFFICACY) ........................................................................................ 16 C. EFFICIENCY ........................................................................................................................... 21 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 22 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 23 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 24 A. KEY FACTORS DURING PREPARATION ................................................................................... 24 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 24 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25 A. QUALITY OF MONITORING AND EVALUATION ....................................................................... 25 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 26 C. BANK PERFORMANCE ........................................................................................................... 27 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 28 V. LESSONS AND RECOMMENDATIONS ............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 30 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 47 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 49 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 50 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 53 ANNEX 6. MAIN CHANGES DURING RESTRUCTURING ........................................................... 54 ANNEX 7. QUALITY ASSURANCE SYSTEM (QAS) .................................................................... 57 ANNEX 8. ORIGINAL PROJECT RESULTS CHAIN AND THEORY OF CHANGE ............................. 60 ANNEX 9. SCHOOL CLOSURES BETWEEN 2018 AND 2023 IN HAITI ........................................ 62 ANNEX 10. SUPPORTING DOCUMENTS ................................................................................ 66 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 1 of 68 DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P155191 Providing an Education of Quality in Haiti (PEQH) Country Financing Instrument Haiti Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Ministry of Economy and Finance (MEF) Ministry of National Education and Professional Training (MENFP) Project Development Objective (PDO) Original PDO The objectives of the Project are to: (i) strengthen public management of the education sector; (ii) improve learning conditions inselected public and non-public primary schools; and (iii) support enrollment of students in selected public and non-public primary schools. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 2 of 68 FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing IDA-D1230 30,000,000 30,000,000 29,505,277 TF-B0083 17,800,000 17,600,000 17,600,000 IDA-D4700 39,000,000 39,000,000 39,458,568 Total 86,800,000 86,600,000 86,563,845 Non-World Bank Financing 0 0 0 Total 0 0 0 Total Project Cost 86,800,000 86,600,000 86,563,846 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 10-Nov-2016 07-Mar-2017 30-Oct-2022 30-Jan-2023 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 16-May-2019 15.62 Additional Financing 03-Feb-2021 41.61 Change in Results Framework Change in Components and Cost Reallocation between Disbursement Categories Other Change(s) 19-Nov-2021 58.93 Reallocation between Disbursement Categories 07-Apr-2022 77.64 Reallocation between Disbursement Categories 06-Sep-2022 81.62 Reallocation between Disbursement Categories 26-Oct-2022 82.06 Change in Loan Closing Date(s) 23-Jan-2023 85.75 Cancellation of Financing Reallocation between Disbursement Categories The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 3 of 68 KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs No. Date ISR Archived DO Rating IP Rating Actual Disbursements (US$M) 01 27-Jan-2017 Moderately Satisfactory Moderately Satisfactory 0 02 10-Aug-2017 Satisfactory Satisfactory 2.00 03 27-Mar-2018 Satisfactory Moderately Satisfactory 5.78 04 31-Oct-2018 Satisfactory Moderately Satisfactory 10.42 05 18-Mar-2019 Satisfactory Moderately Satisfactory 13.35 06 24-Sep-2019 Satisfactory Moderately Satisfactory 19.37 07 08-Apr-2020 Satisfactory Moderately Satisfactory 31.48 08 22-Oct-2020 Satisfactory Moderately Satisfactory 39.40 09 08-May-2021 Satisfactory Moderately Satisfactory 52.50 10 24-Nov-2021 Satisfactory Satisfactory 58.93 11 06-May-2022 Satisfactory Satisfactory 79.58 12 26-Nov-2022 Satisfactory Satisfactory 83.72 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 100 Public Administration - Education 17 Primary Education 83 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 4 of 68 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100 Education 100 Access to Education 90 Education Financing 45 Teachers 52 Education Governance, School-Based Management 62 Education Facilities 52 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Carlos Felipe Jaramillo Country Director: Mary A. Barton-Dock Lilia Burunciuc Director: Amit Dar Jaime Saavedra Chanduvi Practice Manager: Reema Nayar Emanuela Di Gropello Task Team Leader(s): Juan Baron, Melissa Ann Adelman Quynh Thu Nguyen ICR Contributing Author: Axelle Latortue The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 5 of 68 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Country Context 1. The Providing an Education of Quality in Haiti (PEQH) Project (P155191) was prepared in 2016, during a time when Haiti had made positive strides in economic growth, all the while maintaining its status as one of the poorest countries in the world. The largest Caribbean country by area and population (estimated at 10.7 million in 2016 1 ), Haiti had attained a Gross Domestic Product (GDP) per capita of US$1,305 in 2016, but also had a Gini coefficient of 0.6, the highest in the Americas. Six years out from the catastrophic magnitude 7.0 earthquake in 2010, reconstruction efforts, which tapped into one of Haiti’s sources of resilience, had yielded tangible progress. The earthquake killed about 230,000 people (including many professionals and public servants), displaced 1.5 million people, and caused damages and losses equivalent to 120 percent of GDP. Through an increase in external funding, remittances from the Haitian diaspora, borrowing by the Government of Haiti (GoH) and private sector investments, tangible progress had been made post-earthquake. Key reconstruction achievements by the time of Project appraisal included rebuilt infrastructure, including the main road network, improved connectivity between major cities, private sector development, and the re-establishment of education services. However, Hurricane Matthew then struck the country’s southern departments in October 2016, leaving 1.4 million people in need of lifesaving assistance, and damaging more than 3,500 schools. 2. Political violence and instability—identified as both symptoms and drivers of fragility in Haiti— had been kept at bay during a period of relative calm within the country’s tumultuous political history; however, this tenuous calm had been showing signs of decline in the months leading up to Project approval. Following the first peaceful transition of presidential office to an opposition candidate in the country’s history in 2011, tensions between the Executive branch of Government and the Parliament prevented the organization of legislative elections in 2012 and 2014. In 2015, a government was put in place, holding nearly simultaneous municipal, parliamentary, and presidential elections, which exacerbated divisions and yielded results that were contested in late 2015, giving rise to a transitional government. Then, due to Hurricane Matthew, elections rescheduled for October 2016 were again postponed, concluding with the seating of a new Government in February 2017. In fact, classified by the World Bank Group (WBG) as a country in a situation of Fragility, Conflict and Violence (FCV), a key driver in Haiti perpetuating fragility has been the State’s dysfunctional social contract with its people, characterized by, inter alia, limited institutional capacity to provide basic services and security, challenges in creating an environment conducive to growth, low human capital, and a lack of transparency which undermines trust in institutions. The country is trapped in a vicious cycle of poverty, inequality and conflict. Sector Context 3. In response to insufficient provision of education services by the GoH, inadequately regulated 1 World Bank Group (WBG) DataBank: https://databank.worldbank.org/source/2?country=HTI. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 6 of 68 non-public providers filled the gap, causing a financial burden for the average household. Non-public schools constitute over 80 percent of primary schools 2 , including non-profit and for-profit institutions owned and operated by a range of actors, such as religious institutions, non-governmental organizations, and individuals. Dialogue and collaboration between the public and non-public sectors as well as GoH capacity to regulate non-public schools was minimal. Although the GoH had financed tuition waivers programs for non-public schools earlier, recent reductions in financing and perceptions of low quality of some of these programs meant that gains in access were at risk of reversal. 4. In general, capacity for sector management remained a long-standing GoH priority and challenge. Entrenched issues affecting sector management included discrepancies between regulatory texts or standards and the de facto functioning of the institutional structures; excessive centralization within the Ministry of National Education and Vocational Training (Ministère de l’Éducation Nationale et de la Formation Professionnelle, MENFP) and a lack of information flow; a lack of regular, reliable sectoral data 3 ; the lack of a culture of strategic planning at both central and decentralized levels of government, and difficult working conditions for public servants. 5. Student achievement had remained low. Once in school, only about half of children entering the first grade would reach grade 6. A 2015 pilot fourth-grade learning assessment in mathematics conducted in public and non-public schools by MENFP revealed scores only slightly better than random guessing. A key factor associated with the weak results was the low quality of the teaching and learning environment. Classrooms suffered from a dearth of teaching and learning materials. Only 56 percent of primary schools had a source of drinking water, according to the 2013-14 school census. Public sector teachers were regularly not paid on time. Eighty percent of teachers in the first two cycles of basic education were found to be underqualified in 2015-2016. 4 6. To address the sector’s governance and quality challenges, MENFP took initial steps in 2014 to increase its oversight of the education system, and the PEQH Project invested in developing the systems to do so. In August 2014 MENFP announced Twelve Policy Measures and a National Pact for Education Quality to work on sector oversight through enforcing minimum requirements for schools. In line with these new MENFP priorities, the vision for the PEQH Project was that it would build on ongoing and prior WBG- financed support - in particular, the Education for All Project – Phase II (EFA II, P124134) that closed in June 2018 and overlapped with the PEQH Project for about 15 months - to invest in developing systems to, in the long term, enhance primary education access and quality. The PEQH Project also accounted for an important shift in GoH priorities that occurred in the months prior to its preparation: a shift toward higher investment and support by the GoH for the public system. 7. For more focused support and in coordination with development partners, the PEQH Project would concentrate its key interventions in four of Haiti’s administrative departments in southern Haiti. 2 Unless otherwise cited, original sources for data presented in the Sector Context section of this document can be found in the original Project Appraisal Document (Report No. PAD1690). 3 Only two complete statistical yearbooks had been produced and made public in the five years prior, weakening MENFP’s sector strategic decision-making position and its ability to monitor and manage school quality. MENFP’s Directorate of Planning and External Cooperation (DPCE), Studies and Programming Unit (UEP), and Information Systems Unit (USI), in reliance on the regional Departmental Directorates of Education (DDEs) for data collection, operated with fragmented data collection and storage systems, with few data protocols in common and little cross-directorate sharing of information. 4 Cambridge Education 2019. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 7 of 68 EFA II had financed activities in eight departments, with most beneficiary schools benefiting from only one or two project-financed interventions. The PEQH Project, while it had a systemic vision, would provide a more comprehensive package of services to a smaller set of schools, which would allow potential synergies, enhance impacts of interventions on quality, and facilitate local capacity building and coordination with development partners. Theory of Change (Results Chain) 8. A results chain was provided in the Project Appraisal Document (PAD). In the first Project restructuring in May 2019, a Theory of Change (ToC) illustration was provided. This ToC was partially different from the results chain at appraisal, with no change to expected Project outcomes and minimal changes to linkages between activities and outcomes. For the ICR, the ToC in Figure 1 incorporates elements from both the original results chain and the May 2019 ToC. The original results chain and May 2019 ToC are included in Annex 8 for reference. Figure 1. Theory of Change for the Project Activities/Outputs Intermediate Outcomes PDO goals/parts Long-Term Outcomes 9. Project design was influenced by and aimed to respond to the country’s drivers of fragility. By helping mitigate fiscal constraints and deliver basic social services in a context of: (i) a lack of trust by the people in the GoH, and (ii) high levels of social discontent and unrest, the Project helped to strengthen the social contract between the State and Haitian people, while also alleviating schooling cost pressures on the disadvantaged. The Project’s design and its ToC were based on the WBG’s prior knowledge and experience with the fragile context and its past impacts on education sector projects. An underlying assumption for the ToC hence was a reasonable likelihood that there could be shocks over the Project life. The Project design therefore maintained tried and tested mechanisms of support to enrollment and learning conditions from • Develop Education Management Information System (EMIS) • Refine and implement a primary school Quality Assurance System (QAS), including student learning assessments Public School Improvement Model Community Education Grant Program Results-Based Financing Model for Non-Public Schools initiated Tuition Waiver Program (until phase-out) • Fully functional EMIS • Implementation of QAS enabled Strengthened public management of the education sector Increase educational outcomes School improvement plans designed and implemented (based on QAS data, for Improvement Model schools) Non-public schools satisfying basic operating conditions Vulnerable students enrolled in non- public schools Improved learning conditions Enrollment supported in public and non- public primary schools The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 8 of 68 prior projects, such as the provision of school feeding services and school materials by non-government service providers, all of which complemented tuition waiver support for impact. Thereby, the ToC aimed to mitigate potential negative impacts of external factors. 10. The ToC also reflected capacity limitations in Haiti by keeping the intervention logic chain closer to output-level rather than outcome-level, especially as it relates to the second and third goals of the PDO. The outcome of “improved learning conditions” allowed for a broad range of types of improvements, including those of more specific and short-term in nature than a goal of improved education quality would have been. This level of Project ambition was on par with the needs and capacity of the context. Project Development Objectives (PDO) 11. The Project Development Objective (PDO), as stated in the PAD and in the Financing Agreement, comprised three goals within its formulation. They were: (i) to strengthen public management of the education sector; (ii) to improve learning conditions in selected public and non-public primary schools; and (iii) to support enrollment of students in selected public and non-public primary schools. Key Expected Outcomes and Outcome Indicators 12. In line with the PDO, the Project had three key expected outcomes, operationalized through associated PDO-level indicators (PDOIs). The first two outcomes had one PDOI; the third had three. 13. The establishment and use of an EMIS was the key achievement meant to demonstrate attainment of Outcome 1 “Public management of the education sector is strengthened”. It was expected that the EMIS’s detailed technical design would be developed and endorsed by MENFP, EMIS data would be made accessible across MENFP directorates and communicated to local stakeholders, and the EMIS data would be used by MENFP to make resource allocation decisions by Project completion. It is important to clarify that while PDOI 1 was formulated to convey that the use of an EMIS would help enable (or lay the ground for) national implementation of a Quality Assurance System (QAS, described in Annex 7) and the QAS was a key Project output, it was not within the scope of the Project itself to implement a QAS on a •PDOI 1. Use of an Education Management Information System (EMIS) to enable national implementation of the Quality Assurance System Outcome 1:Public management of the education sector is strengthened •PDOI 2. Proportion of schools suported by the Project that achieve a "sufficient" level of school learning conditions Outcome 2:Learning conditions have improved in selected public and non-public primary schools •PDOI 3.a. Number of children (of which female) enrolled in primary school through the provision of community education grantsfor school access •PDOI 3.b. Number of children (of which female) enrolled in primary school through the provision of tuition waivers to non-public schools •PDOI 3.c. Number of children (of which female) enrolled in primary school through the provision of results-based financing to non- public schools Outcome 3:Enrollment of students is supported in selected public and non-public primary schools The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 9 of 68 national scale 5 . Under the Project, relevant QAS data would be incorporated in the EMIS in a format allowing for decision-making on the use of MENFP’s resources. 14. With respect to Outcome 2 “Learning conditions have improved in selected public and non- public primary schools”, Project interventions were expected to yield more schools meeting MENFP criteria on learning conditions. The Project would rely on tracking of learning conditions assessment scores per the QAS to identify schools meeting MENFP criteria. At Project preparation, the QAS was under development; therefore, baseline data and annual and end targets were not available. 6 15. Achievement of Outcome 3 “Enrollment of students is supported in selected public and non- public primary schools” was meant to support and track the number of children enrolled in primary schools using various programs financing these schools (that the Project either developed or brought to completion). Three types of programs would support enrollment in public and non-public schools, albeit on the smaller scale than in preceding WBG-financed projects: 7 (i) the Tuition Waiver Program for non-public schools, which was being phased out; (ii) the Community-Based Program for public schools, which was financed under previous WBG operations and was to continue in existing communities; and (iii) the new Results-Based Financing Program for non-public schools, which was to be developed. Components 16. Component 1: Improving Institutional Capacity and Governance (Appraisal: US$3.0 million; Actual: US$3.6 million). This component aimed to strengthen the technical capacity of selected MENFP units to improve service delivery and educational governance. Key activities to be financed included: (i) refining and piloting the QAS; (ii) developing an EMIS that would incorporate data generated by the QAS; (iii) designing and administering learning assessments of public and non-public primary schools as part of the QAS; and (iv) strengthening the relevant MENFP directorates’ institutional planning and budgeting practices. 17. Component 2: Supporting Access to Quality, Public Primary Education in Poor Communities (Appraisal: US$13.5 million; Actual: US$52.8 million). This component aimed to support access to quality primary education in selected schools in disadvantaged communities through two programs. 18. Subcomponent 2.1: Support to the cohorts of the community education grant program under the EFA II Project. This sub-component aimed to continue support provided under the EFA II project to 61 community-managed primary schools (EPGCs - Écoles Publiques à Gestion Communautaire), allowing existing student cohorts to be financed through grade 6. The EPGCs were created under EFA II to contribute to the supply of schools in relatively remote and particularly underserved rural communities, providing them with training and grants to offer educational services to school-aged children. Under the PEQH Project, existing beneficiary communities’ School Management Committees would continue being funded through: (i) designing community education plans based on community engagement and data collection; and (ii) providing community education grants to implement these plans, which could finance expenses such as 5 The Project description in the PAD and original Financing Agreement explicitly called for “refining and piloting the QAS”. The PAD also highlights that active dialogue was underway with other development partners in Haiti for coordination and scale-up. 6 At Project appraisal, baseline data was expected to be collected during the first year of Project implementation. This indicator was later dropped and replaced with a new measure of Outcome 2, as explained in Section B. 7 Under the EFA II project, at its peak (2012-2013), the tuition waiver program alone enrolled 135,000 students. This number had decreased annually thereafter, with EFA II financing 45,027 students in its last year in 2016-2017. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 10 of 68 school rehabilitation, teacher salaries, supplies and equipment, staff training in school management, and students’ tuition and transportation expenses. 19. Subcomponent 2.2: Public School Improvement Model. This subcomponent aimed to improve access to a quality education in beneficiary public primary schools through a Public School Improvement Model. The model aimed to improve the behaviors of and dynamics among school-level actors in relation to school administration and leadership, teaching practices, learning assessment, community involvement, and accountability. QAS assessments would be used to evaluate learning conditions and learning outcomes, identifying each school’s strengths and weaknesses. School directors, in a participatory process with parents and other stakeholders, would then develop annually and implement improvement plans based on priority areas identified. Activities to be financed included: (i) the provision of technical assistance and training to school directors in management and leadership, and to teachers on pedagogical skills and content knowledge; (ii) the development and implementation of information and communication activities for school directors, school officials and other stakeholders; (iii) school improvement grants to fund planned activities, such as small infrastructure works, school materials, uniforms, and furniture; and (iv) school feeding, deworming and nutrition supplements to students. 20. Component 3: Supporting Access to Quality, Non-Public Primary Education in Poor Communities (Appraisal: US$11.5 million; Actual: US$22 million). This component aimed to support access of poor children to non-public schools while improving learning conditions in those schools. It comprised two sub- components: 21. Subcomponent 3.1: Support to the final tuition waiver program cohort under the EFA II Project. This sub-component aimed to bring to completion the non-public school Tuition Waiver Program (TWP) financed by the WBG and development partners since 2007. The final cohort of students to benefit from a full course of TWP support from grade 1 would complete grade 6 by Project closing. Per-student tuition waiver grants were provided to participating schools in exchange for the schools’ commitment to allow students to attend tuition-free and respect basic MENFP school operating criteria, such as maximum class sizes and the provision of textbooks. It also aimed to finance communication activities to raise awareness about the new results-based financing program (sub-component 3.2, described below) that would replace the TWP. 22. Subcomponent 3.2: Support to the development and implementation of a replicable model for results-based financing of non-public schools. This sub-component would support the design and implementation of a results-based financing program for selected non-public primary schools, providing grants to their management committees. To improve on the TWP by more rigorously incentivizing schools to improve teaching and learning conditions, the program would provide a per-student grant conditional on schools’ attainment of measurable results related to their learning conditions and student learning outcomes. Schools found to be out of compliance would be exited from the program and ineligible to return. The program was expected to entail: (i) carrying out an assessment of learning conditions and learning outcomes in participating schools, using QAS tools 8 , with assessment results communicated to schools using a school scorecard tool managed by the Project; (ii) providing technical assistance and training to school directors in management and leadership; (iii) providing technical assistance and training to teachers on 8 The QAS learning conditions questionnaires and a standardized learning assessment would be administered at the beginning of implementation, and periodically thereafter. Learning assessments were planned to be administered once every academic year. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 11 of 68 pedagogical skills and content knowledge; and (iv) school feeding, deworming and nutritional supplements to students, if called for in QAS assessment results. 23. Component 4: Project Management, Monitoring and Evaluation (Appraisal: US$2.0 million; Actual: US$8.3 million). This component aimed to support Project management and supervision, monitoring, and evaluation, and activities to strengthen MENFP’s monitoring and evaluation functions. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Restructuring processes 24. The Project was restructured seven times during its implementation. Key changes introduced through restructuring are summarized below, and more detail can be found in Annex 6. (i) The first restructuring (R1) occurred in May 2019. The Project was advancing in a satisfactory manner despite some delays, with 53 percent of original financing disbursed. With limited revenues and great needs, per GoH request, R1 (i) scaled up interventions to increase impact, and (ii) added a gender dimension in relation to schools’ social and physical environments, to address disparities in dropout rates affecting girls. Additional financing included US$39 million through a grant from the International Development Association (IDA), and up to US$18 million equivalent through another grant (to be disbursed in installments, with US$2 million in the first installment) from Global Affairs Canada (GAC), bringing the total committed amount to US$86 million equivalent. A Contingent Emergency Response Component was also introduced (but never activated) to facilitate response to an eligible emergency. (ii) The second restructuring (R2) was approved in February 2021 and aimed to align the Project with shifted MENFP priorities in response to the changes in the country context and COVID-19 pandemic. Protests over sociopolitical and economic grievances, which started in July 2018 in several cities, culminated in total lockdown during September-December 2019, causing closure of schools, businesses, and the GoH and its services, with about 80 percent of school-aged children in Haiti not attending school, according to MENFP data. Then the arrival of COVID-19 in the country in March 2020 prompted MENFP to close all schools and teacher training centers for about five months. As such, some students were out of school for nearly a full academic year. Project interventions were thus reoriented to: (i) support MENFP’s COVID-19 response plan; and (ii) reach additional beneficiaries to help protect education access and primary school retention. R2 also included an amendment to the Trust Fund Grant Agreement governing GAC funds, to account for a second installment of $8.9 million, increasing the GAC grant amount available to the Project to US$10.9 million. (iii) The third restructuring (R3), approved in November 2021, reallocated funds across disbursement categories to ensure sufficient funds for tuition waivers to non-public schools. It also updated the Project Environmental and Social Management Framework in view of school infrastructure-related activities introduced after the August 14, 2021 earthquake of 7.2 magnitude, which struck three southern departments of Haiti, where Project-supported schools were located. (iv) The fourth and fifth restructurings (R4 and R5) were approved in April 2022 and September 2022, respectively. They each amended the Trust Fund Grant Agreement for GAC funds to add funds. R4 added an installment of US$2.2 million, and R5 US$4.7 million. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 12 of 68 (v) The sixth restructuring (R6) was approved in October 2022, just before the original Project closing date of October 30, 2022, and extended the closing date to January 30, 2023 for all Project grants to allow for completion of repair works in schools, which had experienced interruption due to a lockdown and unprecedented context of insecurity, with rapid increases in gang activity, road blockages, fuel shortages, violent protests—and widespread school and business closures. (vi) The seventh restructuring (R7) cancelled US$200,000 from Project funds to correct a WBG funds calculation error incurred at R5. Revised PDO and/or PDO Indicators (PDOIs) 25. There were no revisions to the PDO during the Project life; however, PDOIs were revised in wording, scope and end targets. PDOI 1 was revised at R1 in its end target to broaden the scope of data use in relation to EMIS implementation: rather than expecting EMIS data to be used for resource allocation decisions by MENFP as was stated in the original end target wording, the end target was changed to expect the utilization of EMIS data for decision making (in a broader sense) by MENFP, as this was considered a sufficient measure of and better proxy for strengthened public management of the education sector. PDOI 2 was replaced at R1: the original PDOI 2 (“Proportion of schools supported by the Project that achieve a “sufficient” level of school conditions”) was replaced by: “Net proportion of schools supported by the Project that improve their school learning conditions score”, which had a new calculation methodology. During initial QAS implementation, the Bank team concluded that tracking the proportion of schools having achieved a “sufficient” level of school conditions 9 might not provide enough granularity to capture changes in school quality under the Project. The revised formulation would allow for school quality scores to be compared to baseline scores while also having data on any schools regressing. 26. PDOIs for outcome 3 were revised both at R1 and R2. First, two new PDOIs were added: “PDOI 3.d. Number of children enrolled in the sub-component 2.2 ‘full package’ public primary schools, of which are female” at R1, to capture at the PDO level enrollment supported through the Public School Improvement Model 10 ; and “PDOI 3.e. Number of children enrolled in sub-component 2.3 ‘light-package’ primary schools, of which are female” at R2, to track enrollment supported through the newly created Critical Support for Public School Operation QAS-based interventions (described below), added to reach additional beneficiaries. Second, annual and end targets were revised to reflect the overall scale up of activities to support student access to school. Specifically, cumulative end targets were increased for PDOIs 3.a (“Number of children enrolled in primary school through the provision of community education grants for school access, of which female”) at R1 to reflect a higher number of children than anticipated; and 3.c (“Number of children enrolled in primary school through the provision of results-based financing to non- public schools, of which female”) to reflect more schools supported, at R1 due to scale up from additional financing and at R2 due to the reorientation of funds towards the protection of education access via this program 11 . PDOI 3.d had its end target reduced at R2 (from 192,000 to 120,000 cumulative annual enrollment) to reflect a reduced number of public schools supported via a full QAS package (with the 9 A description of the QAS and the levels of school quality can be found in Annex 7. 10 These activities were already under implementation as planned in the original Project design, but the associated enrollment numbers had not originally been incorporated at the PDO level of the results framework. 11 At R1, the end target for PDOI 3.a was increased from 15,000 to 16,000 cumulative annual enrollment to reflect a higher number of children enrolled in supported EPGCs in the initial year of Project implementation. To reflect the increase in the number of non- public schools supported through the results-based financing program, the end target for PDOI 3.c was increased during both R1 and R2, from 50,000 to 71,000 at R1, then to 91,000 cumulative annual enrollment at R2. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 13 of 68 schools moved from this activity benefiting instead from the light QAS package, accounted for in the new PDOI 3.e end target). Revised Components 27. Changes were introduced to all components during the Project’s lifetime, in particular through the first two Project restructurings. Component 1: Improving Institutional Capacity and Governance 28. Component 1 activities were scaled up under R1 with new activities thanks to additional financing. An additional US$1.5 million was allotted for more technical assistance to MENFP’s Information System Unit (USI) and other technical and decentralized MENFP departments, to support EMIS data collection, reliability, and use. Additionally, US$900,000 were added to introduce (i) gender upgrading of the QAS; (ii) gender-disaggregated trends in the EMIS; (iii) gender assessments of teacher practices 12 ; (iv) behavioral studies on gender norms; (v) training of MENFP staff at central and decentralized levels; and (vi) the development and distribution of communications materials on gender equality topics. 29. Under R2, new activities were added at a cost of US$2 million, thanks to funds reallocated from other activities that had shown slower implementation than expected and/or had become of lower priority to MENFP, to support the response to COVID-19, including (i) communications activities for COVID-19 prevention; (ii) the operationalization of a MENFP distance learning platform under development at the time, and (iii) the development of pedagogical content for the platform and other media. Component 2: Supporting Access to Quality, Public Primary Education in Poor Communities 30. Under R2, to reach more students using existing funds following the sociopolitical lockdown and COVID-19 school closures, the QAS-based interventions targeting beneficiary schools under this Component were revised. Subcomponent 2.1: Support to the cohorts of the community education grant program under the EFA II Project 31. Under R1, financing of students’ transportation services was eliminated to focus Project financing on tuition expenses, and the 61 community-based schools were to receive a new “light package” of QAS- based interventions instead of the original “full package” 13 . Subcomponent 2.2: Public School Improvement Model 12 The QAS questionnaires already took into account gender considerations so this activity did not need to be implemented; activities related to the gender assessment of teacher practices did not materialize; and a planned study on gender norms was reprioritized and rescheduled to be covered by the new PROMESSE project. 13 The light QAS packages for public schools included QAS data collection, funds to cover schools’ operating costs, delivery of school feeding services or food rations, the provision of school textbooks and kits, training on reading instruction methods, and technical assistance to school management committees. The full package included all interventions in the light package, plus other elements including QAS learning assessments, implementation of school improvement plans, rehabilitation of latrines, and gender clubs. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 14 of 68 32. Additional financing was added to this sub-component under R1, with 69 public primary schools expected to be added to the Public School Improvement Model to scale it up. These schools were expected to benefit from the full package of QAS-based interventions, which was expanded to include training in climate change and response to natural disasters and on gender equity; and gender informed latrine rehabilitation for security, privacy, cleanliness, clean water, and menstruation health—although some of these elements added to the full package did not materialize. Clubs (referred to as “gender clubs”) for girls, boys, and parents, respectively, were also introduced to promote awareness of gender-based violence and harmful stereotypes and enable safer spaces for girls, including by promoting sexual and reproductive health knowledge and parental support. 33. At R2, the Public School Improvement Model was reduced in scale, with only 70 of the original 149 public beneficiary schools receiving the full package of QAS-based interventions. To focus and facilitate implementation and supervision, these 70 schools were in two of the four targeted departments, namely Nippes and Sud. The remaining schools originally benefiting from this model were shifted to a new Sub- component 2.3. (New) Sub-component 2.3: Critical Support for Public School Operation 34. Sub-component 2.3 was added under R2, with reallocated funds from other sub-components. A total of 136 schools—with 79 public schools originally receiving the full package of QAS-based interventions and 57 new schools selected per existing criteria—were expected to benefit from a light QAS package. Component 3: Supporting Access to Quality, Non-Public Primary Education in Poor Communities 35. Changes were made only to Sub-Component 3.2. No changes were made to Sub-component 3.1: Support to the final tuition waiver program cohort under the EFA II Project. Subcomponent 3.2: Support to the development and implementation of a replicable model for results- based financing of non-public schools 36. Under R1, US$6.5 million was added to scale up the results-based financing model to additional schools, with an expectation of more than doubling the number of student beneficiaries. Due to data collection challenges, the criteria for selecting beneficiary schools were reduced to three; and due to costs and a shift in focus, school feeding related activities (food rations, deworming and nutritional supplements) were eliminated for non-public schools (although continued for public schools). The original QAS-based interventions for non-public schools were expanded to include training on climate change, response to natural disasters and gender equity. In addition, financial incentives were to be provided to schools to implement gender clubs and gender-informed latrine rehabilitation; however, these new activities did not materialize as part of this model. 14 37. Under R2, cost savings achieved from other activities were used to add an expected 18 schools to the results-based model. The package of QAS-based support for non-public schools was reduced in scope to consist of technical assistance, results-based tuition waiver grants, teacher training, and (in a sample of schools) learning assessments. In addition, based on implementation experience until that point, school 14 Activities related to gender equity were implemented in selected schools as part of the Project’s public school programs. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 15 of 68 tuition waiver grants were deconditioned from student learning outcomes given the fragile context (including school interruptions and closures) and low capacity. Instead, the grants were conditioned on input- and process-based indicators such as alignment of schools with MENFP’s calendar and programs and availability of books for students. Component 4: Project Management, Monitoring and Evaluation 38. Under R1, about US$3 million were added to sustain Project management and monitoring activities and hire additional staff to support implementation of gender activities for the scaled-up Project. Under R2, a planned impact evaluation was cancelled due to an inability to implement it in the increasingly challenging context. Other Changes 39. Additional changes introduced through restructuring included: (i) revisions to intermediate results indicators (IRIs) for clarity or to align to adjusted activities, including revisions to targets per the pace of implementation or reflect the scaled-up or scaled-down activities, and the addition of indicators (for instance, to introduce gender-focused activities or new WBG corporate indicators); (ii) the triggering of a new safeguards policy (OP 4.09 on Pest Management) under R1, as small works may involve the incidental use of pesticides or herbicides; and (iii) revisions to Project costs and reallocations to disbursement category amounts in connection with changes to components or to address cost under- or over-runs. Annex 6 provides additional information. Rationale for Changes and Their Implication on the Original Theory of Change 40. Of the Project’s seven restructurings, the first two included changes that entailed revisions at the activity level to the ToC. Namely, these changes are the addition of: (i) activities to foster gender equity— with particular attention to providing safe and equal learning conditions for young girls—and climate change considerations across all components (this did not materialize); and (ii) a lighter package of QAS- based activities for more public schools. The rationale for the changes has been described in the preceding paragraphs. All modifications introduced to the Project are considered to fit within the objectives of its original design and support the same intermediate and development outcomes. Therefore, changes to the ToC are not considered substantive. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: High 41. At Project completion, the PDO was highly relevant and consistent with the WBG’s existing partnership strategy for Haiti. At the time of ICR preparation, the latest WBG Country Partnership Framework (CPF) for Haiti covers the period FY16-FY19. This CPF was revised in May 2018 based on a Performance and Learning Review (PLR). There is direct alignment with the PLR’s Focus Area 2 (Enhancing Human Capital), Objective 3 (Improve access to quality primary education), tracked by indicators on (i) The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 16 of 68 primary student enrollment and (ii) the development of QAS standards and assessments by MENFP and their application in schools. The PLR confirmed the anticipated attainment of some results in the education sector (while noting implementation delays in other sectors, given the political transition during 2015- 2017), and called for further investment in the long-term benefits of education and proposed increased targets for human capital outcomes. 42. The PDO is also highly relevant to other PLR adjustments introduced to the WBG’s Haiti country strategy, most notably the introduction of a territorial approach to investments, to foster economic opportunities outside of Port-au-Prince; and the focus on local capacity to prepare for and respond to disasters. The Project, from its inception, adopted a territorial approach in the southern departments, for synergy among its own interventions and with those of other WBG sectors, thereby supporting integrated regional provision of service delivery. In addition, through the addition of activities to support gender equity, the Project was well-aligned with the CPF’s approach of mainstreaming gender considerations and activities in WBG programs. 43. In addition, the PDO is aligned with the WBG’s Strategy for FCV 2020-2025 and highly relevant to Haiti’s fragility context, while remaining in line with the country’s education sector priorities at completion. By helping to strengthen Haiti’s weak social contract through the achievement of its objectives, the Project supports the achievement of the third pillar (Helping Countries Transition Out of Fragility) of the WBG’s FCV Strategy. Despite no change to Project outcomes during implementation, in response to the various shocks and changes to Government priorities, the Project’s strategic emphasis shifted more toward support to school enrollment and the provision of foundational inputs for more beneficiaries. Still, the PDO is consistent with the three strategic pillars of the GoH’s Ten-Year Education Sector Plan for 2020-2030 (PDEF – Plan Décennal d’Éducation et de Formation 2020-2030), namely: (i) Strengthen Governance; (ii) Increase Access and improve Equity; and (iii) Improve Quality and Relevance. The Project targeted of some of the poorest departments and communities, thereby supporting equity; supported governance through capacity enhancements and establishment of an EMIS and a QAS; and supported quality and relevance through efforts to enhance the learning environment. B. ACHIEVEMENT OF PDO (EFFICACY) Rating: Substantial 44. The below assessment of the PDO is organized around the three objectives of the PDO statement. More information can be found in Annex 1. A split rating is not applied to the Project efficacy assessment because, despite the changes introduced to PDO outcome targets during implementation, the Project is considered to have increased its overall level of ambition through additional financing and restructuring. Specifically: (i) R1 was a scale-up with additional financing, increasing the Project scope and level of ambition, as evidenced by an increase in the number of beneficiaries supported under the third objective of the PDO statement. It doubled the number of schools supported by the Project. (ii) R2 reallocated funds to (a) finance COVID-19 related activities, and (b) reach more beneficiaries, in response to a context in which school access and retention were at risk of regression. While there was no increase in financing and the intensity of support was reduced for some existing public and community-based schools and for non-public schools as mentioned above, there were increases in the number of activities overall and in the number of beneficiaries The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 17 of 68 (PDO3 indicators). To reflect the changes introduced, the PDOI 3.d end target was reduced by 72,000 cumulative annual enrollment; however, this was compensated by a larger total increase of PDOI 3.c and 3.e targets (to a combined total of 118,000 annual enrollment), which resulted in a net increase of 46,000 cumulative annual enrollment introduced at R2. The Project then surpassed the majority of PDIO 3 targets, as described below. (iii) Subsequent restructurings did not change Project components or the results framework and did not entail a significant impact to the Project’s level of ambition. (iv) Other changes made to PDOIs did not reflect a lower level of Project ambition. The PDOI 1 end target was adjusted for better alignment with the PDO statement, but this change did not represent a reduction in scale or ambition. PDOI 2 was replaced by a new measure of the related outcome, deemed to better capture changes in school quality; this change also did not represent a reduction in scale or ambition. 45. In addition to the increase in ambition evidenced by the overall increase in Project beneficiaries, the ICR assesses that the PDO represents an increased level of ambition at closing in relative terms compared to the time of its establishment in 2016 in view of changes in the country and sector contexts. The context became increasingly challenging over the course of Project implementation, culminating in unprecedented insecurity, instability, and hardship in the last 1.5 years with the assassination of the president in July 2021, the 7.0-magnitude earthquake in August 2021 affecting southern departments where Project-supported schools were located, and the country-wide lockdown that caused schools to close during September-December 2022. Yet, the Project was brought to successful completion against the same PDO and with just three months of extension of the closing date. Assessment of Achievement of Each Objective/Outcome Objective 1: Strengthen public management of the education sector: Fully achieved. 46. This objective was measured by PDOI 1 and associated IRIs—IRI 1, IRI 4, IRI 5, and IRI 9—whose end targets were all met or surpassed. PDOI 1 (Use of an EMIS to enable national implementation of the QAS) was achieved, with the end target (“EMIS implementation: data utilized for decision making by MENFP”) met, as summarized in table “B. Key Outputs by Component” in Annex 1 and explained below. 47. An EMIS that incorporates available national data from the 2016 school census and data collected by an external verification firm on Project-supported schools was established and contains features that MENFP is currently using for decision-making and better management of the sector. In terms of its features, the EMIS has achieved a national scope, is accessible to multiple MENFP units, directorates, and all DDEs, is set up for various MENFP data collection needs (related to schools, students, and MENFP human resources, for instance) and allows for real-time data sharing. The Project financed not only EMIS hardware, software, equipment, and technical assistance, but also supported the establishment of protocols and processes related to EMIS operationalization. Importantly, coordination mechanisms were established and have become routine, linked to Project interventions, allowing for inter-directorate communication and joint decision-making related to the EMIS 15 . As such, a new dynamic was created of collaboration on sector data, 15 Following an EMIS study tour for MENFP officials to Burkina Faso, recommendations of a SABER-EMIS diagnostic (under a US$5 million Haiti Statistical Capacity Building in Education Grant (P161160), and (under the Project) the development and validation of The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 18 of 68 which was unprecedented within MENFP and has led to its strong ownership of the EMIS. These achievements are considered strengthened public management in the low-capacity context of Haiti. Additionally, in March-May 2023, shortly after Project closing, MENFP built on the EMIS and used its interface to undertake an online school registration exercise, by which all schools were requested to directly provide administrative information. This exercise was the first nationwide data collection on schools since the 2015- 2016 school census, allowing further data collection to be planned for MENFP regulation and decision- making, such as regarding school accreditation and response to school needs in case of disasters. 48. Moreover, the QAS, with a gender dimension, was established, tested, simplified, and implemented to collect data in Project-supported schools in the southern departments, as well as in other schools in the northern and central departments of the country supported by the Inter-American Development Bank (IDB) and Caribbean Development Bank (CDB), respectively 16 . Collectively, the QAS was implemented in 8 of the country’s 10 departments, with near-national geographic scope. MENFP’s USI developed an EMIS module for QAS data, which contains QAS data from the PEQH Project and the CDB project; data from the IDB project were in the process of integration at the time of ICR writing. Thus, the EMIS enabled national implementation of the QAS with partner coordination set up from Project preparation. This is another illustration of improved public sector management. 49. The development and implementation of a Results-Based Financing Model for Non-Public Schools under subcomponent 3.2 of the Project, by which MENFP could hold non-public schools supported by the Project accountable by leveraging the QAS, also contributed to strengthening public management and exemplifies the use of the EMIS for decision-making. The QAS allowed for: (i) tracking of learning conditions; (ii) increasing school accountability by enforcing minimum standards; and (iii) directing public financing toward better-performing and improving schools. QAS assessment results revealed that as a group, the non- public schools having participated in the results-based financing program increased their level of global learning conditions, from the “basic” level to the “sufficient” level per MENFP’s school quality criteria. Based on QAS data communicated to schools in the form of school scorecards, a decision was made to provide performance bonuses to 48 Project-supported non-public schools that improved their learning conditions score under this model. By Project closure, it was also decided to remove eight schools from the program due to not meeting the minimum requirements of the model. 50. The successful establishment of the QAS also supported improved learning conditions in Project- supported schools by providing information for the development of school improvement plans financed by the Project. As part of the QAS, the Project also developed and implemented student learning assessments, with potential for replicability nation-wide. In particular: (i) manuals and evaluation materials were developed and pilot-tested, and a first round of evaluations was conducted in Creole, French and Mathematics on 6,119 students in grade 3 in 166 Project-supported public and non-public schools in 3 EMIS framework documents, a MENFP ministerial decree created the EMIS Coordinating National Technical Team (CETN), which comprises directorates including the Information Systems Unit, Planning Unit, and Directorate of Support to Non-Public Schools and other units, operates in alignment with a consolidated EMIS annual action plan (coordinating different sources of financing), and has been meeting weekly. 16 IDB- and CDB-financed projects implemented the same QAS tools simplified under PEQH. The IDB, whose financing of the QAS was on a later implementation timeline than was PEQH, financed the development of questionnaires for two additional QAS dimensions (infrastructure and student socioemotional development). Its projects implemented the full QAS-based package in 245 schools, benefiting about 80,000 students. The CDB financed the full QAS-based package of interventions, benefiting 41 schools and about 14,460 primary students. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 19 of 68 departments; and (ii) learning standards in French and Mathematics were developed and validated for these schools. A total of 159 school directors and 266 teachers in public and non-public schools received training to administer the assessments. Assessments results were shared with the public via the Project website and contribute to the EMIS database for sector management. 51. Finally, the Project financed the development of a new National Accreditation Policy and a strategy document for local school management, in synergy with the QAS for harmonized, systemic learning conditions standardization and monitoring per Project design. The two policy documents were adopted by MENFP in a consultative process; and an implementation plan was being finalized at the time of ICR writing. This is another achievement toward strengthened public management of the education system. Objective 2: Improve learning conditions in selected public and non-public primary schools: Exceeded. 52. This objective was measured by PDOI 2 and associated IRIs—IRIs17 2, 3, 6, 11, 12, 13, 14 and 16—whose end targets were met or surpassed in all cases but two, as summarized in table “B. Key Outputs by Component” in Annex 1 and explained below. 53. PDOI 2 (Net proportion of schools supported by the Project that improve their school learning conditions score) was surpassed by 21 percentage points in public schools and 4 percentage point in non- public schools. This indicator tracked progress on learning conditions/school quality in 376 schools benefiting from the Public School Improvement Model, EPGCs (during part of Project implementation), the Critical Support for Public School Operation sub-component, and the Results-Based Financing Model for Non-Public Schools (subcomponents 2.2, 2.3 and 3.2, respectively). At Project completion, a net 51 percent of the supported public schools and 55 percent of non-public schools had improved their QAS learning conditions score (net of those with score regressions), significantly surpassing the end target of 30 percent. Schools made improvements in one or more composite dimensions including leadership of the director, pedagogy, the user friendliness of the school environment, and personnel. In particular, QAS data indicate that most participating schools (63 percent) improved in three to five dimensions during the last two school years of the Project. Schools that improved their learning conditions scores showed the most improvement on teamwork, pedagogical supervision, and performance evaluation. Annex 7 provides more information on the QAS dimensions and tools. 54. Under the Public School Improvement Model, 18 school improvement plans were developed and tailored by each school to its specific needs, informed by areas for improvement indicated by QAS score, with activities ranging widely from small school repairs to pedagogy or community participation. From School Year 2018-2019 (SY18-19) through SY21-22, 100 percent of Public School Improvement Model schools implemented improvement plans financed by the Project. As part of the school improvements plans, 195 classrooms received small rehabilitation works using grants provided by the Project, to improve the schools’ physical conditions. Ten water and sanitation facilities in Public School Improvement Model schools 17 There was also the IRI “Number of water and sanitation facilities upgraded in Component 2.2 schools based on gender-informed protocol” was added to the RF during R1 in 2019, reported on in two ISRs in 2019 and 2020 and was unchanged in R2’s RF of February 2021, but subsequently dropped from the RF and not reported on from the ISR of May 2021 onwards. The target for this IRI was 80 facilities, and the last reported value in February 2021 was 10 facilities. 18 The Project also supported the development and implementation of improvement plans in community-managed schools until R2 (February 2021) when the community-managed schools were shifted to the light package of QAS-based interventions for public schools which did not include school improvement plans. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 20 of 68 were upgraded based on guidelines from a gender-informed protocol (developed under the Nordic Trust Fund). Moreover, 66 percent of teachers under the Public School Improvement Model and Results-Based Financing program participated in training provided by the Project in back-to-school planning, the use of textbooks, and scripted reading instruction in Haitian Creole and French in the early grades. The Project also provided school textbooks, workbooks and kits and teacher textbooks, and school feeding to public and community-based schools, which contributed in an important manner to the teaching and learning environment and experience. Furthermore, by Project completion, 171 classrooms in 19 public schools that were damaged by the August 2021 earthquake had been rehabilitated. Finally, gender-responsive activities and considerations were introduced; details are provided in Section II.E. Over the course of the Project, cumulative annual enrollment of 405,845 students benefited from direct interventions to enhance learning conditions, with 46 percent female. Objective 3: Support enrollment of students in selected public and non-public primary schools: Fully achieved. 55. This objective was tracked using five sub-PDOIs (Table 3) that reported on cumulative annual enrollment in primary school through the different modalities supported by the Project. Each sub-PDOI is equally weighted to arrive at an average achievement of Objective 3. Two IRIs—IRI 7 (number of direct Project beneficiaries) and IRI 8 (number of additional classrooms built or rehabilitated at the primary level from Project interventions), whose end targets were both surpassed—were also linked to Objective 3. 56. Through five programs, the Project succeeded in supporting enrollment, with final results surpassing most targets. Each program and its achievement are briefly discussed below. (i) PDOI 3.a: Community education grants for school access. Community-managed schools, created under the EFA II project to provide access to primary students in remote, underserved areas, were provided technical support to develop annual budgets, which, once approved, served as the basis for the disbursement of annual grants allowing them to function and enroll students tuition-free. During the Project, students attending these schools benefited from school feeding, school textbooks and kits, and their teachers participated in training in reading instruction and other themes, such as gender. QAS data was collected on school learning conditions which informed the development of community education plans. The end target (16,000) was significantly surpassed (243 percent achievement rate), with cumulative annual enrollment of 38,855 over the Project’s lifetime. (ii) PDOI 3.b: Tuition waivers to non-public schools. Under the TWP, non-public schools received a per-student tuition waiver grant to enroll students tuition-free. The end target (16,500) was surpassed (134 percent achievement rate), with cumulative annual enrollment of 22,141 during SY17-18, the final year of the program and the only year under the Project. While the PEQH Project’s enrollment programs were not evaluated, a 2015 TWP impact evaluation under the EFA II project confirmed that, compared to a control group, TWP beneficiary schools had more students enrolled and fewer repeaters and overaged students. Given that this last TWP cohort participated in the program within three years of this impact evaluation with no major changes to the program since then, these findings could reasonably apply. (iii) PDOI 3.c: Results-based financing to non-public schools. Under this sub-component, 145 non-public schools which met compliance criteria based on QAS assessments received per-student grants to enroll students tuition-free. Well-performing schools also received performance The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 21 of 68 bonuses. The end target (91,000) for this intervention was surpassed (131 percent achievement rate), with cumulative annual enrollment of 119,718 in non-public schools via the results-based financing model. (iv) PDOI 3.d: “Full package” support to public primary schools. The original target of 192,000 cumulative annual enrollment was introduced in R1 then revised downward in R2 to 120,000. This did not reflect a shrinkage but rather a reorientation of Project resources to increase overall student coverage. At R2, as the social turmoil and risk of regression in children’s access to schooling were severe, it was decided to reduce the number of “full package” schools (this PDOI 3.d) and introduce “light package” schools (PDOI 3.e) to reach more children. (See footnote 12 regarding the packages). Thus, PDOIs 3.d and 3.e should be viewed together. The revised PDOI 3.d target was largely attained (98 percent), with cumulative annual enrollment of 117,935. (v) PDOI 3.e: “Light package” support to public primary schools. The end target (98,000) was surpassed (109 percent achievement rate), with a cumulative annual enrollment of 107,196 in public schools benefiting from the light package of QAS interventions. Taken together, the two public school package enrollment targets summed originally to 192,000 (only “full package”) and after revision to 218,000 (“full” and “light” packages), with joint achievement of 225,131 in cumulative annual enrollment, or about 103% of the target. 57. In addition, the end results on IRIs 7 and 8 under Objective 3 also exceeded the targets in terms of direct project beneficiaries as measured by cumulative annual enrollment and number of classrooms built or rehabilitated (which clearly supported enrollment). 58. All but one sub-indicator end targets related to Objective 3 were surpassed, with the end target of PDOI 3.d being largely achieved. Hence, this objective is considered, at least, fully achieved. Justification of Overall Efficacy Rating Overall Efficacy Rating: Substantial. 59. The Project is rated Substantial for Efficacy, having achieved PDO-level targets for two objectives of the PDO statement and surpassed the target for the remaining objective. C. EFFICIENCY Assessment of Efficiency and Rating Rating: Substantial Implementation Efficiency 60. As the Project was fully implemented and achieved its objectives despite encountering a host of exogenous challenges (see Section III.B), efficiency of implementation was strong. The resilience of the Project teams, supported by an experienced Project Implementation Unit (PIU), a relatively supple Project design that built on continuity from and foundations put in place by the previous EFA II project, and seven restructurings, allowed the Project to achieve its main objectives with minimal delays. The Project was largely implemented within the expected timeline, with only a 3-month extension required during the final month The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 22 of 68 of implementation to allow completion of repair works in schools, delayed due to the rapidly deteriorated country safety and security context. Economic Efficiency: Net Present Value and Internal Rates of Return (more details in Annex 4) 61. The Project is expected to have improved learning conditions and supported student enrollment in selected public, community-managed and non-public primary schools. A Net Discounted Present Value (NDPV) analysis was performed for Component 2 (Supporting access to quality, public primary education in poor communities), and Component 3 (Supporting access to quality, non-public primary education in poor communities). These activities represent 82 percent of the Project’s total grant amount. The analysis does not capture the broader impacts related to public management of the education sector (Component 1) and other potential externalities. At Project appraisal and R1 in 2019, net economic benefits were also estimated for Components 2 and 3 only. 62. The cost-benefit analysis uses the standard methodology for computing the aggregated private returns to work in adulthood of the students benefitting from Project interventions. The effects of school closures due to COVID-19, the 2021 earthquake and insecurity issues are also considered. Project benefits rely on three channels: (i) Increase of wage incomes associated with the expected gains in student learning because of improved conditions for teaching and learning in schools. Those impacts were not considered at appraisal of the Project, nor at the time of additional financing (see Annex 4). (ii) Increase of beneficiaries’ educational attainment resulting from the decrease of dropout rates due to the different interventions. (iii) Increase of beneficiaries’ educational attainment resulting from tuition waivers. These benefits were not considered at Project appraisal. 63. The NPV of the Project is about US$25 million, corresponding to an estimated Internal Rate of Return (IRR) of 9 percent. Those results are higher than the NPV and IRR estimated at Project appraisal and at the additional financing of R1, confirming that the Project is a good investment from an economic point of view. The difference is mainly driven by the increase of wage incomes associated with the expected gains in student learning because of improved conditions for teaching and learning in schools. A sensitivity analysis was conducted assuming a more conservative discount rate of 10 percent, a lower rate of employment and a lower increase of income per additional year of schooling, given that benefits are mainly measured using wage earnings, which are dependent on the economic context. The results show that the IRR remains positive, although it decreases to 8 percent with a lower rate of employment of 45 percent. Table 4. NPV and IRR at Project Appraisal, Additional Financing (R1) and ICR At Project Appraisal Additional Financing (R1) ICR NPV IRR NPV IRR NPV IRR Discount rate: 8% $13.3 million 6% $10.6 million 9% $25 million 9% D. JUSTIFICATION OF OVERALL OUTCOME RATING Overall Outcome Rating: Satisfactory. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 23 of 68 64. The Project was highly relevant at appraisal and remains so at completion, substantially achieved its PDO, surpassing end targets for one of its three main objectives and achieving the remaining two. It is assessed to have produced substantial economic benefits to beneficiary Haitian youth and achieved or exceeded, all its objectives with minimum delay despite an increasingly challenging implementation context. Considering the High ratings on Relevance and Substantial rating on Efficacy and Efficiency, the overall outcome rating is assessed as Satisfactory. E. OTHER OUTCOMES AND IMPACTS Gender 65. The Project integrated a gender dimension across many activities to contribute to addressing gender gaps in relation to schools’ social and physical environments. At the system level, the QAS and EMIS both incorporated measures to collect gender-disaggregated data and include gender considerations, allowing for the tracking of gender-related progress. The Project contributed to inclusive education for Haitian girls by building or repairing single-sex toilets and water access points to encourage girls to attend school (including during their menstruation); and training MENFP and school staff on gender considerations, which helped develop competencies related to gender-differentiated behavior and equity. Given only 28 percent of public schools having functioning toilets in Haiti, the Project also provided hygiene products for latrines annually starting in SY20 in the supported public schools. Girls were found to attend school at the rate of 84 percent in Project-supported schools during SY21-22 (from the baseline of 70 percent although falling short of the 95 percent target). Additionally, by implementing gender clubs in 92 public schools for girls, boys, and parents, the Project raised awareness on gender equality issues and supported the development of interpersonal skills with a focus on gender equality, and creation of safe spaces. Anecdotal feedback from participants indicate they found that the clubs were enriching and contributed to their sense of ownership over the school environment. Institutional Strengthening 66. The Project contributed to MENFP institutional strengthening at the central and school levels. As previously explained, at the central level, the Project, notably, supported the development of the EMIS architecture and the related coordination and collaboration processes and tools (such as annual plans, etc.). By Project closing, the EMIS architecture was in place and its continued development was being supported by a multi-directorate technical committee that met regularly. In public schools, the Project supported the functioning of school management committees, provided training to school directors and teachers in multiple areas linked to the school environment, including leadership, fiduciary management, and gender, supported schools’ capacity to prepare and implement school improvement plans, and reinforced the use of data in planning and resource allocation. Non-public schools’ management was strengthened thanks to the QAS via the results-based financing model, tracking schools’ learning conditions and prompting school directors and teachers to consider and improve on key dimensions that matter for school quality. Mobilizing Private Sector Financing 67. No private sector financing was mobilized under the Project. Poverty Reduction and Shared Prosperity The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 24 of 68 68. The Project was instrumental in contributing to the Bank’s Twin Goals. It provided targeted support to the four departments in the southern region of Haiti (Sud, Grand’Anse, Nippes and Sud-Est), which includes particularly socio‐economically disadvantaged populations vulnerable to external risks, as evidenced by, inter alia, damages caused by Hurricane Matthew in 2016 and the August 2021 earthquake. Sixty-one of the supported schools were community-managed schools located in some of the most isolated areas. The Project also provided enhanced financial support to non-public schools in the more disadvantaged departments of Grand’Anse and Nippes. School feeding also provided a critical safety net for public school students, teachers, and families. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 69. Haiti’s political and governance context was relatively calm during Project preparation, and MENFP articulated clear priorities which guided Project development. While the country was undergoing a complex election process, the context during preparation had moved beyond the state of emergency brought on by the 2010 earthquake, with increased GoH spending beginning to yield favorable results. At the time of Project appraisal, there was strong leadership and commitment at MENFP, with a vision that included focusing on improving sector oversight and the quality of public education. The Project teams designed the PEQH in close consultation with MENFP instances, in alignment with MENFP’s Twelve Policy Measures for improved sector management, with features that would allow it to begin tackling the challenge of improving school quality (should prevailing conditions allow), while maintaining a pragmatic approach in mind (adaptable and scalable collection of data and generation of information; accountability; combined with service provision at the school level). 70. Lessons from previous WB-financed projects and significant preparatory elements were incorporated into the PEQH Project design. These elements and lessons included (i) technical and advisory works implemented with careful sequencing as well as partner coordination arrangements put in place during the two years leading up to Project appraisal, which was critical for the Project’s technical soundness and implementation readiness; (ii) a balance between tried and tested and new interventions, (ii) a feedback loop established at the design stage between the data produced by the Project from implementation monitoring and the planning and adjustment of future activities, and (iii) continued use of an experienced PIU. Project teams incorporated lessons from the EFA I and II projects, including a flexible design which featured continuing existing, well-established school-level interventions balanced with innovations and system-level support. The QAS was built to be modular and adaptable, to be simplified and/or implemented in stages as the context and capacity allowed. The use of regular phone surveys for monitoring was maintained to resolve bottlenecks and strengthen accountability. Finally, the current PIU includes core staff that have worked on Bank-financed education projects for over 10 years and other staff with extensive experience with projects financed by international partners, which greatly facilitated implementation. B. KEY FACTORS DURING IMPLEMENTATION 71. The Project was subject to and overcame an extraordinary number of exogenous shocks of severe intensity during implementation. Indeed, the Project’s implementation performance could be considered exceptional in the context of the many severe, disruptive events, as it was able to maintain a sufficient pace of implementation to achieve its development objectives, largely in the time allotted. There The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 25 of 68 was an armed attack on the PIU Coordinator’s home in October 2017, causing him to leave the country and requiring the Project to be coordinated by an interim coordinator for several months; followed by political turnover within MENFP which led to the reassessment of the Project’s implementation arrangements. The COVID-19 pandemic led to extended closures of schools and government offices. The President of Haiti was assassinated in July 2021, leading to an extended period of violence and social breakdown resulting in the closure of schools and businesses, as did other periods of sociopolitical unrest. The August 2021 earthquake damaged some Project-financed schools. During the year before Project closure, the country was in an unprecedented situation of insecurity, instability, and hardship. 72. Close collaboration between the Bank team, the continually strengthened PIU and the dedicated MENFP staff mitigated the negative impacts from shocks, resulting in relatively minor implementation delays. The PIU continued to be strengthened during the Project, with the addition of an internal audit team and a new school infrastructure team (which was critical to managing the delivery of school repairs in time for Project closing in the challenging context); and was complemented by the hiring of technical assistants based in DDEs and non-governmental service providers to provide services, technical support to schools and support data collection. The core Bank team provided close technical and supervision support, with close collaboration with extended team colleagues. As for the concerned MENFP units (such as USI and the Directorate of Support to Private Education and Partnerships), the US$5 million Haiti Statistical Capacity Building in Education Grant (P161160) that was initiated in January 2017 and approved in June 2017 was critical to providing them with resources and technical capabilities and to building a strong relationship among implementation team members. Regular planning and meetings occurred among MENFP, the PIU and WB, and led to timely identification and resolution of problems. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION M&E Design 73. Relevant and measurable results indicators were established to track progress toward each part of the PDO, with M&E arrangements appropriate for the context, with some weaknesses at the intermediate level of the Results Framework (RF). Despite some differences between the representations of the ToC at appraisal and at R1, the linkages between the Project’s interventions and its PDO were clear, and the ToC reflected Haiti’s capacity limitations. M&E arrangements established clear responsibilities for data collection and foresaw regular stakeholder surveys. A mechanism (a phone hotline, continued from EFA II) for grievance redressal was tracked in the RF through an indicator (“IRI 10. Stakeholder hotline to MENFP – Proportion of calls responded to/resolved within the service standards for response time”). However, there was no tracking of the integration of gender-disaggregated data in the EMIS 19 , and targets were not set at restructuring for female beneficiaries with respect to enrollment (PDOI 3) indicators. During R1 and R2, for most new or revised IRIs, annual targets were set for only some (and not all) Project years without explanation, creating gaps in guidance for implementation progress. M&E Implementation 19 This omission did not preclude achievement of the related indicator target, but it would arguably have been beneficial to revise PDOI 2 annual targets related to inclusion of gender-disaggregated data —which was a gender gap the Project aimed to address. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 26 of 68 74. As planned at Project design, the PIU led M&E implementation, relying on external verification by the United Nations Office for Project Services (UNOPS), supplemented by data provided by the Project’s technical assistants as needed to overcome ground-level challenges, to track most indicators. Detailed methodologies for data collection and calculation of results indicators were developed and regularly monitored and/or updated and reported on. Data was collected to establish baselines and end targets for indicators which were missing them at appraisal or restructuring. The information collected by UNOPS for school scorecards and school profiles were fed into the EMIS regularly. The phone hotline was used to receive and respond to beneficiary feedback, and the indicator that tracked grievance redressal was fully achieved. 75. There were some shortcomings, however. Upon Project effectiveness, the PIU, based on UNOPS data, started using cumulative annual enrollment for the PDOI 3 sub-indicators, with each year’s total added to the previous year’s. This method did not provide the number of uniquely identified students or correct for repeated grades and other variations. Nonetheless, it did provide a fair measure to track progress of enrollment support and increase. The indicator “Number of water and sanitation facilities upgraded in Component 2.2 schools based on gender-informed protocol” was added to the RF during R1 in 2019, reported on in two ISRs in 2019 and 2020 and was unchanged in R2’s RF of February 2021, but subsequently dropped from the RF without explanation and not reported on from the ISR of May 2021 onwards. Finally, a planned evaluative study was dropped. M&E Utilization 76. The M&E system and data were used by the PIU and the Bank to adjust implementation and targets, receive, and respond to feedback beneficiaries, including in circumstances of crisis and change. Surveys were carried out after the height of the COVID-19 pandemic, the August 2021 earthquake and the September-December 2022 crisis to assess the situation on the ground (such as on school operations and student attendance) and inform Project activities. The call-in hotline was open to the public during the school year and registered and responded to feedback from the beneficiaries regarding Project activities. Justification of Overall Rating of Quality of M&E Quality of M&E Rating: Modest 77. The M&E mechanisms were adequately designed, and the data collected sufficient and reliable, for monitoring Project activities and the progress towards the PDO in an FCV context. RF and supplementary data allowed timely adjustments during implementation. Shortcomings existed with respect to: (i) RF results tracking for some outputs; (ii) the calculation methodology for PDOI 3 sub-indicators, and (iii) the dropping of a planned evaluative study. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 78. Environmental and social risk was rated Moderate at approval and stayed at this level throughout Project implementation. The Project did not entail physical displacement/relocation or economic displacement. The phone hotline for grievance redressal with an agreed response protocol developed in SY18 began implementation in SY19 and was reported on through IRI 10 for which the end target of 95 percent of calls responded to/resolved within stipulated service standards for response times was surpassed. The PIU conducted weekly reviews of call logs on issues reported and took action to address them. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 27 of 68 79. Fiduciary Risk was rated High at approval and was twice reduced during the lifetime of the Project because, over time, it was assessed that financial management arrangements were adequate for sound implementation, fiduciary reporting was assessed to be of good quality and was regularly submitted to the WBG in a timely manner, and procurement implementation was timely. In fact, Fiduciary Risk was reduced to Substantial in Implementation Status and Results report (ISR) #9 approved on May 8, 2021 (although no rationale was given) and reduced further to Moderate in ISR #11 approved on 6 May 2022 (as both Financial Management and Procurement residual risks were no longer assessed as Substantial given significant improvements in implementation in both areas). Financial Management and Procurement ratings were Moderately Satisfactory for most of Project implementation and were upgraded to Satisfactory in ISR #11 approved on 6 May 2022 to reflect, respectively: (i) the resolution of internal control weaknesses and strengthening of financial management capacity with additional staff; and (ii) the improved pace of procurement execution and elimination of procurement delays. C. BANK PERFORMANCE Quality at Entry 80. The Bank team worked closely with MENFP during preparation and designed a Project that was relevant to national priorities while being aligned with the WBG’s priorities in Haiti, and also built upon previous WBG operations in Haiti’s education sector and the lessons they provided. The Project’s design was adaptable, responding to the FCV context and helping address the country’s drivers of fragility; it found a balance between tested, successfully implemented interventions and innovations that could be simplified should the context call for it. Through Non-Lending Technical Assistance (NLTA) 20 to MENFP during initial Project preparation, the Bank team brought technical expertise, fostered dialogue and delivered analyses and technical products that served as the foundation for the development of the QAS and learning standards at the primary level in math and reading; and leveraged trust fund resources to implement complementary activities to the Project (such as the Haiti Statistical Capacity Building in Education Grant approved in June 2017.) Project objectives were realistic for the context, oriented more toward output-level rather than outcome-level, but ambitious in addressing school quality within the scope of the Project timeline. Implementation arrangements facilitated readiness for implementation at the time of effectiveness. Risks to the attainment of the PDO were identified and appropriate mitigation measures (flexible Project design, MENFP capacity strengthening, reinforcement of PIU fiduciary arrangements, etc.) put in place. M&E arrangements were given due consideration. Quality of Supervision 81. The Bank worked closely with the PIU and MENFP, providing technical assistance and operational support on a regular basis, including both formal meetings and communications and informal exchanges. For much of the Project life – except during the two COVID-19 years of 2020-2022 – there was constant presence of a Bank task team leader on the ground. There was close collaboration with the extended team to both facilitate implementation and enhance PIU functions. Transitions between task teams were aided by periods of team overlaps for transfer of materials and knowledge. The team brought in simplifications for 20 “HT Education NLTA: Addressing Key Constraints to Sustainable, High-Quality Education in Haiti” (P153557) was approved in January 2015 and completed in October 2015. It delivered, inter alia, a proposal for the dimensions that would eventually become a cornerstone of the QAS; and a series of workshops/discussions leading to a shared understanding with MENFP and other development partners (notably the IDB) regarding learning standards for curriculum development and learning assessments. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 28 of 68 several activities (such as conditions for non-public school grants and package of support for public schools), adjustments to the RF and reallocation of resources through Project restructuring in response to circumstances. The team also ensured timely restructuring processes in response to shifting needs as the sector context evolved. There were some shortcomings in the RF, which however did not preclude tracking progress in the relevant indicators. Justification of Overall Rating of Bank Performance 82. Overall Bank Performance Rating: Satisfactory. The Bank built upon previous WBG operations in Haiti’s education sector and the lessons they provided to prepare a well-balanced Project and provided intensive, adaptive support to implementation in the face of repeated shocks, resulting in effective and timely Project completion. D. RISK TO DEVELOPMENT OUTCOME Risk to Development Outcome Rating: Moderate. 83. MENFP’s priorities have remained consistent, with continued development partner financing on key priorities in the Education Sector Plan of 2020-2030. The Project’s achievements are expected to be sustained through the new PROMESSE project, effective since May 19, 2022, co-financed by the WBG and the Global Partnership for Education, and implemented by MENFP and the same experienced PIU. The PROMESSE project provides medium-term continuity of financing for activities including implementation of the EMIS and QAS and modalities to support school enrollment and school quality. The fragility of the socioeconomic and political environment and the possibility of future exogenous shocks (such as climate or seismic events) constitute, nonetheless, the most significant risks to the sustainability of Project achievements. 84. In addition to the financing sustainability provided by PROMESSE, the Project has laid important foundations in terms of institutional processes and structures that are expected to help to sustain its outcomes. MENFP’s multi-directorate CETN which was established by Ministerial decree, supported by the Project, supports continued MENFP coordination and operationalization of the EMIS and education sector data collection. Efforts were underway at the time of Project completion (through policy dialogue and the new PROMESSE project) to fully integrate QAS tools into MENFP’s efforts to operationalize school standards nation-wide. Finally, the community-managed schools (whose teachers were at one point predominantly financed by the WBG), have been converted to national public schools with access to GoH resources. V. LESSONS AND RECOMMENDATIONS 85. In a fragile context, it is particularly important to put in place foundational elements to prepare for implementation well in advance and incorporate in the Project design lessons learned from prior investments as well as margins of flexibility to facilitate responses to shocks. The Project preparation team progressively put in place foundational NLTA works from 2015 to support the development of the EMIS initiated in early 2017, and the hiring of an international consultant for the development of the QAS also in early 2017. The team also adopted a balance between tested approaches which have demonstrated success (such as school feeding services), and new approaches (such as new financing models to support enrollment and implementation of quality assurance tools), which can help to provide continuity and facilitate implementation readiness while innovations are introduced. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 29 of 68 86. Continued reinforcements of PIU’s fiduciary and technical teams at the central level as well as supervisory technical assistants at the local level allowed the project to maintain critical functions in varying circumstances and take on new activities with sufficient capacity. Examples were the ready availability of a highly experienced operational lead to serve as acting PIU coordinator after the previous PIU coordinator left due to an attack, the timely hiring of supplementary Project administration support for this acting coordinator, the establishment of the internal control unit to enhance quality assurance in use of Project funds especially given the new conditional grants to schools, and the establishment of the infrastructure unit after the August 2021 earthquake. 87. Proactive and pragmatic contract management was critical. For example, two agencies were competitively recruited in a timely manner to provide school feeding services to the supported public schools at the start of Project implementation; and the PIU followed up with one of the firms due to quality issues and decided to proceed with the well-performing, highly experienced agency after the other firm failed to improve and as implementation risks became elevated. This process and decision helped the Project continue important service delivery with quality in the face of heightened risks. Another example was the weekly follow-ups with UNOPS during April-December 2022 on the school infrastructure contract after the 2021 earthquake, given serious delays incurred earlier and the deteriorating context. A dedicated task force was organized, mobilizing management, fiduciary and engineering experience as well as Hands-on Expanded Implementation Support (HEIS) to get two reimbursements from UNOPS totaling US$3.3 million within eight months, which were used to pay for school feeding and school grants before Project closing on January 30, 2023, so that no resources were wasted. 88. An agile M&E system incorporating appropriate expertise, multiple approaches for data collection and monitoring throughout the year is highly valuable. Calculation methodologies and data collection methods should be set up clearly and be appropriate for the implementation context. In addition, it is critical to keep beneficiary feedback and grievance redress communication channels open at all times (rather than only during the school year) as many activities continue even when schools are closed; the need to be informed and respond to situations at any time is particularly important in an FCV context of great constraints and frequent shocks faced by beneficiaries. At Project closing, a new SMS platform for communication with beneficiaries had been added, a mobile phone had replaced a fixed phone for the hotline, and beneficiaries could contact at any time on any day. 89. Pragmatic coordination among development partners is key for synergy and system-wide impact. Coordination with development partners should seek to align interventions around government priorities and through relevant government units for ownership and capacity building, and ideally aims to streamline development partners’ approaches and processes, thereby alleviating coordination demands on the government. The case of the development of the EMIS where the WBG, IDB and other development partners’ resources were considered together in MENFP planning was a relevant example; the focus should not be limited to data and activities specific to WBG- and IDB-financed projects but be on the system as a whole. . The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 30 of 68 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: (i) strengthen public management of the education sector. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PDO1. Use of an Education Management Information System (EMIS) to enable national implementation of the QAS Text Not started EMIS implementation: data utilized in resource allocation decisions by MENFP EMIS implementation: data utilized decision making by MENFP EMIS implementation: data utilized for decision making by MENFP 01-Apr-2016 10-Nov-2016 16-May-2019 30-Jan-2023 Comments (achievements against targets): The end target has been achieved. Objective/Outcome: (ii) improve learning conditions in selected public and non-public primary schools. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PDO2. Net proportion of Percentage 0.00 0.00 30.00 52.55 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 31 of 68 schools supported by the Project that improve their school learning conditions score 30-Sep-2016 10-Nov-2016 16-May-2019 30-Jan-2023 Comments (achievements against targets): Original target was "The target will be set when the baseline is collected". The Unit of Measure is now percentage; this original text target is therefore replaced with the number 0. The end target has been achieved. Objective/Outcome: (iii) support enrollment of students in selected public and non-public primary schools. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PDO3.a.Number of children enrolled in primary school through the provision of community education grants for school access, of which female Number 0.00 15,000.00 16,000.00 38,855.00 01-Apr-2016 10-Nov-2016 16-May-2019 30-Jan-2023 Comments (achievements against targets): The target and achievement numbers are in terms of cumulative annual enrollment. The end target has been surpassed. The actual female number achieved at completion was 16,472 or 42.4%. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 32 of 68 Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PDO3.b. Number of children enrolled in primary school through the provision of tuition waivers to non-public schools, of which female Number 0.00 16,500.00 22,141.00 01-Apr-2016 10-Nov-2016 30-Jan-2023 Comments (achievements against targets): This indicator was closed in 2018; the end target had been surpassed. The actual female number achieved at completion was 11,082 or 50.1%. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PDO3.c. Number of children enrolled in primary school through the provision of results-based financing to non-public schools, of which female Number 0.00 50,000.00 91,000.00 119,718.00 01-Apr-2016 10-Nov-2016 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): The target and achievement numbers are in terms of cumulative annual enrollment. The end target has been surpassed. The actual female number achieved at completion was 56,620 or 47.3%. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 33 of 68 Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PDO3.d. Number of children enrolled in the sub- component 2.2 “full-package” public primary schools, of which are female (Number,cumulative) Number 23,000.00 192,000.00 120,000.00 117,935.00 30-Jun-2018 16-May-2019 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): The target and achievement numbers are in terms of cumulative annual enrollment. The end target has been largely achieved. The actual female number achieved at completion was 53,298 or 45.2%. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PDO3.e. Number of children enrolled in sub-component 2.3 “light-package” primary schools, of which are female (number, cumulative) (Number) Number 29,000.00 98,000.00 107,196.00 09-Nov-2020 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 34 of 68 The target and achievement numbers are in terms of cumulative annual enrollment. The end target has been surpassed. The actual female number achieved at completion was 48,516 or 45.3%. A.2 Intermediate Results Indicators Component: Improving Institutional Capacity and Governance Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI1. Development of QAS and learning assessments by MENFP and application in schools supported by the Project Text Student learning standards and assessments in Creole reading developed; Learning conditions assessments applied in public schools supported by the Project. Student learning assessments and learning conditions assessments applied in schools supported by the Project Learning assessments (public schools in sub- component 2.2 and sample of non-public schools of sub- component 3.2) conducted Learning assessments (public schools in sub Component 2.2 and sample of non-public schools of sub- component 3.2) conducted 01-Apr-2016 10-Nov-2016 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): The end target has been achieved. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 35 of 68 Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 5. Development and provision of school director training program to sub- components 2.2 and 2.3 public primary project schools Text Training policy for teachers and education system leaders prepared by MENFP Program developed and provided to at least 55 percent of directors of schools supported by the Project Training delivered to at least 65% of the directors Training delivered to 95% of the school directors. 01-Apr-2016 10-Nov-2016 16-May-2019 30-Jan-2023 Comments (achievements against targets): The end target has been surpassed. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 9. System for learning assessment at the primary level Yes/No Yes Yes Yes 01-Apr-2016 16-May-2016 30-Jan-2023 Utility of the learning assessment system Number 1.00 3.00 3.00 Comments (achievements against targets): Original target for the sub-component Utility of the learning assessment system was: "Stage 2-3". The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 36 of 68 The end target has been achieved. Assessment results were presented to MENFP in May 2022 and key results have been posted on the Proje ct website, which increases the value of the sub-indicator on utility of the learning assessment system from 2 to 3. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Students benefiting from direct interventions to enhance learning Number 40,500.00 350,000.00 405,845.00 30-Jun-2018 03-Feb-2021 30-Jan-2023 Students benefiting from direct interventions to enhance learning - Female Number 20500.00 175,000.00 185,988.00 Comments (achievements against targets): Student beneficiaries sum up numbers from the 5 sub-indicators PDOI3a-e. The target and achievement numbers are in terms of cumulative annual enrollment. The end target for this indicator has been achieved. The end target for the sub-indicator for female is not achieved. Component: Supporting Access to Quality, Public Primary Education in Poor Communities Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 2. Number of communities proposing and implementing community education plans Number 0.00 20.00 57.00 60.00 01-Apr-2016 10-Nov-2016 16-May-2019 30-Jan-2023 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 37 of 68 Comments (achievements against targets): The end target has been surpassed. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 3. Proportion of public schools supported by the Project that develop and implement school improvement plans approved by MENFP Percentage 0.00 100.00 100.00 01-Apr-2016 10-Nov-2016 30-Jan-2023 Comments (achievements against targets): The end target has been achieved. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 6. Development and provision of school teacher training program to sub- components 2.2 and 2.3 public primary project schools Text Training policy for teachers and education system leaders prepared by MENFP Program developed and provided to at least 55 percent of teachers in schools supported by the Project Training delivered to at least 65% of the teachers Training delivered to 66% of teachers The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 38 of 68 01-Apr-2016 10-Nov-2016 16-May-2019 30-Jan-2023 Comments (achievements against targets): The end target has been surpassed. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 11. Proportion of public schools receiving maintenance products for latrines (sud-components 2.2 and 2.3) Percentage 0.00 100.00 100.00 01-Nov-2020 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): All the public schools supported by the Project received maintenance products for latrines each school year. The end target has been achieved. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 12. Percentage of sub- component 2.2 public primary schools offering girls, boys and parent clubs established to develop interpersonal skills (with a Percentage 0.00 75.00 100.00 30-Dec-2018 16-May-2019 30-Jan-2023 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 39 of 68 focus on gender equality) Comments (achievements against targets): The end target has been surpassed. The clubs were implemented in May 2022. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 13. Percentage of teachers and directors in Component 2.2 schools having received in-service training on gender issues Percentage 0.00 95.00 100.00 78.00 16-May-2019 16-May-2019 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): This target has been partially achieved. An additional training on gender-based violence was planned for September-October 2022, but is no longer possible due to the severe country context. It has also been confirmed to be essentially impossible to achieve the 100% target even if every teacher/director has received at least one training related to gender, because the Project ’s M&E system to date has not been set up to uniquely identify each teacher/director participating in a certain training. For these reasons, and given multiple urgent competing priorities, neither a rescheduling of the GBV training nor the organization of another gender-related training is planned before Project closure. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 14. Attendance rate for Text NA TBC 95.00 84% The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 40 of 68 primary education girls in public schools supported by the Project 16-May-2019 16-May-2019 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): The end target is not achieved. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 16. Percentage of public schools in sub-component 2.2 that maintain a "basic" level of learning conditions Percentage 100.00 100.00 100.00 30-Jun-2020 03-Feb-2021 30-Jan-2023 Comments (achievements against targets): The end target has been achieved. Component: Supporting Access to Quality, Non-Public Primary Education in Poor Communities Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 4. Data from the scorecard results used in decision-making for results- Text NA Exiting schools from the results-based financing program if Eight private schools have been exited from the program in SY23 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 41 of 68 based financing of non-public schools supported by the project they do not achieve minimum results as specified in the POM due to not achieving minimum results as specified in the POM. 30-Jun-2018 16-May-2019 30-Jan-2023 Comments (achievements against targets): The end target has been achieved. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 7. Direct project beneficiaries Number 110,487.00 120,000.00 350,000.00 424,616.00 01-Apr-2016 10-Nov-2016 03-Feb-2021 30-Jan-2023 Female beneficiaries Percentage 0.00 50.00 50.00 48.00 Comments (achievements against targets): Direct beneficiaries include students, teachers and school directors. The target and achievement numbers are in terms of cumulative annual numbers. The end target for this indicator has been surpassed. The end target for the sub-indicator for female is not achieved. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 42 of 68 IRI 8. Number of additional classrooms built or rehabilitated at the primary level resulting from project interventions. Number 0.00 100.00 366.00 01-Apr-2016 10-Nov-2016 30-Jan-2023 Comments (achievements against targets): The end target has been achieved. Between the last ISR and Project closure, rehabilitation of 171 classrooms in 19 public schools damaged by the earthquake of August 2021 was completed, increasing the final value for this indicator to 366. Component: Project Management, Monitoring, and Evaluation Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion IRI 10. Stakeholder hotline to MENFP - proportion of calls responded to/resolved within stipulated service standards for response times Percentage 0.00 95.00 97.00 01-Apr-2016 10-Nov-2016 30-Jan-2023 Comments (achievements against targets): The end target has been surpassed. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 43 of 68 B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Strengthen public management of the education sector Outcome Indicators 1. PDOI 1: “Use of an Education Management Information System (EMIS) to enable national implementation of the QAS” Intermediate Results Indicators 2. IRI1. Development of QAS and learning assessments by MENFP and application in schools supported by the Project 3. IRI 5. Development and provision of school director training program to sub-components 2.2 [Public School Improvement Model] and 2.3 [Light Package] public primary project schools 4. IRI 9. System for learning assessment at the primary level 5. IRI 4. Data from the scorecard results used in decision-making for results-based financing of non-public schools supported by the project 6. IRI 10. Stakeholder hotline to MENFP - proportion of calls responded to/resolved within stipulated service standards for response times 21 Key Outputs by Component (linked to the achievement of the Objective/Outcome 1) 1. Outcome: Based on QAS data incorporated into MENFP’s EMIS, performance bonuses have been provided to 48 Project- supported non-public schools that improved their learning conditions score. QAS is being used also by other projects to reach a sector-wide impact 2. An EMIS is established with a technical design developed and endorsed by MENFP, and incorporates available national data on schools, students, MENFP administrative and teaching staff, as well as data generated by the QAS. EMIS is used for school registration nation-wide 3. EMIS data was made accessible across MENFP directorates 4. Student learning standards developed and validated by MENFP in Creole reading, French reading and mathematics 5. Gender dimension is integrated into QAS tools 6. Student learning conditions assessment based on QAS questionnaire conducted in public and non-public schools (beneficiary schools under sub-components 2.2 and 3.2 [Results-Based Financing Model for non-public schools]) 7. Student learning assessments conducted in Creole, French and mathematics in 166 Project-financed schools in three departments, on a total of 6,119 students; key student assessment results were made public on the Project website 8. School director training program based on MENFP’s training policy developed, incorporating gender considerations 21 This indicator is associated with the Project’s Component 4 (Project Management, Monitoring and Evaluation) and not directly related to one Project outcome, but fits best under Outcome 1, so has been included here. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 44 of 68 9. 95% of Project-financed school directors (of beneficiary schools under sub-components 2.2 and 3.2) benefited from training program based on MENFP’s training policy 10. Scorecards produced using data collected through external verification displaying non-public school performance on key MENFP indicators 11. Data from scorecard results were used for decision-making: 8 schools were removed from the results-based financing program for non-public schools due to not meeting minimum results of the program 12. Stakeholder hotline operational, with over 95% of calls consistently responded to/issues resolved within stipulated service standards for response times during the Project life Objective/Outcome 2: Improve learning conditions in selected public and non-public primary schools Outcome Indicators 1. PDOI 2: “Net proportion of schools supported by the Project that improve their school learning conditions score” Intermediate Results Indicators 1. IRI 2. Number of communities proposing and implementing community education plans 2. IRI 3. Proportion of public schools supported by the Project that develop and implement school improvement plans approved by MENFP 3. IRI 6. Development and provision of school teacher training program to sub-components 2.2 and 2.3 public primary project schools 4. IRI 11. Proportion of public schools receiving maintenance products for latrines (sub-components 2.2 and 2.3) 5. IRI 12. Percentage of sub-component 2.2 public primary schools offering girls, boys and parent clubs established to develop interpersonal skills (with a focus on gender equality) 6. IRI 13. Percentage of teachers and directors in Component 2.2 schools having received in-service training on gender issues 7. IRI 14. Attendance rate for primary education girls in public schools supported by the Project 8. IRI 16. Percentage of public schools in sub-component 2.2 that maintain a "basic" level of learning conditions 9. Students benefiting from direct interventions to enhance learning (Corporate Results Indicator) 10. Number of water and sanitation facilities upgraded in Component 2.2 schools based on gender-informed protocol 22 Key Outputs by Component 1. Outcome: 51% of public schools and 55% of non-public schools supported by the Project- improved their learning conditions score 22 This indicator is not reflected in Annex 1.A because it was dropped in ISR Sequence Number 9 (May 2021) and not reincorporated before Project closing into the WGP project portal, which does not allow it to be incorporated into that portal-generated portion of the annex. Introduced during R1, the baseline for this indicator was 0, and the end target was 80 water and sanitation facilities rehabilitated. The last achievement reported in ISRs was 10 water and sanitation facilities rehabilitated. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 45 of 68 (linked to the achievement of the Objective/Outcome 2) 2. 60 communities proposed and implemented community education plans annually, from SY18-19 to SY21-22, in view of improving schools’ learning conditions 3. 100% of schools in the Public School Improvement Model (sub-component 2.2) developed and implemented school improvement plans approved by MENFP, during SY18-19 through SY21-22. During SY17-18, 75% of participating schools developed and implemented school improvement plans (with the remaining schools having developed plans which were implemented during SY18-19) 4. 66% of teachers from schools benefiting from the Public School Improvement Model (sub-component 2.2) and the Results- Based Financing Program for Non-public Schools (sub-component 3.2) participated in training in back-to-school planning, the use of textbooks, and/or the reading instruction approaches in Creole and French (respectively, called M’ap li nèt ale and Je parle bien français) 5. 100% of schools benefiting from the Public School Improvement Model (sub-component 2.2) and the Critical Support for Public School Operation sub-component (2.3) received hygiene and latrine maintenance products during SY20-21 and SY21- 22. Products distributed included cleaning products for hands, classrooms, and playgrounds. 6. 100% of Public School Improvement Model (sub-component 2.2) primary schools implemented girls’, boys’ and parents’ clubs to develop interpersonal skills (with a focus on gender equality) 7. 78% of teachers and directors in sub-component 2.2 schools (721 teachers and directors) received in-service training on gender issues in June 2021 8. Girls attained an attendance rate of 84% in the primary public schools supported by the Project during SY21-22 9. 100% of schools in the Public School Improvement Model (sub-component 2.2) maintained a "basic" level of learning conditions during SY20-21 and 21-22 10. Cumulative annual enrollment of 405,845 benefited from direct interventions to enhance learning under the Project, of which 185,988 (46%) female 11. Ten water and sanitation facilities upgraded in Component 2.2 schools based on gender-informed protocol Objective/Outcome 3: Support enrollment of students in selected public and non-public primary schools Outcome Indicators 1. PDOI 3a: Number of children enrolled in primary school through the provision of community education grants for school access, of which female 2. PDOI 3b: Number of children enrolled in primary school through the provision of tuition waivers to non-public schools, of which female 3. PDOI 3c: Number of children enrolled in primary school through the provision of results-based financing to non-public schools, of which female The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 46 of 68 4. PDOI 3d: Number of children enrolled in the sub-component 2.2 “full-package” public primary schools, of which are female 5. PDOI 3e: Number of children enrolled in sub-component 2.3 “light-package” primary schools, of which are female Intermediate Results Indicators 1. IRI 7. Direct project beneficiaries, of which female 2. IRI 8. Number of additional classrooms built or rehabilitated at the primary level resulting from project interventions Key Outputs by Component (linked to the achievement of the Objective/Outcome 1) 1. Outcome: cumulative annual enrollment of 405,845 in primary schools supported by the Project, including: a. 38,855 in community-based schools over the Project life b. 22,141 in non-public schools via the tuition waiver program during SY18 c. 119,718 in non-public schools via the results-based financing program d. 117,935 in public schools benefiting from the full package of QAS interventions, including QAS data collection and learning assessments, school textbooks and kits, implementation of school improvement plans, teacher training, school feeding, rehabilitation of latrines, and gender clubs e. 107,196 in public schools benefiting from the light package of QAS interventions 2. A cumulative annual number of 405,845 benefited from the Project, of which 46% female. Direct beneficiaries include students, teachers and school directors. The target and achievement numbers are in terms of cumulative annual numbers. 3. 195 classrooms received small rehabilitation works using grants provided by the Project, to improve the schools’ physical conditions as part of their annual school improvement plans. An additional 171 classrooms in 19 schools damaged by the August 2021 earthquake were rehabilitated. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 47 of 68 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Juan Baron, Melissa Ann Adelman Task Team Leaders Prosper Nindorera Procurement Specialist Fabienne Mroczka Financial Management Specialist Emmanuel Ngollo Environmental Specialist Felipe Jacome Safeguards Specialist Asli Gurkan Social Specialist Elena Segura Labadia Counsel Ramiro Ignacio Jauregui-Zabalaga Counsel Aboucabar Magassouba Team Member Andrianirina Michel Eric Ranjeva Team Member Eva Junyent Team Member Marie Tamagnan Team Member Tania Fragnaud Team Member Violeta Arancibia Team Member Supervision/ICR Elena Maria Roseo, Yves Jantzem, Quynh Thu Nguyen Task Team Leaders Escarlata Baza Nunez Counsel Matthieu Louis Bonvoisin Counsel Khadija Faridi Procurement Specialist Mamata Tiendrebeogo Procurement Specialist Maria Cristina Rosa Lucia Villani Procurement Team Rahmoune Essalhi Procurement Team Vladimir Mathieu Procurement Team Beth Wanjeri Mwangi Financial Management Specialist Emeline Bredy Financial Management Specialist Lucas Carrer Financial Management Specialist Emmanuel Ngollo Environmental Specialist Kevin McCall Environmental Specialist Bruce MacPhail Social Specialist Dorothee Georg Social Specialist Aboubacar Magassouba Team Member Amira Nikolas Team Member Andrianirina Michel Eric Ranjeva Team Member Angelica Herrera - Massana Team Member The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 48 of 68 Axelle Latortue Team Member Catalina Danae Lillo Lopez Team Member Christian Mario Ubertini Team Member Claudia Marie Muriel Lopez Team Member Georges Tony Abou Rjaily Team Member Ingrid Sandra Milord Team Member Iris Teluska Team Member Khaled Mohamed Ben Brahim Team Member Luis Barajas Gonzalez Team Member Nathalie Mihajloski Zivkovic Team Member Silvia Guallar Artal Team Member Vania Jocelyn Salgado Hernandez Team Member Zoe Routhier-Drab Team Member B. STAFF TIME AND COST Stage of Project Cycle Staff Time and Cost No. of staff weeks US$ (including travel and consultant costs) Preparation FY16 27.251 245,798.27 FY17 .239 1,789.78 FY18 0 0.00 Total 27.49 247,588.05 Supervision/ICR FY17 32.599 266,907.58 FY18 64.930 639,172.45 FY19 89.397 625,135.61 FY20 104.645 462,959.69 FY21 33.292 173,139.21 FY22 33.844 -1,794.81 FY23 47.656 413,566.27 FY24 0 6,481.70 Total 406.36 2,585,567.70 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 49 of 68 ANNEX 3. PROJECT COST BY COMPONENT Components Amount at Approval (US$M) Actual at Project Closing (US$M) Percentage of Approval (US$M) Improving Institutional Capacity and Governance 3.0 3.6 120% Supporting Access to Quality, Public Primary Education in Poor Communities 13.5 52.8 391% Supporting Access to Quality, Non-Public Primary Education in Poor Communities 11.5 22.0 191% Project Management, Monitoring, and Evaluation 2.0 8.3 415% Contingency Emergency and Response 0 0 0 Total 30.0 86.70 289%.0 The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 50 of 68 ANNEX 4. EFFICIENCY ANALYSIS 1. The Project is expected to have improved learning conditions and supported enrollment of students in selected public and non-public primary schools. A net discounted present value (NDPV) analysis was performed based on Component 2 (Supporting access to quality, public primary education in poor communities), and Component 3 (Supporting access to quality, non-public primary education in poor communities). These activities represent 82 percent of the project’s total credit. The analysis does not capture the broader impacts derived from improved public management of the education sector (Component 1) and other externalities. At appraisal of the Project and at the additional financing in 2019 (R1), the economic benefits were also estimated based on Components 2 and 3. 2. The cost-benefit analysis for the project uses the standard methodology for computing the aggregated private returns to work in adulthood of the students benefitting from project interventions. The effects of school closures due to COVID-19, the 2021 earthquake and insecurity issues are also considered in the analysis (see Table 4.2). Project benefits rely on three channels: (i) Increase of wage incomes resulting from the future enhanced labor market earnings of students. The increase in earnings is associated with the expected gains in student learning in selected public and non-public primary schools because of improved conditions for teaching and learning in schools. The analysis builds on research findings from Latin America on the relationship between interventions and learning outcomes and the relationship between learning outcomes and labor market outcomes (Table 1). Those impacts were not considered at appraisal of the Project, nor at the additional financing (R1). (ii) Increase of beneficiaries’ educational attainment resulting from the decrease of dropout rates due to the different interventions. (iii) Increase of beneficiaries’ educational attainment resulting from tuition waivers. These benefits were not considered at appraisal of the Project. 3. Project costs include the costs of interventions and private costs from households. Table 4.1. Impact of Intervention on Increase in Salary Increase in Salary (%) - School feeding (Subcomponents 2.2 and 2.3) 0.6 - Providing technical assistance and training to school directors - Providing training to teachers on pedagogical skills and content knowledge (Subcomponents 2.2, 2.3 and 3.2) 0.5 0.5 - Classroom rehabilitation (Subcomponent 2.2) 0.9 - Tuition waivers (Subcomponent 3.2) 0.2 - School scorecards (Subcomponent 3.2) 0.6 Source: Patrinos H. A., and Psacharopoulos G. 2020. Returns to education in developing countries. The Economics of Education, Second Edition; Schiefelbein, E., and L. Wolff. 1998. Cost-effectiveness of Education Policies in Latin America: A Survey of Expert Opinion. Washington, DC EDU-109. 4. Table 4.2 summarizes the other points considered in this implementation completion and results report, compared to the assumptions at appraisal of the Project and at the time of additional financing: The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 51 of 68 Table 4.2. Other assumptions at Project appraisal, Additional Financing, and at the Implementation Completion and Results Report Stages At Project Appraisal Additional Financing in 2019 Implementation Completion and Results Report • The discount rate was 8 percent. • The discount rate was 8 percent • The discount rate was 8 percent. • Additional year of schooling increases incomes by 11 percent. • Additional year of schooling increases incomes by 11 percent 23 . • Additional year of schooling increases incomes by 11 percent • At the end of the five years of the Project, the dropout rate would drop to 7 percent for students under Component 2, and to 9 percent for students under Component 3. • Dropout rates decreased by 24 percentage points translated into 0.3 additional years of education per year. • Dropout rate for grade 6 is around 8 percent at the end of the project 24 . • About 20 percent of total beneficiaries in private schools would have not been in school without tuition waivers. • About 20 percent of total beneficiaries in private schools would have not been in school without tuition waivers 25 . • Haiti’s GDP per capita was used as a proxy for average expected income. • Haiti’s GDP per capita was used as a proxy for average expected income for the target beneficiary populations, with an annual increase of 1 percent, and annual inflation of 14 percent 26 . • Haiti faced school closures of 4 months due to COVID-19, 1 month in 2021 due to an earthquake, and 2 months in 2022 due to insecurity issues. The losses due to those school closures are estimated based on recent evidence, suggesting that school closures lasting 3 months could result in a reduction of 2 percent in expected earnings every year for students 27 . 23 Adelman, M., Baron, J., and Heidelk, T. (2015). “The Returns to Education in Haiti”. Working Paper. World Bank 24 Based on data from 2022 on PEQH-financed public schools, 6,688 students were enrolled in grade 6 and 563 of them had dropped out of school at the end of the school year; therefore, the school dropout rate was 8 percent for grade 6. 25 Source: World Bank, Implementation Completion and Results Report of Haiti: Education for All Project Phase II (2018): “It is assumed that 20 percent of the total beneficiaries would not have been enrolled in school without the tuition waiver, based on the results from the household survey “Promoting the Right to Education in Haiti”. However, it is worth mentioning that this assumption is very conservative; in the focus group implemented at the end of the Education for all Project, Phase II, 59 percent of the parents of beneficiary students interviewed responded that without the tuition waiver some of their children would not have been enrolled in Grade 1” 26 Source: World Development Indicators: average inflation of 2017-2019 27 Azavedo J. and al. (2020), Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes. A set of global estimates. World Bank Group. Education Global Practice. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 52 of 68 • The employment rate in Haiti for individuals aged 15 years and above is estimated at 57 percent 28 . • The length of an individual’s working life was assumed to be 40 years. 5. In the analysis, Project benefits consist of incomes of beneficiaries, while Project costs consist of costs of interventions and household costs of schooling. Household costs for one student per year in non- public school are between US$75 and US$118, and between US$58 and US$86 for one student in public schools. 29 However, opportunity costs of not generating income while in schooling could not be estimated. 6. The NPV of the Project is about US$25 million, corresponding to an estimated Internal Rate of Return (IRR) of 9 percent. Those results are higher than the NPV and IRR estimated at the time of Project appraisal and at additional financing (R1), confirming that the Project is a good investment from an economic point of view. The difference is mainly driven by the increase of wage incomes associated with the expected gains in student learning because of improved conditions for teaching and learning in schools. Table 4.3. NPV and IRR at the Project Appraisal, Additional Financing and ICR At Project Appraisal Additional Financing 2019 ICR NPV IRR NPV IRR NPV IRR Discount rate: 8% $13.3 million 6% $10.6 million 9% $25 million 9% Sensitivity Analysis 7. A sensitivity analysis was conducted assuming a, a lower rate of employment of 45 percent, which is dependent on the economic context. The results show that the IRR remains positive, although it decreases to 8 percent. Table 4.4. NPV and IRR Sensitivity to Principal Assumptions NPV IRR Lower rate of employment of 45% $3.8 million 8% 8. Nevertheless, it must be noted that under the challenging FCV circumstances in Haiti during much of the Project life, it has not been possible to fully calibrate the values of improved education and enrollment. While the wage returns to education have been somewhat discounted to accommodate the many periods of school closure, as noted above, it is not possible at this time to gauge more exactly what effect upheavals have had on student learning and dropout rates. It is certain that the education received under the Project was more and better than would have been the case without the Project, but less clear what the specific dollar value of that will eventually be. 28 The study on the evolution of living conditions in Haiti between 2007 and 2012, the social aftershock of the earthquake, estimates that nearly six out of ten people (56.9%) are active (employed or unemployed). Ministère de l’Education nationale et de la Formation Professionnelle (2020), Plan décennal d’éducation et de formation (PDEF). 29 Ministère de l’Education nationale et de la Formation Professionnelle (2020), Plan décennal d’éducation et de formation (PDEF). The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 53 of 68 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS 1. The MENFP provided comments that have been incorporated in the report. Some of the key comments included: • The MENFP agreed with the Project ratings. • The MENFP clarified certain Project activities as implemented and the measurement of certain PDO-level results indicators. • Regarding lessons and recommendations: Given the climate risk and geological fragility of the country, it is important that the project remains open to supporting emergency response following natural disasters, in order to maintain the relevance of project interventions. 2. Global Affairs Canada provided inputs of an editorial nature which are reflected in the main text of this report. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 54 of 68 ANNEX 6. MAIN CHANGES DURING RESTRUCTURING Restructuring & Date Rationale Key revisions Detailed revisions Additional Financing (AF) and Restructuring 1 (R1): May 2019 (Report no. PAD3168) 1- Scale up interventions to increase development impact 2- Add gender dimension to help address two gender gaps (schools’ social and physical environments) affecting girls 3- Support improved climate change resilience 4- Adjust activities based on lessons learned after nearly two school years of implementation 5- Cover cost overruns under Subcomponent 2.2, 3.1, as well as from the project implementation unit’s (PIU) operational costs 1- Number of beneficiary schools doubled 30 (to 130 public schools and 105 non-public schools) 2- Gender dimension added to Project interventions 3- School Health and Nutrition Program expanded to all (130) Project- supported public schools 4- Contingent Emergency Response Component (CERC) added 5- New Safeguard policy triggered (OP 4.09 on Pest Management) 1- AF of US$38 million equivalent from IDA; and of US$18 million equivalent from Global Affairs Canada (GAC), of which US$2 million was made available 2- Updated Results Framework (RF), economic analysis, components and costs, and legal covenants R2: February 2021 (Report no. RES39493) 1- Align to adjusted MENFP priorities due to disruptive national socio-political events, climactic and economic trends, and the COVID- 19 pandemic 2- Reorient and simplify original interventions to generate cost savings directed toward interventions to protect education access and retention and fund COVID-19 response 3- Take into account implementation delays due to change in context 1- Support COVID-19 response through distance learning and communication interventions 2- Introduce a “light package” of QAS interventions to support access to schools by more students 3- Reduce number of schools benefiting from the original (“full”) QAS package to cover a limited number of public schools only, to free up funds 4- Simplify results-based demands on schools 1- Component (C) 1: new activities added on design of distance learning platform and national COVID-19 communication strategy; using reallocated funds 2- Subcomponent (SC) 2.1: all 61 community-based schools shifted to QAS “light package” 3- SC 2.2 reduced in scale (only 70 of 149 existing beneficiary public schools to benefit from full QAS) 4- New SC 2.3 added to fund QAS “light package” for 136 public schools (79 existing SC 2.2 schools 30 Numbers of beneficiary schools listed in this table reflect estimations/expectations at the time of restructuring; numbers may have been adjusted during implementation. For instance, during implementation after this restructuring, 88 new schools were added instead of the planned 69, bringing the total of beneficiary public schools to 149 (of the 731 total public schools in Haiti’s four southern departments targeted by the Project). This more than doubled the number of beneficiary schools as compared to original expectations in the parent Project. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 55 of 68 Restructuring & Date Rationale Key revisions Detailed revisions 5- Overall number of beneficiaries increased 6- Costs reallocated across components 7- Amendment of grant amount in the Grant Agreement (GA) to account for a second installment ($8.9 million) received by GAC, under the TF0B0083, increasing the grant amount from US$2 million to US$10.9 million and 57 new beneficiary schools), using reallocated C2 and C4 funds 5- Revision of SC 3.2: grants to non- public schools conditioned on school compliance and processes rather than on student learning outcomes; fewer student learning assessments and less teacher / director training; and 18 new beneficiary schools added, participating in “light package” for non-public schools 6- C 4: impact evaluation cancelled due to inability to conduct it in the volatile context 7- Updates to RF (see Annex 1A) R3: November 2021 (Report no. RES47771) 1- Respond to emergency needs of the education system and support resumption of service delivery to Project-supported schools, following August 2021 earthquake in southern departments of Haiti 2- Allow Project to continue to disburse fluidly by reallocating between disbursement categories of the Project’s grants, thereby correcting cost under- and over-estimations, cost overruns and adjusting to actual expenses. 1- Funds reallocated across disbursement categories to have sufficient funds to pay tuition waivers to non-public schools 2- Update of the Environmental and Social Management Framework (ESMF) 1- ESMF adjusted to include construction of about 100 temporary structures 2- Revisions to disbursement categories: Grant IDA-D1230: Category 1 increased; Categories 2, 3, and 4 reduced. Grant IDA-D4700: Category 1 decreased, Category 3 increased. Grant TFB0083 (“Support for Girls Access to Secondary Education in Haiti”): Funds reallocated from Category 2 to Category 3, allowing continued disbursement of funds covering grants and operating costs to public schools. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 56 of 68 Restructuring & Date Rationale Key revisions Detailed revisions R4: April 2022 (Report no. RES50760) 1- Amend the GA for Project TF0B0083, funded by GAC, to account for new (third) tranche of funds received by GAC 1- Second amendment to GA of the TF0B0083 1- CAD4 million (US$2.2 million) added to the TF0B0083 R5: September 2022 (Report no. RES51818) 1- Amend the GA for Project TF0B0083, to account for new (fourth) tranche of funds received by GAC 1- Third amendment to GA of the TF0B0083 1- CAD5.5 million (US$4.7 million) added to the TF0B0083 R6: October 2022 (Report no. RES53417) 1- Allow completion of repair works in schools, delayed due to rapidly deteriorated country safety and security context. This included fuel shortages, road blockages, interruption of business and social activities, and socioeconomic damages caused by destructive crimes such as looting and kidnapping 1- Extend Project closing date by three months, for all three GAs funding it. 1- Project closing date extended to January 30, 2023 R7: January 2023 (Report no. RES54481) 1- Correct WBG funds calculation error whereby US$200,000 were not deducted from the last tranche of GAC financing, added in the September 2022 restructuring (R5) 1- Reduced total amount of fourth tranche of GAC funds available to the Project, to reflect the total corrected allocation 1- Cancellation of US$200,000 under TF0B0083-HT, decreasing the total grant amount from US$17.8 million to US$17.6 million (and an associated reduction of US$200,000 in Category 1) The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 57 of 68 ANNEX 7. QUALITY ASSURANCE SYSTEM (QAS) 1. The development and application of a primary education QAS was at the heart of the Project’s design. The QAS was originally designed as a set of norms and standards, indicators, tools, and procedures used to: (i) monitor school quality, teacher competencies, and student learning; (ii) implement actions to support: favorable learning conditions in schools, the professional development of teachers, and student learning and socio-emotional development; and (iii) measure the impact of program interventions. It had begun to be piloted during Project preparation, and under the Project was intended to further the following goals: help establish and track key dimensions that matter for education outcomes; increase school accountability for improvement; and promote the efficient use of public financing. More specifically, the QAS would support MENFP in ensuring that school environments were conducive to learning, which included instruction in classrooms meeting a minimum standard of quality. Under the Project’s first component, MENFP would be accompanied to establish systems and the technical capacity to administer and implement the QAS; under the second and third components, the QAS tools would be applied to monitor and measure progress in public schools, leading to their development of improvement plans (Component 2), and enable non-public schools to meet eligibility requirements for public financing, as well as monitor and measure progress in these schools (Component 3). Figure 7.1. Summary of QAS Conceptual Framework 2. The QAS is composed of nine dimensions, organized into three pillars (School Quality, Teacher Competencies, and Student Learning); each dimension is tracked using pre-defined indicators, on which data is collected using several sets of tools. The QAS’s nine dimensions and its main indicators are as follows: Figure 7.2. QAS Dimensions and Indicators The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 58 of 68 3. Under the Project, tools were developed or refined and administered for the first five dimensions 31 (School Quality pillar) and on the ninth dimension (Student Learning pillar); the other 31 These five dimensions are: (i) leadership; (ii) pedagogy; (iii) school environment user friendliness; (iv) community participation; and (v) personnel. 1. Leadership •Teamwork •Clear educational objectives •Monitoring of educational objectives 2. Pedagogy •Teaching practices •Pedagogical supervision •Teaching and learning conditions 3. School Environment User Friendliness •School climate •Violence-free school •Disciplinary code 4. Community Participation •School management committee •Parent participation 5. Personnel •Vocational training •Performance evaluation •Professional satisfaction 6. Efficiency & Infrastructure •Number of of students per classroom, repeaters, absenteeism, dropout, student-teacher ratio, teacher absenteeism •Infrastructure: playground, library, food, electricity, drinking water, toilets A 7. Teacher Competencies •Minimum teaching competencies 8. Student Socioemotional Development •Attitudes toward school •Relationship with others •Joy •Anxiety 9. Student Learning Levels •Haitian Creole •French •Mathematics The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 59 of 68 dimensions were not operationalized. Three tools were implemented on School Quality, each of which provided data on school learning conditions as tracked through the five dimensions—a teacher questionnaire, school director questionnaire, and school questionnaire. Each questionnaire contained questions that provided data on the indicators for each of the five dimensions and that allowed schools to be scored. On Student Learning, student assessments in Haitian Creole, French and Mathematics were administered, and a set of learning standards for these core subjects were developed. Data were not collected on the sixth, seventh and eight dimensions (Efficiency & Infrastructure, Teacher Competencies, and Student Socioemotional Development), due to limited capacity to develop and implement more tools and the need to prioritize interventions. In addition, after the first pilot round of QAS implementation in 2017 and as planned in Project design, it was decided to shorten and simplify the existing tools, which proved to be overly complex for the low-capacity context. As a results, only the teacher and school director learning conditions questionnaires were used as data collection tools for the first five QAS dimension in subsequent data collection exercises for the School Quality pillar 32 . Questionnaires were administered in Creole, in paper format during the first year (2017) and in electronic format (using tablets) in subsequent years (2019-2020, 2020-2021 and 2021-2022 school years). The results from the 2019- 2020 school year served as the baseline for results framework tracking. 4. To assess school quality, based on teachers’ and directors’ responses to the QAS learning conditions questionnaires, for each indicator and each of the five dimensions, schools were rated using a 4-level scale of school quality: Insufficient (indicating a score of less than 5 on a 10-point scale), Basic (score ≥5 and <7.5), Sufficient (score ≥7.5 and <9), or Superior (score ≥9 and ≤10) level. Scores were also consolidated to provide an overall quality level for each school of Insufficient, Basic, Sufficient or Superior, taking into account all dimensions and indicators assessed. Approximately annually, schools were re- assessed, allowing MENFP and the schools to track their areas of progress, and allowing the selected public and community-managed schools which benefitted from Project-financed school improvement plans, to target their improvement plans accordingly. Non-public schools, expected to hold primary responsibility for meeting MENFP requirements, were expected to work on improvements largely outside the scope of Project financing, with financial incentives (performance bonuses of 20 percent of the per- student grant amount) offered for non-public schools showing a certain level of score improvement. 33 5. With respect to the Student Learning dimension, third graders were administered tests on Creole, French, and Mathematics subjects in 2021, and their scores were used to assess student learning in participating schools. Participants at the school level were trained to administer and correct the tests; teachers corrected the tests using a correction guide, analyzed the results, and were expected to use them to make pedagogical adjustments. Consolidated student results were displayed at the schools. Based on the number of question items answered correctly, students were ranked as scoring “Below low,” “Low,” “Medium,” “High,” and “Advanced.” The results displayed at schools showed the percentage of students having reached the Advanced, High, Medium or Low learning levels. (The number of questions required to be correct for each level varied based on the subject, and student results were compared against MENFP’s learning standards—linked to the grade 3 primary school curriculum). Student results were also expected to inform school’s improvement plans while monitoring learning outcomes. However, these tests were only administered once during the Project life, due to the difficult implementation context and to the eventual deconditioning of student learning results from school tuition waiver grants. 32 In the case of non-public schools, verification firm data were used to measure compliance with compliance with QAS criteria. 33 Schools with a score of at least 5 in their overall quality rating and showing an increase of at least one point (to a score of six or higher) in their overall school quality rating from the beginning to the end of a school year, qualified for a performance bonus. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 60 of 68 ANNEX 8. ORIGINAL PROJECT RESULTS CHAIN AND THEORY OF CHANGE Table 8.1: Original Project Results Chain, as presented in the PAD Sample Activities by Component Outputs Project Outcomes Long-term Outcome (Higher level Objective) Improving Institutional Capacity and Governance - Training - Supervision inputs - Quality Assurance System - Assessments and consultancies to improve processes of data collection, analysis, planning and accreditation o Trained staff at MENFP o Functioning M&E system o More supervision and higher quality support to schools o Consistent approach to building capacity of director and teachers o Improved accreditation process Improved capacity to collect and analyze data, plan, and make decisions Increase student learning in language and math Supporting Access to Quality, Public Primary Education in Poor Communities - Assessments of learning conditions, student learning, and teacher practices - Training school directors and teachers - Provision of grants for school improvement plans: rehabilitation according to standards (including energy, water and sanitation, and security), pedagogical materials, etc. - Provision of multi-school interventions, such as school feeding o Assessments of learning conditions and learning outcomes of participating schools using QAS completed o School directors trained in effective management and leadership o Teachers trained based on needs and active learning strategies o Improved physical conditions at school Learning conditions as measured through QAS improved Supporting Access to Quality, Non- Public Primary Education in Poor Communities (QEAP) - Assessment of learning conditions and student learning - School grants to non-public schools serving poor children to waive tuition - Communication campaigns - Provision of multi-school interventions, such as school feeding o Assessments of the learning conditions and learning outcomes of participating schools using QAS completed o Schools lacking capacity receive director and teacher training to respond to program incentives and improve quality o Per-student financing provided based on meeting standards of QAS Learning conditions as measured through QAS improved Accountability of non- public schools increased (only well-performing schools get financing) Project Management, Monitoring and Evaluation o Provision of training, materials, and other support needed to improve functioning of key ministry directorates o Data collection for Project monitoring and system monitoring Ministry staff possesses improved skills and tools for completing duties The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 61 of 68 Figure 8.1: Project Theory of Change as presented in May 2019 Restructuring (R1) The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 62 of 68 ANNEX 9. SCHOOL CLOSURES BETWEEN 2018 AND 2023 IN HAITI Figure 9.1. School closure days during 2018-2023 1. Over the course of the Project, Haiti suffered a series of events and shocks that led to multiple school closures, resulting in days of learning lost for primary students. The below information recounts available data, organized by school year, on school closures and estimated days of schooling lost. 2018-2019 School year The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 63 of 68 2. According to the official calendar, school was supposed to start on September 3, 2018, and end on June 28, 2019, which represents 192 school days. Schools were closed several times during the 2018- 2019 school year primarily due to violent protests. This would represent about 20 days of school closures according to an article from Le Nouvelliste. 34 Due to the “peyi lòk”, (country sociopolitical lockdown) in late November, schools were closed for ten days 35 . As of February 7, 2019, schools closed again due to violent protests. According to an article published by Le Nouvelliste citing a communication from the Ministry of National Education and Vocational Training (MENFP), all schools (especially public schools) allegedly restarted from February 26 36 . The schools were therefore closed 3 weeks in February. 2019-2020 school year 3. Originally, the school calendar released by MENFP scheduled 189 days of schooling from September 9 to July 1, 2020 37 . Because of the “peyi lòk”, 60 days of school were lost from September to November 2019. Following the protest period, a new school calendar was released that included 147 school days from December 2 through July 17, 2020. 38 4. The Haitian government decided to shut down schools on March 19, 2020 due to the COVID-19 pandemic. A catch-up period was planned from August 10 to October 22, 2020, which represents approximately 50 days of teaching. Students in the ninth grade of the fundamental cycle and the third grade of the secondary cycle were scheduled to return to school on August 10 to prepare for their exams. All classes were expected to start on August 17. The end of classes was supposed to take place on October 9 and the exams were scheduled for October 12 to 22 39 . Some schools decided to reopen before the official date 40 . However, this period was also disturbed by social protests and teacher strikes, preventing some students from returning to school. Although schools were officially open, students gradually returned to class in November and December 2020. 41 5. According to a UNICEF press release, 6 out of 10 students lost 80 school days and 4 out of 10 students lost 20 school days. Due to the “peyi lòk”, UNICEF estimates that 60 percent of schools have lost 60 school days during the first semester. According to the same release, 4 million children lost 70 school days due to the closure of schools because of the COVID 19 pandemic 42 . MENFP estimates that more than 70 percent of students lost a full school year due to the “peyi lòk” and the COVID 19 pandemic. 43 To compensate for school closure, MENFP created a digital platform for remote courses called PRACTIC. The ministry also had planned to distribute printed materials as well as broadcasting lessons and reading and writing programs on television and radio. Since the use of these tools has not been documented, there is no official data regarding their implementation. 44 34 Lambert 2019. 35 Ricardo 2019. 36 Lambert 2019 37 Haïti Libre 2019. 38 USAID 2021. 39 Alternative report by a coalition of Haitian and regional civil society organizations on the right to education in Haiti, July 15, 2021. Accessed on April 7, 2023, at https://uprdoc.ohchr.org/uprweb/downloadfile.aspx?filename=9446&file=FrenchTranslation 40 Daudier 2020. 41 USAID 2021. 42 UNICEF 2021. 43 UNICEF 2020. 44 Stone and Ziegler 2021. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 64 of 68 2020-2021 school year 6. The initial school calendar planned 172 days of schooling from November 9, 2020 to July 30, 2021. 45 However, the Haitian government ordered school closures as of June 11, 2021, as part of the extension of the public health state of emergency decided on May 22, 2021 46 . This represents a loss of 40 school days, representing 132 school days during the 2020-2021 school year. During the 2020-2021 school year some schools or classrooms closed for several days or weeks due to COVID-19. There was no official directive on how schools should proceed when COVID-19 cases were detected in schools. In an interview published in May 2021 by Le Nouvelliste, the vice-president of the Professional Association of Private Schools, Roody Edmé, stated that school or class closures were decided on a case-by-case basis. 47 Some schools completely closed while others shut down the classes concerned by COVID-19 cases. Some schools rotated student attendance days. Finally, other schools implemented remote courses as of spring 2020 48 . 2021-2022 school year 7. The beginning of the school year was postponed to October 4, 2021 in the departments of Sud (South), Grande’Anse and Nippes due to the August 14, 2021 earthquake. The new school calendar anticipated 176 days of classes with an end date of June 30. 49 According to a report by the GoH’s General Directorate of Civil Protection dated September 4, 2021, 171 schools were destroyed, representing 16 percent of the total number of schools in the three departments affected by the earthquake of August 2021. 566 schools were damaged. MENFP had evaluated 1,064 schools out of the 2,800 existing in these three departments at the time of publication of these data. According to a UNICEF report, 1,250 schools were destroyed or damaged in the departments of Nippes, Sud, and Grand’Anse. 50 According to the United Nations Office for the Coordination of Humanitarian Affairs (OCHA), during the morning of January 24, 2022, two aftershocks led to the closure of all schools in the departments of Sud and Nippes. 51 8. The Ministry of Public Works, Transport and Communications (MTPTC) announced the construction of temporary buildings to accommodate students. There is no official data on the construction of these temporary schools. 52 Students from some destroyed schools were welcomed in other schools that escaped damage from the earthquake. For example, students from the public school Pressoir in the city of Jérémie were able to have classes in the building of the Adventist school of Jérémie. 53 2022-2023 school year 9. Because of insecurity and violence, the first day of school, originally scheduled for September 5, was postponed twice. Classes started gradually in November 2022. 45 Haïti Libre 2020. 46 Haiti Ministry of Public Health and Population 2021. 47 Sénat 2021. 48 Stone and Ziegler 2021. 49 Haïti Libre 2021. 50 Haiti 2022 End-Year Humanitarian Situation Report. 51 OCHA 2022. 52 Verdieu 2021. 53 Janvier 2021. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 65 of 68 Table 9.1. Percentage of open schools by department from November 7 to December 12, 2022 7-Nov 14-Nov 22-Nov 24-Nov 28-Nov 5-Dec 12-Dec Average Nippes 18% 35% 54% 58% 64% 84% 90% 58% Sud 19.97% 19.97% 73% 90% 97% 60% Grand’Anse 15.3% 24,18 27% 76% 78% 49% Sud-Est 19.14% 22,33 36% 45% 45% 36% Average 18% 35% 27.11% 39.02% 50% 73.75% 77.50% Source: MENFP Twitter and Facebook data 10. As of November 7, 2022, in the department of Nippes 17.61 percent of schools opened representing 127 schools including 108 primary schools (out of 577) according to data provided by MENFP on Twitter. During the week of 14 November, 249 schools (high schools and fundamental schools) opened, i.e., 35 percent of the schools in the department of Nippes, according to data published by Le Nouvelliste. In the Sud department, 235 primary schools and 22 high schools reopened in several cities except in the city of Les Cayes. According to the head of the MENFP in the Sud department, Director Auxène Roc, schools in the city centers did not open the first week of November. Only 60 schools, mostly in the school district of Aquin, had opened their doors to students. 54 In an interview with the newspaper Le Nouvelliste, the departmental director of Grand’Anse, Marcel Jeanty, said that some schools had been able to open classes in the town of Jérémie from mid-November. These are the Institution Saint-Jean Evangéliste, Beraca, Collège Jean Wesley, Lycée Saint-Luc and the Institution of Applied Sciences (ISA). 55 11. According to data published by MENFP and reported by UNICEF, 73 percent of schools in Haiti were open at the end of December 2022. In its December 28, 2022 press release, MENFP stated that in some large cities, schools had restarted with 100 percent of students. However, due to the increasing insecurity and violence in the education sector, many school directors had to close schools to protect children. UNICEF estimates that by January 2023, students had lost at least one and a half school days per week. As a result, children could lose 36 days of schooling by the end of June. 56 Figure 9.2. Reopening of Schools in Haiti in November-December 2022 Source: MENFP Facebook page 54 Césaire 2022. 55 Janvier 2022. 56 UNICEF 2023. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 66 of 68 ANNEX 10. SUPPORTING DOCUMENTS Adelman, Melissa; Holland, Peter (2015). “Increasing Access by Waiving Tuition: Evidence from Haiti”. Policy Research Working Paper 7175. The World Bank Group. Education Global Practice. Cambridge Education. (2019). “Analyse sectorielle détaillée PDEF 2018-2028 Haïti” MENFP (2020). “Plan décennal d’éducation et de formation (PDEF) ”. Port-au-Prince. Rozo, Marcela; Singh, Raju. (2019). “Haiti Risk and Resilience Assessment: Moving from the ‘What’ to the ‘How’”. Washington, DC: The World Bank Group. WBG (2007). “Social Resilience and State Fragility in Haiti”. Report No. 40927. The International Bank for Reconstruction and Development/The World Bank Group. WBG (2014). “Investing in people to fight poverty in Haiti: Reflections for evidence-based policy making”. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/222901468029372321/Reflections-for-evidence-based- policy-making WBG (2015a). “Haiti: Towards a New Narrative: Systematic Country Diagnostic”. Report No. 97341. Latin America and the Caribbean Region. WBG (2015b). “Country Partnership Framework for the Republic of Haiti for the Period FY16-FY19; Report No. 98132-HT”. Haiti Country Management Unit. Latin America and the Caribbean Region; The International Finance Corporation; Multilateral Investment Guarantee Agency. WBG (2016). “Project Appraisal Document for a Providing an Education of Quality in Haiti Project”. Report No. PAD1690. Education Global Practice. Latin America and the Caribbean Region. WBG (2018a). “Implementation Completion and Results Report for the Haiti – Education for All Project – Phase II”. Report No. ICR00004083. Education Global Practice, Latin America and Caribbean Region. WBG (2018b). “Performance and Learning Review of the Country Partnership Strategy for the Republic of Haiti for the Period FY16-FY19”. Report No. 124812-HT. Haiti Country Management Unit, Latin America and Caribbean Region. WBG (2021a). “Bank Guidance. Implementation Completion and Results Report for Investment Project Financing Operations”. Washington, DC: The World Bank Group. https://documentsinternal.worldbank.org/search/31564801 WBG (2021b). “Implementation Completion and Results Report for a Haiti Statistical Capacity Building in Education Grant.” Education Global Practice, Latin America and Caribbean Region. WBG. Implementation Status & Results Reports for the Project - Sequence Numbers 1 to 12. The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 67 of 68 References for Annex 9: School closures between 2018 and 2023 in Haiti Haiti National Emergency Operations Center (2021). National Emergency Operations Center Progress Report. Accessed on April 7, 2023, from https://protectioncivile.gouv.ht/wp-content/uploads/2021/09/Seisme- du-14-Aout-2021._-Rapport-de-situation-detape- No-1-du-Coun.pdf Césaire, M. (2022, November 16). “Far from the big cities, students are heading back to school”. The Nouvelliste. Accessed on April 7, 2023, from https://lenouvelliste.com/article/238952/loin-des-grandes- villes-des-eleves-reprennent-le-chemin-de-lecole Daudier, V. (2020, August 3). “Schools are reopening in Haiti, nearly four months later”. The Nouvelliste. Accessed on April 7, 2023, from https://lenouvelliste.com/article/219318/des-ecoles-rouvre-leurs- portes-en-haiti-pres-de-quatre-mois-apres Haïti Libre. (2019, July 31). Haiti – “FLASH: 2019-2020 school calendar, all dates”. Accessed on April 7, 2023, at https://www.haitilibre.com/article-28379-haiti-flash-school-calendar-2019-2020- all-dates.html Haïti Libre. (2020, October 29). Haiti – “FLASH: Final school calendar 2020-2021 (official)”. Accessed on April 7, 2023, at https://www.haitilibre.com/article-32156-haiti-flash-calendrier-scolaire-final-2020-2021 - official.html Haiti Ministry of Public Health and Population (2021, May 31). "Order renewing, for a period of fifteen days, the state of health emergency declared throughout the national territory by the order of May 22, 2021”. Accessed at https://www.mspp.gouv.ht/wp-content/uploads/Special-N-28-Etat-durgence.pdf January, F. (2021, November 23). “Grand'Anse/Earthquake: still no adequate structures to accommodate students from public schools”. The Nouvelliste. Accessed on April 7, 2023, from https://lenouvelliste.com/article/232777/grandanseseisme-toujours-pas-de-structures-adequates-pour- accueil-les-eleves-des-ecoles-publiques January, F. (2022, November 15). “Schools are reopening in Jérémie after months of crisis”. The Nouvelliste. Accessed on April 7, 2023, from https://lenouvelliste.com/article/239014/des-ecoles-rouvre-leurs- portes-a-jeremie-apres-des-mois-de-crise Lambert, R. (2019, February 26). “Public school students return to class”. The Nouvelliste. Accessed on April 7, 2023, from https://www.lenouvelliste.com/article/198598/les-eleves-des-ecoles-publiques-reviennent- en-classe OCHA (2022). “Haiti: Aftershocks - Flash Update No. 1 (As of January 24, 2022)”. In Reliefweb. Accessed on April 7, 2023, from https://reliefweb.int/report/haiti/ha-ti-r-pliques-sismiques-flash-update-no-1-au-24- janvier-2022 Ricardo, A. (2019, November 29). “A consensus on the functioning of schools and universities in times of crisis”. The Nouvelliste. Accessed on April 7, 2023, from https://www.lenouvelliste.com/article/195264/un- consensus-sur-le-fonctionnement-de-lecole-et-des-universites-en-temps-de- crisis The World Bank Providing an Education of Quality in Haiti (PEQH) (P155191) Page 68 of 68 Sénat, J. (2021, May 31). “COVID-19: the epidemic is reaching schools”. The Nouvelliste. Accessed on April 7, 2023, from https://www.lenouvelliste.com/article/229422/covid-19-lepidemie-gagne-les-ecoles Stone, R.; Ziegler, N. (2021). “Challenges and opportunities in the education system in Haiti”. Working Series Papers SUMMA. No. 12. Published by SUMMA. Santiago de Chile. UNICEF (2020). “UNICEF Haiti Country Office Annual Report 2020”. Accessed on April 7, 2023, at https://www.unicef.org/media/100291/file/Haiti-2020-COAR.pdf UNICEF (2021, March 3). “Three out of five children who have lost an entire school year in the world live in Latin America and the Caribbean” [Press release]. https://www.unicef.org/haiti/communiqu%C3%A9s- de-presse/3-children-out-of-5-who-have-lost-ann%C3%A9e-scolaire-enti%C3% A8re-in-the-world UNICEF (2023, February 9). “Haiti. Armed violence against schools has increased ninefold in one year”. [Press release]. https://www.unicef.org/press-releases/haiti-armed-violence-against-schools-increases-nine- fold-one-year-unicef USAID (2021, July). “Rapid education and risk analysis Haiti”. Evaluation and Survey Services Verdieu, J.J (2021, September 9). “Temporary structures will temporarily replace demolished schools in the South to facilitate the return to school”. The Nouvelliste. Accessed on April 7, 2023, from https://lenouvelliste.com/article/231508/des-hangars-remplaceront-provisoirement-ecoles-demolies- dans-le-sud-pour-faciliter-la-rentree-des- Classes