Implementation Completion and Results Report - Haiti Education for All Project

Document of The World Bank Report No: ICR2425 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H2860 IDA-H5880) ON A GRANT IN THE AMOUNT OF SDR 24.8 MILLION (including additional financing) (US$37.0 MILLION EQUIVALENT) TO THE REPUBLIC OF HAITI FOR A EDUCATION FOR ALL PROJECT IN SUPPORT OF THE FIRST PHASE OF THE EDUCATION FOR ALL PROGRAM November 30, 2012 Human Development Department-Education Sector Caribbean Country Management Unit-Haiti Latin American and the Caribbean Region 1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized CURRENCY EQUIVALENTS (Exchange Rate Effective August 30, 2012) Currency Unit = Haitian Gourdes (HTG) HTGl.00 = US$ 0.0238 US$ 1.00 = 42.05 FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS APG Adaptable Program Grant BND Bureau de Nutrition et de Developpement (Office for Nutrition and Development) CAS Country Assistance Strategy CCT Conditional Cash Transfer CDB Caribbean Development Bank CIDA Canadian International Development Agency DAA Department of Administrative Affairs DAEPP MENFP's Department for Private Education and Partnership DDEs Regional Education Departments DHS Demographic and Health Survey 2005 DPCE Planning Department DRH Department of Human Resources EFA FTI Education For All Fast Track Initiative EFACAPs Ecole Fondamentale d'Application et de 1'Appui Pedagogique (Practical Application and Teaching Support Basic School) EGRO I/II Economic Governance and Reform Operations (I/II) EPCA Emergency Post-Conflict Assistance Program EU European Union FMA Financial Management Assessment FMRs Financial Management Reports GDP Gross Domestic Product GER Gross Enrollment Rates (total number of students enrolled in a particular level divided by the total population of school-age for that same level). GIS GIS-based school mapping database GOH Government of Haiti HIPC Heavily Indebted Poor Countries ICF Interim Cooperative Framework ICT Information and Communication Technology IDB Inter-American Development Bank ii IFR Interim Financial Report I-PRSP Interim Poverty Reduction Strategy Paper IROR Internal Rate of Return ISN Interim Strategy Note ISR Implementation Status and Results Report LICUS TF Low Income Country Under Stress Trust Fund M&E Monitoring and Evaluation MDG Millennium Development Goals MENFP Ministere de 1'Education Nationale et de la Formation Professionnelle (National Ministry of Education and Vocational Training) MIS Management Information System MTEF Medium Term Expenditure Framework NEPO National Education Partnership Office NER Net Enrollment Rates (total number of students with the appropriate school-age for a particular level, divided by the total population of that age) NGO Non Governmental Organization NSFP National School Feeding Program PCF Post-Conflict Fund PCR Primary Education Completion Rate PCU Project Coordination Unit PNCS Programme National de Cantines Scolaires (National School Canteen Program) PPF Project Preparation Facility SMC School Management Committees TORs Terms of Reference UNESCO United Nations Education, Scientific and Cultural Organization UNICEF United Nations Children's Fund USAID United States Agency for International Development Vice President: Hasan Tuluy Country Director (Special Envoy): Alexandre V. Abrantes Sector Manager: Reema Nayar Project Team Leader: Patrick Ramanantoanina ICR Team Leader: Patrick Ramanantoanina ICR Primary Author: Richard J. Carroll iii HAITI EDUCATION FOR ALL ADAPTABLE PROGRAM GRANT PHASE 1 (APG-1) CONTENTS B. KEY DATES ................................................................................. V C. RATINGS SUMMARY..........................................................................V D. SECTOR AND THEME CODES..................................................................VI E. BANK STAFF ................................................................................ VI F. RESULTS FRAMEWORK ANALYSIS.............................................................ViI G. RATINGS OF PROJECT PERFORMANCE IN ISRS ....................... ............................ IX H. RESTRUCTURING (IF ANY) ..................................................................... X I. DISBURSEMENT PROFILE......................................................................X1 . PROJECT CONTEXT, DEVELOPMENT OBJECTES AND DESIGN.......................................... 1. Context at Appraisal........................................................ 1 1.2. Original Project Development Objectives (PDO) and Key Indicators.................... 3 1.3. Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification ............................................................. 3 1.4. Main Beneficiaries....................................................... 3 1.5. Original Components..................................................... 4 1.6. Revised Components...................................................... 5 1.7. Other signiicant changes ................................................................................. 5 I. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ................................... 6 2.1 Project Preparation, Design and Quality at Entry............................ ..... 6 2.2 Implementation............................................................................................ 8 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization .. ............... 10 2.4 Safeguard and Fiduciary Compliance PDOn.........................................11 2.5 Post-completion OperationNext Phase.........................................12 1. ASSESSMENT OF OUTCOMES ................................................................ 12 3.1 Relevance of Objectives, Design and Implementation ............................... 12 3.2 Achievement ofProject Development Objectives .......................... ........ 13 3.3 Efficiency ............................................................. 15 3.4 Justfcation of Overall Outcome Rating ....................................... 15 3.5 Overarching Themes, Other Outcomes and Impacts ....................... ..... ... 16 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops.. ............. 17 IV. ASSESSMENT OF RISK To DEVELOPMENT OUTCOME.............................................. 17 V. ASSESSMENT OF BANK AND BORROWER PERFORMANCE .............................. .......... 17 5.1 Bank Performance ...................................................... 17 5.2 Borrower Performance ..................................................... 18 VI. LESSONS LEARNED......................................................................... 19 VII. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES/PARTNERS ................ 20 Annex 1. Project Costs and Financing ..... ........................................ 22 Annex 2. Outputs by Component ................................................... 23 Annex 3. Economic and Financial Analysis Workshops........................................... 26 Annex 4. Bank Lending and Implementation SupportSupervision Processes ............. ....... 28 Annex 5. Beneficiary Survey Results ................................................ 30 Annex 6 Stakeholder Workshop Report and Results.............................. ....... 31 Annex 7. Summary of Borrower's ICR and/or Comments on Draf ICR ............... ............ 32 Annex 8. Comments of Cofinanciers and Other PartnersStakeholders ........................... 45 Annex 9. List of Supporting Documents IR........ ....................................... 46 1V A. Basic Information - HT Education For All Country: Haiti Project Name: Adaptable Program Grant Phase 1 Project ID: P099918 L/C/TF Number(s): IDA-H2860,IDA-H5880 ICR Date: 11/30/2012 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: APL Borrower: HAITI Original Total XDR 16.80M Disbursed Amount: XDR 24.70M Commitment: Revised Amount: XDR 24.80M Environmental Category: C Implementing Agencies: Ministry of Education Cofmnanciers and Other External Partners: Caribbean Development Bank B. Key Dates Revised / Actual Process Date Process Original Date (s) Concept Review: 10/05/2006 Effectiveness: 09/28/2007 09/28/2007 05/27/2010 Appraisal: 01/29/2007 Restructuring(s): 05/7/2012 Approval: 04/26/2007 Mid-term Review: 05/04/2009 Closing: 01/31/2011 05/31/2012 C.Ratings Summnay C.1 Performanc e R ag b yl CR Outcomes: MS Risk to Development Outcome: Substantial Bank Performance: MS Borrower Performance: MS C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: MS Government: S v Quality of Supervision: MS Implementing MS Agency/Agencies:. Overall Bank Overall BorrowerMS Performance: S Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation Indicators QAG Assessments Rating Performance (if any) Potential Problem ProjectYes Quality at EntryYes at any time (Yes/No): (QEA): Problem Project at any N Quality of No Yes time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 13 13 Other social services 8 8 Primary education 75 75 Teriay duaton4 4 The me Co de =(as %Oo oft to ta IBa nk finan cing) Administrative and civil service reform 17 17 Child health 17 17 Education for all 33 33 Social safety nets 33 33 E.Bank S taff Positions At ICR At Approval Vice President: Hasan Tuluy Pamela Cox irc Alexandre V. Abrantes (Special LCountry Director: Eno)Caroline =Anstey Envoy) Sector Manager: Reema Nayar Eduardo Velez vi Project Team Leader: Patrick Philippe Ramanantoanina Samuel C. Carlson ICR Team Leader: Patrick Philippe Ramanantoanina ICR Primary Author: Richard J. Carroll F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To (a) improve access and equity of primary education; (b) operationalize partnerships between public and non public sectors; and (c) build capacity to assess learning outcomes. Revised Project Development Objectives (as approved by original approving authority) No change in the PDO, though the original Development Credit Agreement had erroneously used the program objective instead of the Project objective (see Section 1.2). The May 27, 2010 restructuring paper pointed this out and the Legal Agreement was amended accordingly. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Inictor BaeieIau approval Target Completion or documents) Values Target Years Indicator 1 Children receiving tuition subsidies (person/years) Value quantitative or 0 210,000 390,000 425,000 Qualitative) Date achieved 03/21/2007 01/31/2011 05/31/2012 5/31/2012 General target met and exceeded after upward revision. The revised target included the impact of the Project and of other donor assistance catalyzed by the Comments Project. Specific outcome targets for the Project were not established. Project (incl. % attributed impact was 285,000, with an additional 140,000 children receiving achievement) subsidies through funding from other donors. At the 5/27/2010 restructuring, the indicator was changed from "Number of Poor Children benefiting from Public subsidies to attend Non Public Schools." Indicator 2 : Grade 1 capacity utilization in participating schools Value (quantitative 47% 100% /A /A or Qualitative) Date achieved 03/21/2007 01/31/2011 Comments Originally "Intake Rate for Grade 1 in Participating Schools," this indicator was (incl. % dropped at the 5/27/2010 restructuring because capacity utilization was not an achievement) issue after the January 2010 earthquake. vii Indicator 3 : Number of teachers receiving pre-scrvice teacher education Value (quantitative 0 3,600 /A N/A or Qualitative) Date achieved 03/21/2007 01/31/2011 Comments This indicator was shifted to Meeting Teachers' Needs project, which spun of from component 2 of this (APG-1) Project, and formally dropped at the 5/27/2010 restructuring. See ICR for Meeting Teachers' Needs project for data achievement) for this indicator. Indicator 4: Number of accredited non-public schools participating in per student subsidies. Value quantitative or 0 500 1,100 1,212 Qualitative) Date achieved 03/21/2007 01/31/2011 05/31/2012 05/31/2012 Comments Target met and exceeded. (incl. % The indicator was changed in the 5/27/2010 restructuring from the PAD version achievement) (p.4 of restructuring paper). Share of beneficiary schools submitting use of funds reports to DEAPP and/or Indicator 5 : P ~NEPO Value (quantitative 0 80% Not revised 76% or Qualitative) Date achieved 03/21/2007 01/31/2011 05/31/2012 Comments Target largely, but not fully met. Indicator changed from, "Percentage of School (incl. % Management Committees (SMCs) which submit use of funds reports to DEAPP and/or NEPO" at 5/27/2010 restructuring because of capacity constraints in achievement) SMCs. Indicator 6 : Children participating in integrated nutrition/health program (person/years) Value quantitative or 0 70,000 210,000 260,000 Qualitative) Date achieved 03/21/2007 01/31/2011 05/31/2012 5/31/2012 General target met and exceeded after upward revision. Similar to above tuition Comments subsidies indicator, target reflects impact of additional donor funds catalyzed by he Project. Project attributable person/years was 150,000, but there is no Project (incl %i tJ t i (l. specific target against which to assess that achievement. Indicator was changed achievement) at 5/27/2010 restructuring from "Students benefiting from program-financed health and nutrition programs." Number of schools where Grade 2 literacy competency test is applied (to establish baseline) Value quantitative or 0 200 86 84 viii Qualitative) Date achieved 03/21/2007 01/31/2011 05/31/2012 5/31/2012 Comments Revised target largely, but not fully met. (incl. % Indicator was modified at 5/27/2010 restructuring (p. 4 of restructuring paper). In addition, target was reduced to 86 because of higher than anticipated costs and achievemeInt) capacity constraints. (b) Intermediate Outcome Indicator(s) Original Target Actual Value FormallyromAchieved at Indicator Baseline Value Values (from Revised cmplet approval Completion or documents) Target Years Indicaorl Multi-Age Learning Program implemented in at least 100 schools Value (quantitative 0 100 N/A N/A or Qualitative) Date achieved 03/21/2007 01/31/2011 Comments (Cmmd %The sub-component to which this indicator pertains was dropped because of ac. inadequatecapacity in the implementing agency (p. 6 Restructuring paper). achievement) Percentage difference between the unit cost of the NSFP's school feeding Indicator 2: program and that of the average unit cost of 3 largest NGO school feeding roviders Value (quantitative 315% 20% N/A / or Qualitative)KA Date achieved 03/21/2007 01/31/2011 (Cmm.t% This is an efficiency measure that was dropped during the 2010 restructuring a(icivmet %because improving efficiency was not an objective of the Project. G. Ratings of Project Performance in ISRs Date ISR Actual No. DO IP Disbursements (USD millions) 1 12/20/2007 Moderately Satisfactory Moderately Satisfactory 0.20 2 04/11/2008 Satisfactory Satisfactory 2.32 3 11/18/2008 Satisfactory Moderately Satisfactory 5.43 1x 4 11/25/2008 Satisfactory Moderately Satisfactory 5.43 5 04/21/2009 Satisfactory Moderately Satisfactory 7.44 6 07/11/2009 Satisfactory Moderately Satisfactory 8.87 7 07/31/2009 Satisfactory Moderately Satisfactory 8.87 8 08/30/2009 Satisfactory Moderately Satisfactory 8.87 9 09/13/2009 Satisfactory Moderately Satisfactory 9.24 10 10/31/2009 Satisfactory Moderately Satisfactory 7.13 11 05/07/2010 Moderately Satisfactory Moderately Satisfactory 21.29 12 06/21/2010 Satisfactory Moderately Satisfactory 21.29 13 02/09/2011 Satisfactory Moderately Satisfactory 33.19 14 08/08/2011 Satisfactory Moderately Satisfactory 34.99 15 02/21/2012 Satisfactory Moderately Satisfactory 37.85 16 05/28/2012 Moderately Satisfactory Moderately Satisfactory 37.85 H. Restructuring (if any) ISR Ratings at Amount Restructuring Board Restructuring Disbursed at Reason for Restructuring & Date(s) Approved Restructuring Key Changes Made PDO Change DO IP in USD millions 1. Additional financing of SDR8 million (US$12 million) to increase beneficiaries of school feeding and tuition subsidy programs and additional institutional support to MENFP. 2. Adjust indicator targets to reflect changes in priorities, financing and Project 05/27/2010 No MS MS 21.29 environment. 3. Reallocate funds to priority activities including tuition subsidies and school feeding. 4. Physical inspections replace financial reporting to verify tuition subsidy program. 5. Extension of project closing date by 16 months from January 31, 2011 to May 31, 2012. Reallocation of funds to adjust 05/07/2012 S MS 37.85 Grant expenditure categories in preparation for Project closing. x ISR Ratings at Amount Restructurit Board Restructuring Disbursed at Reason for Restructuring & IOBRs) Approved Restructuring PDO Change DO IP in [SD millions Categories I and 2 were slightly overdrawn because of higher than expected participation in tuition subsidy program Categories 3 (school health and ndtrition) and 4 (reinforcement of MENFP's technical capacity) were lower than expected. The latter because expenditures on post-earthquake capacity building were lower than initial estimates. . Disbursement Profile Original Formally Revised -- Actual 40- 30- . 20- 0- 11111a1 ** d dd d- d d * XI I. Project Context, Development Objectives and Design 1.1. Context at Appraisal 1. At the time of appraisal, 78 percent of Haitians were living on less than US$2 a day, and more than half living on less than US$1 a day,1 making Haiti the poorest country in the Latin America and Caribbean (LAC) Region. Living conditions were particularly difficult in rural areas, where poverty and extreme poverty rates were estimated to be even higher. With an infant mortality rate of 76 per 1,000 births, illiteracy of 47 percent, and the highest incidence of HIV/AIDS outside Sub-Saharan Africa, Haiti ranked 15 3 rd of 177 countries worldwide in the United Nations 2005 Human Development Index. 2. The country was emerging from several years of political, economic and social strife. From 1996 until the time of appraisal, the country has suffered from weak or negative GDP growth, accompanied by mounting political and social strife exacerbated by the departure of Jean-Bertrand Aristide in February of 2004. Governance of the sector was also a disadvantage. There was a lack of procedures and transparency in the use of public funds, and poor coordination across technical departments and of externally financed investment projects. Many trained MENFP civil servants left the system for opportunities abroad or as externally-financed consultants. The MENFP's basic capacities to conceptualize, plan, execute, monitor and evaluate educational programs in a coordinated manner were weak. Reliable up-to-date education and demographic statistics for each region and nationwide, essential for planning purposes, were not available to MENFP staff. 3. Until the preparation of the Education for All (EFA) Adaptable Program Grant- Phase 1 (APG-1), 2 the Bank had not been engaged in the Haiti education sector for eight years. In 2006, after many years of political turmoil, Haiti held democratic elections and launched an I-PRSP that focused on expanded access to education and health services. The I-PRSP informed the strategic pillars of the Bank's Interim Strategy Note. In the four years leading up to appraisal, Haiti prepared a National Strategy for achieving Education For All. This strategy was developed in partnership with the United Nations Education, Scientific and Cultural Organization (UNESCO), the Bank, the United Nations Children's Fund (UNICEF) and other donors. This ten-year plan placed Haiti on a path toward achieving Education For All by 2015. 4. The available data for the education sector revealed weak outcomes. The primary net enrollment rate was 71 percent in 2006, with a gross enrollment rate of 127 percent and a primary level completion rate of 66 percent. An estimated 500,000 children aged 6-11 did not attend school of any kind, and only about half of all six year olds enrolled in first grade. 3 More than 80 percent of all primary level students attended non-public schools, financed mainly by parents and religious associations, NGOs and other sources, while the remainder attended virtually free public schools. Non-public schools had become increasingly distant from the GOH and were not as concerned with accreditation 1 IMF Article IV Consultations, August 10, 2010. 2 EFA and APG-1 are used interchangeably and refer to the same project. 3 DHS Survey 2005. 1 standards. The Living Conditions Measurement Survey revealed that 43 percent of all parents (50 percent for parents in the lowest income quintile) did not send their children to school because of the costs of school tuition, books, uniforms and transportation for the private-based education system. 5. Quality of education. Curricula were outdated and not relevant for over-age students who made up the bulk of the primary education population. The APG-1 project appraisal document (PAD) described teaching practices as "chalk and talk", requiring students to recite words and phrases they frequently did not understand. In addition, about 75 percent of all teachers lacked adequate training with many having just a 9th grade education, with no formal teacher training. 6. Affecting both quality and access was the state of children's nutrition as approximately 40 percent of Haitian children suffered from a caloric deficit of an average of 460 kilo-calories per day. 4 Many schoolchildren were commuting long distances from home to school, often without eating any breakfast. At the same time, school feeding programs in Haiti were declining in coverage, due to cutbacks from key international donors, from 800,000 children in 2002 to about 400,000 in 2006. 7. Rationale for Bank involvement. The EFA grant was consistent with the Bank's role at both sectoral and macro-economic levels to assist the Government in implementing its national strategy for Education For All. The Bank's policy dialogue and ongoing support for education governance reform under the Low Income Country Under Stress Trust Fund (LICUS TF) contributed to preparation of the Project's activities toward improved public-private partnerships (PPP) in the education sector. The Bank also used its comparative advantage in development of demand-side education sector financing mechanisms. Through the APG-1, the Bank intended to generate additional funds from other donors, which included US$10 million of parallel financing from the Caribbean Development Bank (CDB). Lastly, the reputational risk for the Bank of not re-engaging in the education sector after eight years was also a rationale for Bank involvement. 8. Complementarity with other projects. The APG-1 complemented other World Bank investments, such as the Rural Water Supply and Sanitation Project, Roads and Territorial Development Project, and the Community-Driven Development Project, which aimed to provide tangible improvements in living conditions. APG-1 also helped the Bank maximize synergy and leverage of its other operations in Haiti to support education. The second Economic Governance Reform Operation (EGRO II) and the Heavily Indebted Poor Countries (HIPC) Initiative included important education policy and programmatic reforms. The Meeting Teachers' Needs project (effective 9/26/2008) and the Emergency School Reconstruction project (effective 6/25/2009) later complemented APG-1, bringing supply-side interventions of more teachers and schools to the demand-side interventions of tuition subsidies and school nutrition programs. Thus, the Bank supported multi-pronged approach to improve access to primary education. 4 Height for age is a strong predictor of school enrollment and is positively associated with poor cognitive function, school attendance, and school performance. 2 1.2. Original Project Development Objectives (PDO) and Key Indicators 9. Program Level PDO: To improve access to primary education for poor children aged 6-12, while improving equity, quality and governance of the education sector. 10. Project Level PDO: To (a) improve access and equity of primary education; (b) operationalize partnerships between public and non-public sectors; and (c) build capacity to assess learning outcomes. 11. The Development Credit Agreement inappropriately quoted the program objective rather than the Project objective. The ICR uses the PDO in the PAD for the purpose of assessment. The key indicators, along with their revisions, are presented in Table 1. 1.3. Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 12. Several of the PDO indicators were revised as shown in Table 1. The Legal Agreement was amended to correct the error in the wording of the PDO. The impact of additional funds from both the Bank and other donors are reflected in the scaled up targets for the tuition subsidy and school feeding programs. The Grade 2 literacy baseline was to be established in fewer schools (86 rather than 200), a target that was reduced because of capacity constraints in schools. The two dropped indicators are explained in the table. TABLE 1: PDO Indicators and Revisions Original from PAD Revised w/ Additional Financing-5/27/10 135,000 children benefit from primary Children receiving tuition subsidies (person education enrollment subsidies. yrs.)--390,000 Grade 1 Capacity Utilization in participating Dropped-capacity utilization is not an issue schools increases to 100% (access). post-earthquake At least 3,600 new teachers receive pre-service Dropped-teacher training is delivered through teacher education. the Meeting Teachers Needs project-P106621 At least 500 accredited non-public schools Number of accredited non-public schools receive per student subsidies allocated by multi- participating in per student subsidies-- 1,100 stakeholder committees. (public/private partnership) At least 80 percent of beneficiary schools submit Share of beneficiary schools submitting use of use of funds reports to DAEPP and/or NEPO. funds reports to DEAPP and/or NEPO-80% (public/private partnership) At least 25,000 schoolchildren participate in Children participating in integrated integrated nutrition/health program. (quality) nutrition/health program (person yrs.) 210,000 Application of Grade 2 literacy competency test Number of schools where Grade 2 literacy in at least 200 schools (capacity to assess competency test is applied (to establish learning outcomes), to establish baseline. baseline)-86 Note: Original indicators from PAD Annex 3, pp. 45-6. 1.4. Main Beneficiaries 13. The main beneficiaries of the Project included: * Students receiving a tuition waiver: Original-135,000 (number of students by year 3, or 210,000 student-years), revised-390,000 (student-years). 3 * Students receiving free school food: Original-70,000 (students-years), revised- 210,000 (student-years). * Teachers receiving training (2,500) and improved curriculum (5,000-10,000). * Agencies that received institutional capacity building including the Ministere de 1'Education Nationale et de la Formation Professionnelle (National Ministry of Education and Vocational Training-MENFP), the MENFP's Department for Private Education and Partnership (DAEPP), Regional Education Departments (DDEs) and Programme National de Cantines Scolaires (PNCS). * School Management Committees (SMCs) at schools receiving tuition waivers benefiting from financial management and reporting training: Original-500, revised- 1,100. 1.5. Original Components 14. APG-1 was the first phase of a planned three-phase program consisting of the following components: Component 1: Improving Access to Primary Education * The provision of tuition waivers for primary school students in private schools. The tuition waivers paid by the Project cover various school costs particularly the cost of teacher salaries, as well as school materials. * Training programs for pre-service teachers. * Strengthening SMCs' capacity for, inter alia: (a) the identification and preparation of tuition waiver subprojects; (b) financial management; (c) supervision of selected school-level projects, including quality improvement projects and school feeding programs; and (d) school management under increased transparency and accountability procedures. Component 2: Improving the Quality of Primary Education * The development of multi-grade learning programs to support learning in rural areas, including all training and all monitoring and evaluation procedures. * The improvement of schoolchildren's performance through: (a) the provision of support for the steady and efficient delivery of school feeding services, including a package of micronutrients supplementation and hygiene education that promotes balanced nutrition, clean water and high sanitary standards, both at school and at home; and (b) the provision of School Feeding Grants to Beneficiary CBOs, including SMCs and NGOs, to finance community-driven School Feeding Subprojects in the poorest areas of the Recipient's territory. Note: This subcomponent is also referred to as the school canteen program and the school health and nutrition program. * The improvement of literacy skills development in the first two grades for public and non-public schools and the improvement in the competencies in primary education curriculum for grade 1 through 6 through, inter alia: (a) the provision of textbooks in local Creole language, and all associated training in relation thereto, including the training of trainers; (b) the strengthening of MENFP's capacity to monitor the acquisition of literacy skills and related quality of teaching and learning at the primary level; and the training of teachers in the utilization of textbooks to be provided under the Project. 4 Component 3: Improving Institutional Governance of MENFP and the Education Sector * The strengthening of the institutional capacity of MENFP to manage public funds and complete public procurement in a transparent and fully accountable manner, through the modernization of: (a) its department of administrative affairs and department of human resources; and (b) the electronic communication between MENFP's central and departmental levels. * The provision of support for the establishment, institutional capacity building and operation of the National Education Partnership Office (NEPO) as a public entity responsible for channeling public funds to non-public SMCs following transparent, rigorous, equitable and monitorable eligibility criteria. * The provision of support for the strengthening of the capacity of MENFP to; (a) increase coordination and effectiveness between its technical departments at central and regional levels; (b) develop and maintain a GIS-based school mapping database; and (c) carry out evaluation and accrediting activities for non-public schools through its department for private education and partnership. Part 4: Project Coordination and Evaluation * The Provision of support for the establishment and operation of the Project Coordination Unit (PCU) within MENFP to be responsible for Project coordination. * The carrying out of studies to evaluate Project impact, including the qualitative and quantitative impact of: (a) the implementation of Student Enrollment Subprojects; and (b) the delivery of school feeding programs and their contribution towards enhancing the quality of education. 15. The components were designed to provide quick benefits to primary schools students, as well longer term benefits in the form of improved governance and implementation capacity at central and local levels, and an increased supply of trained teachers. 1.6. Revised Components 16. There were several component revisions made in the May 27, 2010 Restructuring/Additional Financing, which was a Level 2 restructuring and therefore did not require Board approval. The pre-service teacher training (component 1) evolved into a new self-standing project, as other IDA funds became available. Targets associated with teacher training (e.g., 2,500 teacher graduates per year) were assigned to the new project (see ICR for Meeting Teacher Needs Project, Report number ICR00002426). The multi-grade learning program (component 2) was never launched as there was little government ownership for the initiative. The pilot school feeding program (component 2) to be run by SMCs was not implemented because of capacity constraints. 1.7. Other significant changes 17. The other main changes that occurred in the May 2010 Project restructuring were: (i) additional financing of US$12 million to scale up the school feeding and tuition subsidy programs; (ii) Reallocation of funds from the pilot school feeding program and multi-grade learning subcomponents to the tuition waiver and school canteen 5 components; (iii) scaled up targets for tuition subsidies and school nutrition; (iv) physical inspections replaced financial reporting to verify tuition subsidy program; and (v) extension of Project closing date by 16 months from January 31, 2011 to May 31, 2012. The second restructuring in May 2012 was a minor reallocation of funds across components to accommodate higher expenditures for tuition subsidies and slightly lower expenditures for health and nutrition and capacity building at MENFP post-earthquake. II. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 18. The APG- 1 was a high risk operation in a high risk country, but with potentially high returns. The challenges to implementation of APG-1 were high not only because Haiti is a fragile state and because the Bank had not been engaged in Haiti for eight years, but also because implementation depended on a careful sequence of steps. The tuition subsidy depended on a number of important factors: (i) the proper functioning of the public financing mechanism that applied school eligibility rules; (ii) proposal evaluation/ranking criteria; (iii) transparent, multi-stakeholder mechanisms to choose participating schools; (iv) use of the commercial banking system to make transfers to school management committees; (v) extensive public information campaigns; and (vi) independent external monitoring of use of financial transfers. The Project was set up to deal with these challenges through the development of the Subsidy Program Operational Manual, which is part of the APG-l's overall operational manual. The subsidy program operational manual included funding criteria, review procedures, and measures to ensure both transparency and accountability in subsidy allocations. 19. The Project was designed with a PCU embedded within MENFP, with a technical unit consisting of a program coordinator and four sub-coordinators, each one responsible for a specific Project component and (ii) an administrative unit with staff responsible for the Project's financial and procurement matters, all established within MENFP's existing structures. This type of PCU was risky because of the low capacity and high turnover at MENFP. 20. The Project's activities depended on the operationalization of public-private partnerships (PPPs), given that 80 percent of schools were non-public. This strategy aimed to create a space for dialogue between public and non public sectors, and to make non public schools accountable to government. Operationally, it meant that the Project would use public funds to support private entities, such as private schools and NGOs, in the provision of educational and nutrition services. The Project was prepared with the expectation that a National Education Partnership Office (NEPO) would be created that would oversee public private partnerships (PPP) in education including the tuition subsidies, as well as the school nutrition program. 21. The APG-1 was designed in close partnership with the CDB. The Bank-financed and CDB-financed projects used the same project management team and project implementation procedures, including harmonized fiduciary arrangements, project reporting procedures, and audits. The Bank handled procurement non objections for the whole program. The Project was also set up in a way that the PCU could manage expanded activities for tuition subsidies with funds from other donors such as IDB. 6 22. The APG-l was implemented in partnership with the Education For All Fast Track Initiative, which, by design, involved all major education donors. This included a single set of monitoring and evaluation indicators by which to measure the Government's inputs and outputs in achieving the national EFA strategy, and a common reporting framework for tracking all education donor activities and financing. This design resulted in one of the Project's major contributions, which was to offer a mechanism to channel a larger amount of funds to Haiti from donors. 23. To ensure relevance of Project activities, a Social Assessment was carried out in September/October 2006, including interviews and focus groups with key stakeholders in Port-au-Prince as well as over 400 stakeholders in four departments: Artibonite, Centre, Ouest, and Nippes. Widespread support and demand for the proposed interventions existed among all consulted stakeholders. Potential benefits raised by local stakeholders of each of the three proposed interventions include, among others, increased enrollment, reduced financial burden on households, improved student performance, reduced dropouts, more stable school budgets, and improved teaching quality. 24. The tuition subsidy component, by far the largest component, was designed to be implemented as a two-tranche disbursement. Tranche 1 for 60 percent of the tuition subsidy amount for each school was released to schools based on a preliminary list of qualifying students provided by each school. The second tranche for the remaining 40 percent was to be disbursed on the basis of submission of a satisfactory financial management (FM) report from the SMC, which had been trained in financial reporting. This was a logical approach that allowed schools access to critical funding to begin the school year, but then to be accountable later in the school year for the number of students it had claimed for the tuition waiver. Actual payments to schools were made by the Department of Administrative Affairs (DAA) to school bank accounts. Withdrawals from accounts required the signature of both the school director and the SMC representative. 25. The accelerated teacher pre-service professional development program also was challenging. It required a new curriculum, new teaching and learning materials, re- training of professors employed at teacher training institutes and a change in organizational culture, all of which had to be done in sequence. It was designed based on Bank experience (i.e. teacher training programs in Burkina Faso, Guinea, and Senegal) that accelerated pre-service teacher training programs are effective when coupled with strong student-teaching programs so as to ensure that student-teachers are applying their training knowledge to teaching practices. The process was endorsed by key officials in the MENFP as well as by the teachers' unions, non public education providers and other education donors. UNESCO also helped with technical assistance for this component. 26. Key lessons learned from previous operations in Haiti were applied in the Project design, including: (i) there was untapped potential of public-private partnerships to improve access to social services; (ii) the GOH had capacity, in the case of the school feeding program, to play a normative, financial and supervisory role, while leaving actual service provision to the non-public sector; and (iii) the need to move beyond supply-side interventions to address critical demand-side constraints and immediate poverty reduction objectives. Finally, the Post-Conflict Fund (PCF) and LICUS TF grants were implemented in collaboration with other multilateral and bilateral agencies, pointing to the importance of effective donor coordination for program success and on-the-ground 7 results. The application of these lessons is consistent with the recommendations from the 2002 Haiti Country Assistance Evaluation (CAE), which argued "social services should be channeled through the private and voluntary sector where possible [private schools in the tuition waiver program and NGOs in the school nutrition program], but with safeguards to promote adequate quality and access by the poor." 5 2.2 Implementation 27. A devastating earthquake occurred on January 10, 2010 that left more than 200,000 people, 5 percent of the population, dead which included staff in the PCU, the MENFP and an unknown number of Project beneficiaries. There was widespread destruction of infrastructure, schools and government buildings, including the MENFP. Some schools financed by the Project were heavily damaged or destroyed and temporarily ceased to operate. From January to June 2010 very little could be done to move the Project forward. Implementation from January 2010 onward should be assessed in the context of this national calamity. 28. The original, pre-earthquake PCU in the MENFP created a bottleneck for Project implementation, mainly because of lack of ownership at MENFP caused by multiple changes of the Minister of Education. The earthquake compounded these problems due to the destruction of the Ministry of Education, and led to the decision to establishing a more autonomous PCU unit. After the earthquake, the PCU staff operated for several weeks from the yard of its rented property and with limited resources, as the building had suffered damage making it structurally unsound. The PCU was subsequently moved to a safer location, which was key to resuming the pre-earthquake pace of Project implementation; this new office also served as a workspace and provided key functions for some MENFP officials who could no longer find office space in the temporary post- earthquake office structures. PCU staff members lost during the earthquake were also replaced, although finding qualified staff in a context in which human resources were further constrained took several months. While the independent PCU kept activities moving forward, efforts to build ownership and capacity in the MENFP waned. However, there was not a viable alternative to reliance on the external PCU in the short- term post-earthquake context. 29. NEPO. The NEPO, which had been planned as a key institution to oversee PPPs, was never operationalized. The reasons were capacity constraints and a lack of political commitment. Although Parliament adopted the law authorizing the establishment of NEPO (Loi Creant et Organisant l'Office National de Partenariat en Education - September 10, 2007), there was infighting over who would have the power to decide which schools should receive tuition subsidies, which proved crippling to efforts to establish the NEPO. Therefore, the per student subsidy program continued to be implemented through the Department for Private Education and Partnerships (DAEPP), in accordance with the Project's contingency plan. The program was implemented under the guidance of a nine-member Steering Committee, which had a similar membership to what would have constituted NEPO (had it been established), but which did not have as 5 Haiti Country Assistance Evaluation, page 21, 2002. 8 much managerial capability. Prior to the 2010 restructuring, the DAAs, DDEs and the PCU had difficulties compensating for the absence of the NEPO. The transmission of use of funds reports from schools to the PCU was time intensive, and the PCU was not able to process use of funds reports adequately, delaying the disbursements of the tuition subsidies, which were at the heart of the Project. 30. Tuition subsidy payments. The tuition subsidy program was valuable post- earthquake because it had pre-established a mechanism to mitigate the impact of the earthquake on families. However, in practice, the original tuition subsidy disbursement process did not function well and needed to be modified during Project implementation: (i) despite Project-financed training sessions, SMCs were still incapable of producing financial reports in a timely manner and needed substantially more time and resources to build the necessary capacity to do so; (ii) there were serious delays in disbursing the tuition subsidies, even for the first tranche, with many schools reporting that they received the grants when the school year was more than half over. The delays in the tuition payments caused hardships for the schools because teachers were paid months late, textbooks could only be purchased on credit and school rehabilitation often had to be stopped for months. A number of schools reported to the ICR team that they never received a second tranche. The explanation given later by the PCU was that some schools had overstated the number of students who were eligible for the tuition subsidy, which led to an overestimate of the amount of the first tranche. In the 2011/12 school year, approximately 120 of the 1,100 participating schools had overstated the number of students. 31. The 2010 restructuring eased the disbursement problem by simplifying the disbursement mechanism whereby the release of the second tuition subsidy tranche was based on a technical audit by independent firms rather than use of funds reports from SMCs. The technical audit approach facilitated the tuition waiver program by compensating for the lack of reports from the SMCs through verification of, inter alia, the number of tuition-subsidy-eligible students in class. This adjustment improved the transfer of tuition subsidy tranches, but they were still delayed until often well after the school year had ended. 32. Early Grade Reading Assessment (EGRA) and Capacity Building. The EGRA provided a basis to assess primary student learning outcomes and was the first assessment of its kind in Haiti. Although the number of schools participating was reduced (from 200 to 86), the number was adequate to provide a meaningful baseline for assessment. The MENFP took advantage of the opportunity to be closely involved in the EGRA exercise and developed capacity to carry out further assessments. Additional capacity was built at the MENFP through the provision of office equipment and training. However, most of the improvements to physical capacity were destroyed in the earthquake. In addition, after the earthquake, there was much turmover in the MENFP with staff leaving the ministry to work outside the GOH, particularly to work in the hundreds of NGOs that were providing humanitarian relief to Haiti. 33. Project risks and mitigation. A significant number of Project risks that were foreseen at appraisal were mitigated during implementation. For example, the Project design was correct in providing a fallback if NEPO was not established even though it had been endorsed by teachers' unions, parent associations and non-public education service providers. The mitigation plan was to apply objective criteria for school subsidy 9 allocation, along with enhanced transparency at ministerial and school level in how schools were selected for the program. Targets for tuition subsidy were met and exceeded even if there were significant disbursement delays. There was the risk that public education spending increases would be insufficient to fund the student subsidy program that was initially financed under Project. 34. One of the Project's major accomplishments was the additional US$60 million in resources that the PCU reported it channeled for the expansion of the tuition waiver and school nutrition programs. The Project, through the PCU, was able to facilitate large amounts of additional resources to the poor during a time of urgent need (the post- earthquake environment). The Project PCU provided a mechanism that allowed alignment of donor procedures, harmonization of objectives and indicators, and joint supervision and assessment. These additional funds also mitigated the risk that the GOH would not establish a medium-term expenditure framework for education, which it did not. The additional funds from CIDA, CDB, IDB, as well as the EFA Fast Track Initiative and APG-2 funded an expanded government program of tuition subsidies covering nearly one million additional tuition waivers. 35. The mitigation of the risk