Rapport sur l'achèvement de la mise en œuvre et les résultats : Renforcement de la gouvernance dans les secteurs de l'éducation, de l'eau et de l'assainissement

Document of The World Bank Report No: ICR00003750 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-17656) FOR A DEVELOPMENT POLICY GRANT FROM THE HAITI RECONSTRUCTION FUND IN THE AMOUNT OF US$17.33 MILLION TO THE REPUBLIC OF HAITI FOR STRENGTHENING GOVERNANCE IN EDUCATION, WATER AND SA NITATION December 15, 2016 Macroeconomics and Fiscal Management Global Practice Haiti Country Management Unit Latin America and Caribbean Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized HAITI GOVERNEMNT FISCAL YEAR October 1 – September 30 CURRENCY EQUIVALENTS (Exchange Rate Effective as of November 30, 2016) Currency Unit = Haitian Gourde (HT) US$ 1.00 = HTG 66.76 WEIGHTS AND MEASURES Metric System ABBREVIATIONS AND ACRONYMS CAEPA Water supply and sanitation committees CNIGS National Geospatial Information Center DDE Departmental Directorate of Education DINEPA National Directorate of Potable Water and S anitation DPG Development Policy Grant DPO Development Policy Operation EU European Union FDI Foreign Direct Investment GoH Government of Haiti GDP Gross Domestic Product HRF Haiti Reconstruction Fund IADB Inter-American Development Bank IDA International Development Association MEF Ministry of Economy and Finance MENFP NGOs Ministry of Education and Vocational Training Non-Governmental Organizations ONAPE National Education Partnership Office OP Professional Operator OREPA Regional Offices of Water and Sanitation PARDH National Recovery and Development Action Pla n for Haiti PDO Project Development Objective PEFA Public Expenditure and Financial Accountabili ty PFM Public Financial Management PPE Provisional Teaching Permit PSUGO Universal, Free and Mandatory Schooling Program SIGE Information System for Education Management SISKLOR Water Chlorination and Quality Surveillance System TEPAC Community-Level Water and Sanitation Technic ian UN United Nations URD Rural Departmental Unit USI Information System Unit US$ United States Dollar Vice President: Jorge Familiar Senior Director Global Practice: Carlos Felipe Jaramillo Special Envoy: Mary Barton-Dock Practice Manager: Miria A. Pigato Program Leader: Raju Singh Task Team Leader: Evans Jadotte ICR Team Leader: Nestor Ntungwanayo HAITI DEVELOPMENT POLICY GRANT FOR STRENGTHENING GOVERNANCE IN EDUCATION, WATER AND SA NITATION CONTENTS Data Sheet A. Basic Information ............................................................................................................ i B. Key Dates ........................................................................................................................ i C. Ratings Summary ............................................................................................................ i D. Sector and Theme Codes ................................................................................................ ii E. Bank Staff ....................................................................................................................... ii F. Results Framework Analysis .......................................................................................... ii G. Ratings of Program Performance in ISRs ...................................................................... v H. Restructuring ................................................................................................................. v 1. Project Context, Development Objectives and Design ................................................... 1 2. Key Factors Affecting Implementation and Outcomes ................................................... 7 3. Assessment of Outcomes .............................................................................................. 14 4. Assessment of Risk to Development Outcome ............................................................. 19 5. Assessment of Bank and Borrower Performance ......................................................... 20 6. Lessons Learned ............................................................................................................ 24 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ............... 25 Annex 1. Bank Lending and Implementation Support/Supervision Processes ................. 26 Annex 2: Performance on Development Policy Outcomes .............................................. 27 Annex 3: Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 30 Annex 4. Beneficiary Survey in the Ministry of Economy and Finance, in the Ministry of Education and Technical Training, and in the National Directorate of Potable Water and Sanitation .......................................................................................... 42 Annex 5. List of Supporting Documents .......................................................................... 47 i A. Basic Information Country: Haiti Program Name: HRF Grant for Strengthening Governance in the Education and Water and Sanitation Sectors Program ID: P147166 L/C/TF Number(s): TF-17656 ICR Date: 12/05/2016 ICR Type: Core ICR Lending Instrument: DP Grant Borrower: Government o f Haiti Original Total Commitment: USD 17.33M Disbursed Amount: US$17.33M Revised Amount: USD 17.33M Implementing Agencies: Ministry of Finance and Economy Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 02/22/2013 Effectiveness: 09/15/201 4 09/15/2014 Appraisal: 06/25/2014 Restructuring(s): Approval: 06/30/2014 Mid-term Review: Closing: 12/31/2015 12/31/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcome: Moderately Unsatisfactory Risk to Development Outcome: High Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Unsatisfactory Quality of Supervision: Moderately Unsatisfactory Implementing Agency/Agencies: Moderately Unsatisfactory Overall Bank Performance: Moderately Unsatisfactory Overall Borrower Performance: Moderately Unsatisfactory ii C.3 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments Rating: Potential Problem Program at any time (Yes/No): No Quality at Entry (QEA): None Problem Program at any time (Yes/No): No Quality of Supervision (QSA): None DO rating before Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) General education sector 50 50 General public administration sector 17 17 Sector Code (as % of total World Bank financing) Sanitation 16 16 Water supply 17 17 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Jorge Famil iar Calderon Global Practice Senior Director : Carlos Felipe Jaramillo Sector Director : Paloma Anos-Casero J. Humberto L opez Special Envoy: Mary Barton-Dock Mary Barton-Dock Practice Manager/Manager: Miria A. Pigato Auguste T. Kouame Program Team Leader: Raju Singh Raju Singh Task Team Leader: Evans Jadotte Elizabeth Ruppert Bulmer/ Evans Jadotte ICR Team Leader: Nestor Ntungwanayo F. Results Framework Analysis Program Development Objectives The Grant’s Program Development Objective (PDO) is “to strengthen transparency and institutions for budget management in education, water and sanitation”. In particular, the reform program supported by the operation aimed to achieve the following specific objectives: (i) improve cash management by the treasury, enabling a better execution of iii sectoral budgets, (ii) improve the Ministry of Education and Vocational Training (MENFP) regulation of non-public education services, (iii) increase transparency through timely and more widely accessible information on education service delivery, (iv) improve the education budget alignment with sector priorities, (v) improve oversight of the water and sanitation sector, and (vi) increase monitoring and transparency of water service availability and quality at the community level. Revised Program Development Objectives: Not applicable (a) PDO Indicator(s) Table 1: Achievement of PDO Indicators Indicator Baseline Value Original Target Values (from approval documents) Actual Value Achieved at Completion Year (December 2015) Policy Objective 1: Improve cash management by the Treasury Department, enabling better execution of sectoral budgets. Indicator 1: The Treasury Department at the Ministry of Economy and Finance has little knowledge of how line ministries’ expenditure requests will be spread throughout the budget year. Line ministries measure payment arrears on a monthly basis. Line ministries did not measure and send payment arrears on a monthly basis to the Treasury Department on time. Monthly budget-execution reports sent to the Treasury Department, however, showed lower amounts than the effective budget commitments made by line ministries. Partially achieved Policy Objective 2: Improve MENFP’s regulation of non-public education services. Indicator 2: 10,000 non-public schools remain unrecognized and non- accredited, and the MENFP and Departmental Directorates of Education (DDEs) have a backlog of 4,000 accreditation requests and lack the staffing to carry out accreditations; the Accreditation Commission does not exist. The current backlog of school accreditation requests is eliminated; accreditations are carried out at the regional DDE level; MENFP consults the public-private accreditation group. While the targets were missed, due to a significant change in Government policy on accreditation process, the following achievements were made: (i) the public-private accreditation group (ONAPE) was revived in order to accelerate the accreditation process, (ii) the identification of 17,191 schools to which a School Identity Card will be delivered once a set of agreed-upon criteria have been fulfilled, and (iii) the delivery of a portion of the School Identity Cards at end-2015. Partially achieved iv Policy Objective 3: Increase transparency by provi ding timely and widely accessible information on education service delivery. Indicator 3: The school census is published irregularly and with delays of up to 8 years; the most recent census was published after a 2- year delay. Annual information on education services, including regional and gender- disaggregated data, is made available online and in flyers at regional DDE offices within 45 days of the end of the school year. While the program targets were missed, important achievements were made toward meeting the target, including: (i) the creation of a database that can generate annual reports on the education sector, (ii) the digital mapping of 78.5% of all schools, (iii) the completion of computerized data treatment and Statistical Yearbook tables for 2012, and, (iv) the publication of annual reports on developments in the education sector, including recent data, on the Ministry of Education’s website. Partially achieved Policy Objective 4: Improve the education budget’s alignment with sector priorities. Indicator 4: 46% of MENFP’s FY14 operating budget is allocated to primary education cycles 1 and 2, and non-salary expenditures account for 30.6%. 52% of the FY15 MENFP operating budget and 55% of the FY16 MENFP operating budget are allocated to primary education cycles 1 and 2; non-salary budget allocations in FY15 and FY16 represent at least 32% and 35%, respectively, of MENFP’s operating budget. At the end of the second quarter of FY15, budget allocations to the first two primary education cycles reached 50% and 52 % of the Ministry of Education’s operating budget for FY15 and FY16, respectively; non-salary budget allocations reached 22% and 26% of the Ministry’s operating budget for FY15 and FY16, respectively. Updated data on the above allocations were unavailable. Budget prioritization and comprehensiveness could not be achieved in FY15 and FY16 because the Ministry was under reorganization, including the directorate overseeing primary education. Moreover, new outlays contributed to further increases in personnel expenditures at the expense of the non-salary budget. Not achieved v Policy Objective 5: Improve oversight of the water and sanitation sector. Indicator 5: The National Directorate of Potable Water and Sanitation (DINEPA) lacks a functioning oversight body. The DINEPA Executive Board approves DINEPA’s planned program of activities and annual budget. The names of DINEPA’s Executive Board members were sent to the Senate, but there was no Senate quorum to approve them. However, DINEPA’s operational budgets for FY15 and FY16 were approved. Partially achieved Policy Objective 6: Improve monitoring and transpa rency of water-service availability and quality at the community level. Indicator 6: DINEPA, through the Rural Departmental Units (URDs) and Community-Level Water and Sanitation Technicians (TEPACs), does not collect performance information from its delegated operators, the water supply and sanitation committees (CAEPAs) and water system professional operator (OPs). DINEPA, through the URDs and TEPACs, has collected and published on a monthly basis, beginning in January 2015, performance information on its delegated operators, including at least 230 CAEPAs and OPs by December 2015. Out of 500 CAEPAS and OPs targeted in the program, only 163 have been reporting performance information to DINEPA. Further progress on this objective was hampered by the lack of funding for water-treatment chemicals which interrupted the activities being measured and the costs of communicating data from CAEPAs to DINEPA’s central database, which is managed by the “Observatoire.” As a result, this publication series was interrupted. Not achieved G. Ratings of Program Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 05/06/2015 Moderately Unsatisfactory Moderately Unsatisfactory 17.33 2 01/26/2016 Unsatisfactory Unsatisfactory 17.33 H. Restructuring Not Applicable 1 This Implementation Completion and Results Report (ICRR) assesses the achievements of the expected results of the Strengthening Governance in Education, Water and Sanitation Development Policy Grant (DPG). The DPG intended to support the Government of Haiti to strengthen transparency and build institutional capacity for budget management in the education and water and sanitation sectors. The standalone DPG of US$17.33 million was approved by the World Bank’s Board of Directors on June 30, 2014, and closed on December 31, 2015. 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Reconstruction after the 2010 earthquake was progressing well and had achieved tangible results. A massive earthquake in January 2010 had killed over 230,000, displaced over a million Haitians, devastated critical infrastructure and destroyed significant institutional capacity in the Government. At the time of the appraisal: (i) most earthquake affected areas had been cleared of rubble, (ii) the majority of internally displaced people had been relocated, (iii) many schools had been reconstructed and opened to children who benefited from government’s support, and (iv) cholera incidence had significantly subsided. However, much still remained to be done to achieve sustained growth rates, improve social conditions and reduce poverty levels, and strengthen institutions in order to curb poor governance and corruption. 2. Per capita GDP growth picked up, but reducing poverty required higher growth rates. Growth had been steady at about 4 percent between FY2011 and FY2014, driven by output from construction, commerce and industry. While per capita income growth increased, poverty levels remained high, due to structural shortcomings, including weak governance, and vulnerability to natural disasters. The central government deficit increased to about 6.3 percent of GDP during FY13-14, leading to an upswing in the current account deficit, which was financed by foreign direct investment (FDI) inflows, concessional Petrocaribe debt and depletion of international reserves. External public debt, which had fallen to 9 percent of GDP in FY11 rebounded to 21 percent of GDP in FY2014, almost exclusively due to Venezuela-related Petrocaribe concessional financing. 3. Macroeconomic stability was largely preserved over FY11 through FY14, thanks to the policies pursued by the Government and Central Bank (see Table 2). Inflation had accelerated to 7.4 percent at end-FY11 mainly because of the surge in international food and commodity prices and weak local agricultural production, but moderated over the period 2012-2014 thanks to a larger supply of local agricultural goods. An increase in reserve requirements in 2013 and 2014 had also contributed to the deceleration of credit growth, helping to control inflation. The exchange rate regime, a managed float, was effectively implemented by the Central Bank, which has allowed some depreciation to contain inflationary pressures. The currency has nevertheless remained relatively stable over the period. External imbalances widened from -4.4 percent in FY11 to -8.9 percent of GDP in FY14, but international reserves were maintained at a healthy level above 5 months of import. The fiscal deficit also widened from -3.8 percent to -6.4 2 percent of GDP over the period FY11-FY14, undermined by stagnant revenues and continued drop in grants. Table 2: Macroeconomic Indicators FY09 FY10 FY11 FY12 FY13 FY14 FY15 FY16 Act. Act. Act. Act. Act. Est. Est. Est. Output and Prices Real GDP growth (%) 2.9 -5.4 5.6 2.8 4.3 2.7 1.2 1.5 Consumer price inflation (e.o.p., % change) 3.4 4.1 7.4 6.5 6.8 3.9 7.5 13.4 Government finances (% of GDP) Total revenue and grants 17.8 23.9 21.9 23.4 20.8 18.9 19.3 17.0 Domestic revenue 9.5 10.2 11 11.2 10.4 10.1 11.5 14.3 Grants 8.3 13.7 10.9 12.2 10.4 8.8 7.7 3.1 Total expenditure 22.4 21.7 25.5 28.2 28.0 25.2 21.7 18.4 Current Expenditures 11.7 11.3 11.8 11.9 11.8 12.5 12.6 12.6 Capital expenditure 10.7 10.4 13.7 16.3 16.2 12.7 9.1 5.8 Primary balance -3.8 2.8 -3.2 -4.4 -6.6 -6.2 -2.2 - 1.2 Overall balance -4.6 2.2 -3.6 -4.8 -7.1 -6.3 -2.4 -1.6 Excluding grants -12.9 -11.5 -14.5 -17.0 -17.5 - 15.1 -10.1 -4.7 Financing -4.6 2.2 -3.6 -4.8 -7.1 -6.3 -2.4 -1.6 External net financing 3.1 3.4 4.6 4.7 4.6 3.8 1.6 0.5 o/w PetroCaribe 2.2 3.5 4.4 4.6 4.7 4.2 1.8 0.2 Arrears (net) 0 0 0 0 0 0 0 0 Internal net financing 1.5 -5.6 -1.0 0.1 2.1 2.6 0.8 1.1 HIPC interim relief 0.5 0 0 0 0 0 0 0 External sector Current account balance (% of GDP) -1.9 -1.6 -4.4 -5.4 -6.8 -8.9 -2.3 0.4 Exports of g. (% change) 12.4 2.2 36.3 0.8 18.1 5.0 7.0 -2.5 Imports of g. (% change) -3.6 48.1 10.1 -7.1 8. 1 10.1 -6.6 -3.0 Current Transfers (net, US$ m) 1,635 3,147 2,997 2,600 2,531 2,540 2,685 2,619 o/w private 1,241 1,307 1,551 1,612 1,781 1,977 2,196 2,349 Foreign direct investment inflows (US$ m) 55 178 119 156 162 99 106 108 Gross official reserves (months of imports) 2.8 5.0 5.8 6.4 6.6 5.5 4.8 5.0 Memorandum items Nominal GDP (billions of gourdes) 268.0 267.0 302. 8 328.1 364.5 392.3 425.7 476.0 Nominal GDP (US$ m) 6,585 6,623 7,518 7,890 8,451 8 ,792 8,766 7,972 Total public debt (% of GDP) 23.1 16.5 11.1 15.6 19.4 22.5 27.2 27.7 Remittances (% of GDP) 21.0 22.5 21.0 20.4 21.1 21. 0 22.0 25.0 Source: World Bank staff estimates. 3 4. Security was uncertain, reflecting a contentious political climate. Despite the relatively peaceful election of President Michel Martelly in May 2011, his term has been marked by strong tensions between the Executive and Parliament. The rising cost of living (inflation averaging 5.1 during 2011-15), the slow pace of reconstruction, and allegations of corruption were fueling discontent and sporadic demonstrations that had at times become violent. Although political dialogue had led to an agreement to hold presidential elections in October 2014, the necessary legal and administrative steps to do so were lagging, leading to uncertainty about meeting the set deadlines. Further disruptions to social stability were expected. 5. As Haiti’s focus shifted from post-disaster emergency actions to implementing a longer term development program, the Grant tried to address structural bottlenecks in order to strengthen Haiti’s governance systems and increase transparency. The significant post-earthquake inflow of reconstruction funds not only created pressure on the existing capacity to manage and execute funds, but also increased demand for better economic governance. In this context, the Grant aimed at enhancing public resource use through improved expenditure planning, better oversight of education and water and sanitation service delivery, and increased transparency around the availability and quality of these services. 6. The HRF Development Policy Grant aimed at supporting the Government’s governance reforms program. In particular, the operation supported three areas of the Government’s strategy for reconstruction and development (PARDH), namely (a) territorial rebuilding with a focus on infrastructure, in particular the water and sanitation sector; (b) social rebuilding to create networks of modern education centers; and (c) institutional rebuilding with a focus on government’s essential functions related to public financial management and oversight. The DPG comprised measures intended to increase public spending efficiency and maximize the impact of the high level of donor financing in the transition period. The grant’s focus was geared toward improved governance by ensuring the efficient use and enhanced oversight, transparency and accountability of public funds, particularly in the sectors of education, water and sanitation. It was deemed necessary to focus on the latter sector as well, i.e. water and sanitation, in order to contain or reverse water-borne diseases. 4 7. The operation provided highly needed resources to the GOH. During the period preceding the DPO approval, domestic revenue had increased modestly (12.6 percent of GDP on average during 2012-2014 against 11.2 percent in 2008), remaining insufficient to meet the Government’s financing requirements for reconstruction and growth. The country continued to depend heavily on donor aid. Aid flows were, however, declining substantially from their exceptionally high levels following the earthquake. The Bank provided exceptional support to Haiti with substantial financing from the IDA 16 Crisis Response Window after the 2010 earthquake. Together with the Government of Haiti and its partners, the Bank set up the Haiti Reconstruction Trust Fund (HRF) and became its Trustee. It coordinated internal and external stakeholders in support of a long term reform program, consistent with the Bank’s 2013-14 ISN and Government priorities. 8. Previous budget support operations and reforms to strengthen public resource management had produced only modest results. Back-to-back hurricanes and the 2010 earthquake negatively affected the performance of a programmatic series of DPOs (see Table 2 below. A subsequent single-tranche DPO did not perform better, hampered by Haiti’s weak administrative and implementation capacity. The operation aimed to support the country’s program in the areas of institution building and strengthening economic governance in sectors critical to reconstruction and growth. Key achievements under this operation included, nevertheless, the clearance of Government’s arrears toward the electricity company, and some progress towards the use of a Single Treasury Account. 9. However, continued donor support to the government’s reform and institutional-building efforts was crucial. Despite the modest results from previous DPOs, it was crucial for international partners to continue to support the Government’s reform and institution-building efforts, including through technical assistance, analysis and policy advice. As this DPO aimed at addressing institutional and transparency issues in education and water and sanitation, it complemented and supported investment lending by many donors in areas that were critical for development impact and in particular human development, through for example better schooling outcomes and reducing cholera risk through improved water quality. Table 3: Haiti: Performance of recent Development Policy Operations Title of the operation Period Nature of the operation Disbursed Amount ($million) IEG Outcome Rating Haiti Second Economic. Governance Reform FY09/10 Programmatic 24.1 Moderately Unsatisfactory Haiti Third Economic. Governance Reform Operation FY09/10 Programmatic 12.5 Moderately Unsatisfactory Economic Reconstruction and Growth Development Policy Grant FY13/14 Single-tranche 20.0 Moderately Unsatisfactory Source: World Bank Data. 5 1.2 Original Program Development Objectives (PDO) and Key Indicators (as approved) 10. The operation’s overarching development objective was to strengthen transparency and build institutional capacity for budget management in the education and water and sanitation sectors. The operation was structured around six specific objectives (see Table 1 above) under three key policy pillars: • Pillar I: Enhancing public financial management. The specific objective under this pillar was to improve cash management by the treasury department, enabling better execution of sectoral budgets. • Pillar II: Enhancing governance and service delivery in the education sector. The three objectives under this pillar were to: (i) improve the capacity of the Ministry of Education and Vocational Training (Ministère de l’Education Nationale et de la Formation Professionnelle, MENFP) to regulate non-public education services, (ii) increase transparency through more timely and widely accessible information on education service delivery, and (iii) improve the alignment of the education budget with sectoral priorities. • Pillar III: Enhancing governance and service delivery in the water and sanitation sector. The two objectives under this pillar were to: (i) improve oversight of the water and sanitation sector, and (ii) increase monitoring and transparency of water service availability and quality at the community level. 1.3 Revised PDO and Key Indicators, and reasons/justification 11. The Development Objectives were not revised during implementation. 1.4 Original policy areas supported by the operation Policy area I: Enhancing Public Financial Management 12. The sole policy objective pursued under this pillar was to improve cash management by the treasury department to enhance the execution of sectoral budgets. The prior action that set the stage for this reform was the publication by the Ministry of Economy and Finance (Ministère de l’Économie et des Finances, MEF) of regulations requiring line ministries to submit annual forecasts of their respective monthly expenditures. Actions under this policy area were intended to enable the Treasury Department to better manage disbursement requests throughout the budget year, and to better align requests with available revenues. 6 Policy Area II: Enhancing Governance and Service Delivery in the Education Sector 13. The first policy objective under this pillar aimed to improve the capacity of the MENFP to regulate non-public education services. As prior actions, the government (i) established a new regulatory framework for decentralized non-public school accreditation and (ii) adopted a regulation to create a public-private advisory committee tasked with establishing accreditation criteria. The expected outcome was a more effective accreditation system marked by increased school coverage, more decentralized processes and greater stakeholder oversight. 14. The second policy objective was designed to increase transparency by providing timely and widely accessible information on education service delivery. Prior actions included (i) the decentralization of the annual collection and compilation of school census data by DDEs, and (ii) the publication of census results in the Statistical Yearbook and in department-level flyers within 45 days of the close of the school year. The expected result was the timely availability of richer data on education services across Haiti, disaggregated by department and gender, and increased consumer access to information on the supply of education services, including class size by education level and department. 15. The third policy objective was to improve the education budget’s alignment with sectoral priorities. While the government has demonstrated a strong commitment to education reform, education financing represents a significant challenge. Financing needs are substantial, and the existence of multiple sources of funding for a plethora of school initiatives complicates the task of implementing a comprehensive development plan. As a prior action, the MENFP developed and adopted an integrated Annual Operational Plan in the context of the MENFP’s medium-term expenditure framework. The expected outcome was a more comprehensive and prioritized budgeting process that better integrates domestic and external resources to advance sector priorities. Policy Area III: Enhancing Governance and Service Delivery in the Water and Sanitation Sector 16. The first objective under this pillar was to improve oversight of the water and sanitation sector. As a prior action, the authorities submitted DINEPA’s proposed Executive Board members for approval by the Senate. The expected outcome was an improvement in DINEPA’s functionality through the establishment of an appropriate institutional and oversight framework. The performance indicator was the Executive Board’s approval of DINEPA’s planned program of activities and annual budget. 17. The second objective was to strengthen monitoring and improve the transparency of water-service availability and quality at the community level. As a prior action, DINEPA expanded its water sector oversight structure beyond water quality to include service quality and coverage, asset monitoring and revenue tracking within the CAEPA monitoring framework. The expected outcome was a more transparent monitoring system that generates performance information on service delivery, which can be used by 7 service providers, consumers and other stakeholders to improve water-service access and quality in rural communities. 1.5 Revised Policy Areas 18. The policy areas were not revised. 1.6 Other significant changes 19. Not applicable 2. Key Factors Affecting Implementation and Outcomes 2.1 Program Performance 20. The DPG was a single-tranche grant of US$ 17.33 million, which supported actions (see Table 3) that complemented previous reforms and that constituted a coherent reform program. At appraisal, several key factors supported a standalone approach. It was hoped that a standalone operation would allow for greater flexibility to capitalize on reform opportunities as they arose, and that its relative simplicity would accommodate the government’s weak implementation capacity. A Joint Budget Support Group, including the World Bank, the Inter-American Development Bank (IADB), the EU, the Spanish Agency for International Development Cooperation and the French Development Agency, convened regularly to define a reform program with the government, the result of which was an annual joint policy matrix. This operation was also the result of understandings among representatives of the HRF trustees involved in sectors supported by the operation. 8 Table 4: Policy Matrix (Prior actions) The Grant’s Program Development Objective (PDO) is to strengthen transparency and institutions for budget management in education and water and sanitation Pillar 1: Enhancing Public Financial Management Policy Objective 1 - Improve cash management by the Treasury, enabling a better execution of sectoral budgets. Prior action #1: The Government of Haiti, through its Ministry of Economy and Finance, has issued and published on June 16, 2014, the regulations requiring line ministries to submit thereto annual forecasts of their respective monthly expenditures. Completed Pillar 2: Enhancing governance and service delivery in the Education Sector Policy Objective 2.1 - Improve MENFP regulation of non-public education services Prior action #2: The Government of Haiti (i) established a new regulatory framework for decentralized non-public school accreditation]; and (ii) adopted a regulation for creating a public-private advisory committee tasked with establishing accreditation criteria. Completed Policy Objective 2.2 - Increase transparency through timely and more widely accessible information on education service delivery. Prior action #3: The Government of Haiti, through the MENFP, adopted regulations for the (i) decentralization of the annual collection and compilation of school census data to Department-level Directorates, and (ii) publication of census results in the Statistical Yearbook and in Department-level flyers within 45 days of the close of the school year. Completed Policy Objective 2.3 - Improve the Education budget alignment with sector priorities. Prior action #4: The Government of Haiti, through the MENFP, has institutionalized the preparation and adoption of an integrated Annual Operational Plan in the context of MENFP’s medium-term expenditure framework. Completed Pillar 3: Enhancing governance and service delivery in the Water and Sanitation Sector Policy Objective 3.1 - Improve oversight of the water and sanitation sector. Prior action #5: The Government of Haiti, through the Prime-Minister’s Office, has submitted to the Senate for approval thereby the nomination of DINEPA’s Executive Board members so as to establish the management structure of DINEPA. Completed Policy Objective 3.2 - Increase monitoring and transparency of water service availability and quality at the community level. Prior action #6: The Government of Haiti, through DINEPA, has expanded its water sector oversight structure beyond water quality to include service quality and coverage, asset monitoring and revenue tracking in the CAEPA/OP monitoring framework. Completed Source: World Bank Official documents. 9 21. Despite the swift completion of all prior actions, progress on the DPG- supported reform program was limited, and half of the result indicators were missed, while the other half was partially achieved. The expected outcomes were not realized, and all target indicators were either missed or partially achieved. The factors behind the DPG’s unsatisfactory outcomes will be analyzed in the chapters of the implementation completion report (ICR) that cover program design and implementation, as well as the performance of both the World Bank and the Borrower. The status of the DPG’s outcome indicators is summarized below and detailed under Section 3.2 and in Annex 2. • While the prior action on PFM was completed, and other positive measures were approved as planned, the execution of sectoral budgets did not significantly improve due to weaknesses in budgetary management and controls, combined with insufficient revenue outturns. • Progress on improving governance and service delivery in the education sector was mixed, as outcome indicators for specific objectives were either missed or partially achieved. School accreditation did not progress as planned in part due to a change in Government’s approach to accreditation, and timely and widely accessible information on education service delivery was not made available, despite positive initial actions. The ongoing institutional restructuring at the MENFP prevented the alignment of the budget with sectoral priorities and triggered unplanned expenses. • Efforts were made to improve DINEPA’s oversight capacity, but the Senate failed to approve DINEPA’s Executive Board, a key driver of progress in this area. Improvements in the transparency of water-service availability and quality indicators at the community level were impeded by unreliable funding to undertake the activities to be measured (supply of water-treatment chemicals) and the costs of data communication. 22. All prior actions were completed before the DPG’s approval and were intended to lay the groundwork for the reform agenda supported by the operation. The achievements of the prior actions is summarized in Table 3 above. 2.2 Major Factors Affecting Implementation (i) A difficult political environment and weak technical capacity limited commitment to the reform agenda. 10 23. The Government’s attention was diverted by the overwhelming pressure of implementing reconstruction activities funded by non-conditional resources. When the operation was being designed and implemented, the Government was deeply involved in post-earthquake reconstruction activities, and significant pledged external resources were being disbursed. The government found it difficult to devote its limited administrative resources to the implementation of the reform program, once the one-off disbursement had been made. The DPG was part of the 18 percent of the HRF resources that were used to provide budget support, while the remainder of the resources were allocated to investment, humanitarian and reconstruction activities. Institutional weakness and fragmentation of Government action, often financed by external resources, led the MEF and the line ministries to prioritize the implementation of reconstruction projects supported by investment funding untied to policy reform. 24. Political instability weakened the oversight of the reform agenda. A fragile political and security environment undermined the implementation of the DPG-supported policy measures. The Martelly administration was characterized by frequent changes in government and delays in the electoral calendar. Tensions between the executive and legislative branches complicated the approval of legislation needed to improve economic performance and the business climate in general, and to advance the DPG-supported reforms in particular. The weakness of political institutions hampered Government’s ability to conclude on pending issues and execute its policy undertakings under the operation. For instance, the Senate’s failure to nominate the members of DINEPA’s Executive Board limited independent oversight of the action plan and budget for the water and sanitation sector, which was a key DPG-supported objective. 25. Weak public institutional and technical capacity also hindered progress on the reforms supported by the DPG. A 2015 Systematic Country Diagnostic noted that low public administrative capacity, as well as a heavy dependence on donor financing and external technical assistance (TA), tend to undermine the government’s ability to use public resources efficiently. Capacity constraints are particularly acute among the institutions devoted to producing key data. The demands of overseeing the management of the disaster- reconstruction agenda overwhelmed the government’s weak technical capacity, preventing the completion of several DPG-supported reforms. For instance, in the second half of 2014 a new DINEPA General Director was nominated, and staff members crucial to the DPG- supported reforms either left the agency or were reassigned to other tasks. The reorganization of human resources and limited domestic funding undermined local ownership of the reform agenda and diminished its institutional momentum. (ii) Lack of analysis of political economy constraints to inform the operational design. 11 26. While the Bank’s team succeeded to convene all stakeholders around a coordinated agenda of reforms, the operation’s design did not rely on an analysis of political economy constraints or on lessons learned from the previous Haiti DPO. In a context of state fragility and ongoing reconstruction efforts, the Bank’s team successfully brought together donors, the Government and interested departments of the Bank around a set of policy reforms that were supported by all. However, the operation’s design was not informed by a political economy analysis of sector reforms and once the budget support operation was disbursed, very few resources and technical assistance were made available to help the implementation of reforms. In such a situation, the incentives for line ministries to implement the measures needed to reach the program’s targets were very weak. Moreover, the lessons drawn later from the most recent IDA DPO, which underlined the risk of stand-alone DPGs were not yet available when this operation was designed. The ICR suggested that a programmatic approach would have been more appropriate, as it would have created much stronger incentives for policymakers to follow through with program objectives. 1 However, in acting as a Partner Entity for the HRF, the World Bank and its project team faced strong limitations. The funding made available by donors to the HRF for this operation needed to be committed within the Haitian fiscal year (as required by donors’ respective authorizing environments). And, the limited scope of this funding could not have accommodated a programmatic approach. 27. Nevertheless, designing the DPG as a stand-alone operation was risky, given the Haitian government’s overextended institutional resources and the short period of reform implementation. While the number of outcome indicators was limited, implementing policy reforms in multiple sectors in a fragile context requires robust coordination capacity and strong incentives to maintain reform momentum. A beneficiary survey revealed that a large majority of the senior government staff involved in implementing the reform program was not satisfied by the level of coordination at the government level (see Annex 4, Table A4.1). 2 (iii) Donor incentives. 28. HRF funding for budget support in Haitian FY2014 was critical to balancing the government budget for that year. Prior actions were swiftly undertaken and the preparation timeline was compressed. HRF donors contributing resources for the operation helped formulating the policy measures and implementation targets. They were eager to ensure that these serve to improve the environment in which their own projects in PFM, education, and water and sanitation were being implemented. As a result, a combination of measures and results were included from diverse policy areas of PFM (World Bank), education (World Bank, France, Canada) and water and sanitation (World Bank, Spain and IADB). The multiplicity and diversity of these measures and the expectations and participation of multiple contributors presented an added challenge for the Government 1 ICR for the Haiti Economic Reconstruction and Growth Development Policy Grant, p.22, June 2015. 2 A total of 68 percent of surveyed senior staff in the three branches of the public service that implemented the reform agenda rated their satisfaction regarding the level of coordination of the reform program as 2 on a scale of 1 to 5, with 5 being the highest score. 12 and the team in developing and implementing a coherent program of reform in the given time frame. (iv) Weaknesses in the mitigation of identified risks. 29. During the DPG’s design phase a number of risks were correctly identified, but mitigating actions were difficult to undertake during implementation. The identified political, governance and institutional risks not only materialized, but significantly weakened the Government’s ability to advance the reform agenda. However, there were few actionable avenues to mitigate these risks. For instance, political gridlock between the President and the legislature became an obstacle to the effectiveness of DINEPA’s leadership structure, and there was no way to overcome this obstacle and complete the reform. In addition, better harnessing of resources available elsewhere in the Bank’s work program (an education operation under implementation and a water and sanitation operation under preparation), to achieve reform objectives would have been beneficial. This would have required stronger collaboration between the Task Team and the Education and Health teams as well as greater creativity in using all instruments to achieve objectives. However, although such collaboration may have generated marginally better results, weak incentives for Government action under the stand alone design would have remained a critical obstacle. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 30. M&E Design. Overall, M&E capacity was in place for the operation. The MEF was responsible for coordinating and implementing the activities and reforms under the DPG. The MEF was also tasked to monitor and evaluate the program’s implementation in close coordination with the MENFP and DINEPA. The results chain appeared sound, as outcome indicators for each specific objective had realistic initial and final benchmarks. A supervision committee headed by the MEF was to be created to facilitate coordination, and the MEF was in charge of reporting progress on the outcome indicators listed in the results framework of the policy matrix. In parallel, collaborative monitoring was to take place among donors with the government’s support. Originally, implementation was to be monitored as part of the joint donor matrix agreed upon by the government and the Budget Support Group, but the Government abandoned the joint donor matrix in November 2014 in favor of a single matrix that focused exclusively on the PFM agenda. 31. M&E Implementation. Although there was a continuous dialogue between the Task Team Leader in the field and the authorities on the implementation of the reform agenda, M&E occurred primarily during supervision missions. Two supervision missions were completed, for which aide-memoires were produced, in March and June 2015— a full 9 and 12 months after the operation’s approval. The mission teams were comprised of World Bank staff and specialists who, in coordination with a large government team, reviewed progress in the implementation of the supported reform program. Each supervision mission determined that the reform program had been delayed and agreed upon a set of remedial measures with their government counterparts. Two additional implementation support missions took place in September and December 2015, and found once again that the reform program was falling short of its targets. While no formal aide 13 memoires were prepared, the Bank team continued to discuss with the authorities the shortcomings of the implementation process as well as corrective measures, which were not followed through. Finally, the supervision committee was never established. Some line ministries complained that the MEF did not effectively coordinate the reform program and suggested that a more participatory approach to M&E would have led to better results. 32. M&E Utilization. While the aide-memoires included detailed assessments of the progress achieved and the obstacles encountered, the implementing agencies did not have a strong incentive to execute the agreed-upon remedial actions. Overall, M&E utilization was not effective because the government’s capacity to guide reforms in DPG’s three sectors was insufficient. M&E is rated Unsatisfactory to reflect the fair quality of M&E design, and weak M&E implementation and utilization. 2.4 Expected Next Phase and Follow-Up Operation 33. In the context of the new Country Partnership Framework for FY16-19, the World Bank Group plans to provide (i) a programmatic budget support operation, (ii) a new statistical and PFM capacity-building project, (iii) TA programs, and (iv) timely policy advice. Donor coordination will continue to be led by the Budget Support Group, which includes all partners providing policy-based support to Haiti. Going forward, a programmatic multiyear approach to policy-based lending, with clear triggers for successive implementation phases, is likely to be more effective than single-tranche DPOs. To maximize predictability and support the programmatic approach, IDA financing rather than Trust Fund resources is being envisaged. In addition, resource to support TA for the implementation of the reform program will be made available. The satisfactory completion of the delayed presidential elections, as well as improved coordination and dialogue between the government and its development partners, will be critical to the success of future operations. 34. The World Bank Group intends to develop Haiti’s data infrastructure, analytical tools, financial policies and general institutional capacity based on effective models of service delivery. In this context, the World Bank Group will support analytical work aimed at improving the financial and technical sustainability of basic services in several key sectors, including water and sanitation, social protection, education and energy. 14 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 35. Relevance of objectives. Overall, the operation’s objectives were highly relevant and consistent with the three main areas of the PARDH: (i) physical reconstruction with a focus on infrastructure, especially in the water and sanitation sector; (ii) social reconstruction to create networks of modern education centers; and (iii) institutional reconstruction with a focus on the government’s essential PFM and financial-oversight functions. The content of the DPG was also aligned with two of the four strategic objectives of the 2013-14 Interim Strategy Note: (i) reducing vulnerability and increasing resilience (e.g. in water and sanitation) and (ii) building human capital (e.g. in education). The operation was also consistent with the note’s overarching theme of strengthening governance and capacity (e.g. in PFM). Evaluations at the appraisal and at the closure phase of the operation found that its design clearly reflected the government’s stated priorities and was well integrated into the agenda of the World Bank and other donors. Table 4 below shows that a large majority of senior government staff working in the agencies responsible for implementing the DPG were aware of and supported the reform agenda promoted by the operation. Table 4: Beneficiary Support for the Operation’s Objectives Questions Number of responses Yes No Other Did you understand and support the objectives of this operation? 35 94% 3% 3% Were these objectives integral to the mission of the department of which you are in charge or to which you belong? 35 74% 23% 3% Total 70 84% 13% 3% Source: ICR team survey. 36. Relevance of Design. Overall, the operation’s design was modestly relevant. Both the overarching objective and the specific objectives were clearly stated in the program document and were complementary. There was also a logical consistency between the overarching objective, the specific objectives, the prior actions and the outcome indicators (although results indicators were stated as objectives instead of indicators). For instance, if the Treasury Department is to improve its cash management, it will need reliable information on line ministries’ commitments and expenditures, and both the prior action and the target outcome contributed to that objective. Similarly, a more effective accreditation system is crucial to improving the regulation of non-public education services by the MENFP. However, in a fragile country context, the results framework could have been more flexible and nuanced. While the completed prior actions were well chosen, the identification of outcome indicators could have been made more flexible so as to capture progress on intermediate outcomes, such as issuing school identity cards. Likewise, the M&E design should have included indicators that were easier to be monitored. 15 37. Moreover, the choice, the specific objectives, and their articulation with the prior actions were not optimal. Not only the prior actions have to be one-off actions, but identified specific objectives must be consistent with the short term nature of a stand-alone operation. On the contrary, many of the supported actions resembled institutional strengthening type of actions and required supervision and TA for their full effectiveness. All reforms in the supported sectors (PFM, education, water and sanitation) were affected by this design flaw, which partially explains the limited performance of the operation. 38. Implementation. All stakeholders were committed to the reform agenda and executed their respective roles during the operation’s appraisal and preparation phases. After grant approval and resources disbursement, stakeholders’ engagement in the implementation of the reform program supported by the DPG varied significantly. The World Bank team supervised the implementation of the reform program, consulting – at times - with HRF donors. Line Ministries lacked sufficient incentives to move ahead with a difficult reform agenda while central authorities were more preoccupied with addressing the challenges of the post-earthquake reconstruction effort. The operation’s outcomes reflect risks and other challenges that were identified at appraisal, but which could not be successfully mitigated during implementation. 3.2 Achievement of Project Development Objectives Pillar 1: Enhancing Public Financial Management Objective 1: Improve cash management by the Treasury Department, enabling better execution of sectoral budgets: Partially Achieved. 39. Improved sectoral budget execution was partially achieved, because the tracking of monthly arrears failed, mainly due to lack of coordination in the budget management and limited government commitment. All ministries submitted their annual forecasts of monthly expenditures to the Treasury Department, and during the implementation period, the MEF was still very much involved in making sure that the Single Treasury Account functioned properly. But a lack of coordination between the line ministries and the financial and budget controller, as well as delayed reporting of budget commitments, were major obstacles to tracking monthly arrears. Monthly budget execution reports showed lower amounts than the effective budget commitments made by line ministries, as a result of which line ministries did not meet their monthly arrears-payment schedule as envisaged under the program. Meanwhile, with the rollout of the Single Treasury Account, the Treasury Department now makes payments directly to line-ministry vendors. In summary, while the prior action was completed on time, and some positive steps were taken, the execution of sectoral budgets could not be significantly improved because the Treasury Department did not receive accurate and timely information on budgetary commitments. While the Government made progress in streamlining the use of the Single Treasury Account and strengthening its technical capacity for PFM, the objective of improving sectoral budget execution was not achieved, mainly because of lack of coordination in the budget management, limited government commitment and inadequate supervision. 16 Pillar 2: Enhancing Governance and Service Delivery in the Education Sector Objective 2: Improve the MENFP’s Regulation of Non-Public Education Services: Partially Achieved. 40. The nomination of a new Director General at the MENFP and the support of Haiti’s development partners led to renewed efforts to establish a public-private accreditation group, the National Education Partnership Office (Office National de Partenariat en Éducation, ONAPE), and to accelerate the accreditation process for non-public schools. Due to a significant shift in policy at the Ministry of Education which led to a complete revision of the accreditation process, original objectives of the operation could not be met. ONAPE has not launched its school-accreditation process yet, as the authorities have adopted a new accreditation approach designed to more precisely reflect a new school structure. Previously, school licenses were awarded to primary and secondary schools separately. The new school structure is composed of two levels; the first includes primary school and the first three years of the former secondary level, and the second comprises only the last four years of the old secondary level. The MENFP is currently delivering 17,191 separate school identity cards that reflect the new school structure, though by end-2015 only a fraction of these cards had been issued. All World Bank activities, including investment financing, in the sector have had to adapt to these substantial structural changes. 41. The World Bank team is already pre-piloting instruments to engage the government under this new approach, ensuring that the important subject of accreditation remains central to a large World Bank-financed Investment Project under preparation in the education sector. In addition, Non Lending TA activities are ongoing to develop new instruments for collecting data on the dimensions of accreditation—school infrastructure, teaching staff, managerial practices and pedagogical techniques. These will be used as a basis to identify which schools have all the elements in place to be recognized as effective learning environments. This objective is assessed as partially achieved due to the government’s efforts to revamp public school accreditation. Objective 3: Increase transparency by providing timely and widely accessible information on education service delivery: Partially Achieved. 42. A digital school-mapping exercise based on a geographic information system was launched, but its completion has been delayed. A full 78.5 percent of schools have been mapped, a program for computerized data treatment has been established, and the Statistical Yearbook tables for 2012 and 2013 have been finalized. The preparation of the Statistical Yearbooks for 2014 and 2015 is now underway, and the MENFP’s website displays annual reports that include data on recent developments in the education sector. Although these measures were not originally included in the program’s objectives, the MENFP’s progress on school mapping and its annual updates will facilitate access to education information and facilitate donor planning and program coordination. This objective is therefore regarded as partially achieved. 17 Objective 4: Improve the education budget’s alignment with sectoral priorities: Not Achieved. 43. At the end of the second quarter of FY15, budget allocations to the first two primary-school cycles reached 50 percent and 52 percent of the MENFP’s operating costs for FY15 and FY16, respectively. At the same time, non-salary allocations reached 22 percent and 26 percent of the MENFP’s FY15 and FY16 operating budgets, respectively. The ICR mission of March-April 2016 was unable to obtain accurate data on these allocations for 2015 and 2016. 44. Due to the ongoing reorganization of the education sector, the MENFP did not achieve the targets of spending 52 percent of its operating budget on primary schools in FY15 and 55 percent in FY16. In addition, three specific factors prevented the authorities from reaching their targets for non-salary allocations. First, two salary adjustments in FY15 increased the wage bill at the expense of other spending items. Second, 4,000 new teachers were added to the payroll and paid via the budget. And third, the recruitment of 3,000 new teachers prompted officials to reallocate 400 million Haitian gourdes from the FY2016 budget that had previously been earmarked for primary- certificate exams. Overall, the reorganization of the education sector proved to be a major obstacle that prevented the DPG from realizing its expected outcomes, as budget reallocation and restructuring can only take place once the sector’s reorganization is complete. Consequently, this objective was not achieved. Pillar 3: Enhancing Governance and Service Delivery in the Water and Sanitation Sector Objective 5: Improve oversight of the water and sanitation sector: Partially Achieved. 45. Progress on this objective was limited. Although the names of the proposed DINEPA Executive Board members were submitted to the Senate, there was no quorum to approve them. As a result, DINEPA’s budget and work program for 2015 and 2016 could not be approved by the Executive Board and were instead approved by the acting management. Nevertheless, as the list of proposed Executive Board members was submitted to the Senate, and the budget and work program for 2015 and 2016 were approved by DINEPA’s management, this objective is assessed as partially achieved. Objective 6: Increase monitoring and improve the transparency of water-service availability and quality at the community level: Not Achieved. 46. There are currently 163 CAEPAS and OPs reporting to DINEPA, which has started to publish their performance information on its website. This was an important first step toward increasing access to information on water-service providers. However, further progress on this objective was hampered by a lack of budgetary funding, and publication was eventually interrupted. The unreliability of funding to cover the supply of water-treatment chemicals and the high cost of communicating data from the CAEPAs and 18 OPs to the DINEPA’s central database managed by the “Observatoire” prevented the authorities from achieving this objective. 3.3 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory 47. The overall rating of the DPG outcome implementation is moderately unsatisfactory. The objectives of the DPG were relevant at appraisal, but the design flaws and the materialization of implementation risks undermined progress on its intended outcomes. As half of the target results indicators identified at appraisal were missed, and the other half were only partially achieved as discussed under Section 3.2 above, the program’s overall outcome is rated as moderately unsatisfactory (MU). This MU rating is justified by progress achieved between the last ISR and this ICR date, including the publication of annual reports on the MENFP’s website, the distribution of school licenses in some departments and the digital mapping of 100 percent of the schools. Though the latter two were not included in the policy matrix of the DPG, they were important and necessary policy measures to address the new school structure issue. 3.4 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development. 48. The Haitian government continues to suffer from limited administrative and technical capacity; social spending remains low, and the delivery of basic services is both inadequate and highly inequitable. 3 The HRF-supported DPG had major poverty implications. The operation’s PFM agenda aimed to improve the execution of sectoral budgets, increase the effectiveness of spending in key sectors such as water and sanitation and education and create the necessary conditions for the government to sustainably provide public investment and social spending. 49. The DPG focused on sectors with especially important implications for poverty and social development. In the water and sanitation sector, while DINEPA has greatly expanded its presence in rural areas and small towns through CAEPAs and OPs, rural water and sanitation infrastructure and services are still frequently funded by NGOs. These NGOs are largely disconnected from the public system, and limited coordination leads to highly uneven service provision. In the education sector, a large number of primary school students attend non-public schools, placing a substantial financial burden on households, and the lack of accreditation creates uncertainty regarding education quality. Moreover, in both the social and infrastructure sectors, an increasing reliance on donors and NGOs has weakened public investment management. The HRF-supported structural reform agenda in the education, water and sanitation sectors provided an opportunity for the World Bank to build the government’s capacity to provide high-quality education and water and sanitation services. 3 The most recent Systematic Country Diagnostic ascribes these features, at least in part, to the winner-take- all nature of Haitian politics. 19 50. While the operation’s potential for poverty and social development impact was substantial at the design stage, the limited results achieved at closure do not guarantee that it will yield positive effects on poor households, unless follow-on operations are swiftly initiated and able to further advance the supported reform agenda. Finally, the operation had the potential to positively influence gender developments in the country, but its limited effectiveness and the lack of data don’t allow an assessment of that impact. (b) Institutional Change/Strengthening. 51. The DPG focused on strengthening management institutions in the MEF, MENFP and DINEPA. The reform program supported by this operation was intended to build the capacity of the treasury department in the MEF, the department in charge of managing resources in the MENFP and DINEPA’s system for monitoring local service quality. Due to the limited progress achieved during the operation, its impact on institutional strengthening was minimal at best. (c) Other Unintended Outcomes and Impacts 52. None. 4. Assessment of Risk to Development Outcome Rating: High 53. Performance of the operation towards expected outcome was limited, as key outcome indicators were either missed or partially achieved as developed under Section 3 above. However, there were some achievements in each area of reform. 54. Overall, achievements fell short of the targeted outcomes in each area of reform, and the identified overarching objective was not attained. The limited achievement of outcomes described in Section 3 may be a first step toward greater success if the Government stays committed to the reform agenda and if the donors stay the course in their lending and technical assistance support. In the context of the new Country Partnership Framework (FY16-19), the WBG envisions a programmatic approach to policy-based lending over several years to Haiti, with clear triggers to move from one phase to another, and this approach is likely a better option than single tranche operations. However, the success of even this option depend on the return of political stability in Haiti and the continuation of a strong post-election dialogue between the Haitian institutions and the donor community. Because of the political instability currently prevailing in Haiti, the risk to development outcome is assessed as high, despite expected positive developments described above. 20 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry. Rating: Moderately Unsatisfactory 55. The World Bank and other donors effectively coordinated their work in Haiti, and this operation represented a first step in the shift in focus from post-disaster emergency assistance to addressing structural development challenges involving governance and transparency. World Bank-donors coordination was particularly effective at the design stage of the operation. There was strong hope that the combination of the policy measures of the DPG and the investment activities supported by other donors could generate the momentum needed to improve measurability and transparency of service delivery in education, water and sanitation. The need to improve PFM was also addressed through ongoing TA provided to the MEF and other central government bodies under the World Bank’s Infrastructure and Institutions Emergency Recovery Project. Development support in the education and water and sanitation sectors was provided in parallel by the World Bank, Spain, France, Canada and the IADB. 56. The design of the operation suffered from limited risk analysis in its targeted sectors. Given that more than 80 percent of the outcome indicators were related to education and water and sanitation, a better risk analysis could have underscored the fact that many inputs were missing to ensure effective reporting, which depended on factors outside the DPG. The attention of the authorities was captured by more immediate demands of post-disaster reconstruction, and there was no time or appetite to conduct in- depth analysis of underlying risk issues and appropriate remedies. Moreover, limited information was available on the status of the water and sanitation sector, partly because of the Bank’s limited engagement in the sector. Finally, the Bank’s team and the donors were over optimistic about their ability to harness additional resources and to link the policy measures of the DPG to supporting actions on the ground under other instruments. 57. The design of the operation was constrained by the fact that it was an HRF financed operation. International experience from DPOs in fragile environments should have weighed more heavily in the balance in the Bank’s dialogue with DPOs donors when designing the operation. For example, it is recognized that stand- alone operations do not provide sufficient incentives for reform targets to be met 12 months on. The team was constrained in the design of the operation as a stand-alone by the institutional decision to support access of GOH to HRF resources at a time of very constrained financing. There was also little time to prepare a programmatic series of DPGs, as the Government was in dire need of resources to close its budget and to comply with IMF program quantitative measures. Moreover, the Bank team and the donors felt that critical policy measures in the water and education sectors should have been supported by the DPG, in the hope that 21 advancement on the policy front would improve the impact of donor-funded investments in these sectors and facilitate the achievement of results. 58. The incorporation of multiple priorities into the DPG contributed to the development of a relatively ambitious reform agenda that exceeded the capacity of Haiti’s fragile public administration, particularly when identified risks materialized. The DPG aimed to serve as a catalyst and a tool to facilitate the implementation of ongoing investments by donors and to strengthen the achievement of results in PFM, education, and water and sanitation. The reform program ultimately included prior actions and outcome indicators for each of the three areas, a challenge for Haiti’s weak public sector hampered by a volatile political environment. In retrospect, the fragmented nature of the proposed reform program outstripped the client’s implementation capacity. 59. All implementation risks materialized and proved very difficult to mitigate or counter. The obstacles to achieving the program’s target outcomes were correctly identified at appraisal; they included political, governance, institutional and fiduciary risks, all of which materialized to one degree or another. Political and institutional risks were the most damaging to the operation. Implementation and M&E arrangements were adequate in principle, but they proved less effective than originally anticipated. (b) Quality of Supervision. Rating: Moderately Unsatisfactory 60. The World Bank effectively executed the project appraisal and preparation phases, but supervision was lacking. The task team leader in the field, charged with the responsibility to supervise the implementation of the DPG, maintained day-to-day interactions with the Government and other stakeholders, facilitating some progress. However, the lack of formal supervision missions during the first 8 months after the approval of the DPG meant the absence of formal communications with relevant Ministers on the absence of progress in advancing the reform agenda. With hindsight, the government needed more direct support than initially anticipated to address implementation problems. Two supervision missions took place in March and June 2015 but the suggestions in their aide-memoires were too late to turn around the reform program. Additional supervision missions took place in September and December 2015, though aide- memoires for these were not produced. The findings from these missions and suggested corrective measures were discussed with the authorities. This allowed some progress to be achieved between the last ISR and the ICR date. 61. World Bank coordination of its own and other donor instruments to advance the reform agenda was ineffective. The DPG team and the WB water and education teams met frequently during preparation, but these interactions became less frequent during implementation. Likewise, lack of timely management guidance to task team negatively impacted progress in advancing the reform agenda. Meetings with donors who financed the DPG through the HRF did take place. However, no documentation exists of concerted 22 and structured efforts to identify synergies between the reform agenda and the projects being implemented in parallel by other donors in the relevant sectors. (c) Justification of Rating for Overall Bank Performance. 62. With regard to design, the stand-alone nature of the operation presented high risks which were recognized but could not be mitigated. Also, the Bank was not successful in streamlining the measures and concentrating on significant irreversible reform measures. However, much of this could not be avoided, given the source of funding and the (mandated) role of the HRF steering committee in endorsing the content of the operation. With regard to supervision, formal missions were not carried out early enough after the operation was approved. Also, although aide memoires spelled out clear objectives and correction measures, they did not lead to concrete and explicit actions on the part of other WB teams or other donors in the relevant sectors, where such actions may have increased the DPG’s chances of success. Because of these shortcomings in the areas of operational design and supervision, the World Bank’s overall performance is rated as Moderately Unsatisfactory. 5.2 Borrower Performance (a) Government Performance. Rating: Moderately Unsatisfactory 63. The DPG program document identified the MEF as the entity in charge of coordinating and implementing the activities supported by the operation. The MEF was also tasked with monitoring and evaluating the program’s implementation in close coordination with its Treasury department, the MENFP and DINEPA. These three institutions actively supported the completion of the prior actions and effectively collaborated with the World Bank and other donors in preparing the operation for Board approval. However, institutional capacity weaknesses undermined their performance in implementing the agreed-upon reform agenda. However, lack of incentives from the implementing agencies and the fact that junior staff was tasked by the MEF to liaise with the Director Generals of MENFP and DINEPA were a hindrance to meeting the DPG’s targets. 64. A Beneficiary Perception Survey confirmed that a large majority of senior staff involved in implementing the supported reforms (80 percent of respondents) were fully aware of the program’s existence and objectives (Table 5 below). However, their ability to advance the reform agenda was negatively affected by institutional capacity limitations, competing domestic priorities and inadequate World Bank supervision. 23 Table 5: Beneficiary Perception of the Quality of the Government’s Internal Supervision Questions Total number of responses Yes No Other Were you informed of these policy objectives in 2014? 18 78% 17% 5% If so, did you receive the information from the administrative official in charge of the indicator? 18 83% 17% 0 Did you know that the implementation of this measure was part of a reform program supported by the World Bank? 18 78% 17% 5% Total 54 80% 17% 3% Source: ICR Team survey. 65. The MEF was expected to create a supervision committee to facilitate coordination, and to report progress on the program’s outcome indicators. In parallel, donors were to undertake a collaborative monitoring process supported by the government, with overall implementation monitored through the joint donor matrix agreed upon between the government and the Budget Support Group. However, the supervision committee was not created, and the joint matrix was abandoned. Reform momentum dissipated, and support for the supervision of the reform agenda from other donors did not materialize. (b) Implementing Agencies Performance. Rating: Moderately Unsatisfactory 66. The MENFP and DINEPA were in charge of implementing 90 percent of the reforms supported by the DPG. Because of weak coordination by the MEF, these two agencies remained focused on priority issues in their respective sectors and devoted little attention to the reform program. Moreover, as the reforms were not aligned with agency priorities, they did not receive sufficient administrative support. During Bank supervision missions, corrective actions were agreed upon to move the reform agenda forward, but recommendations were not acted upon. During the ICR mission, senior officers made it clear that coordination within the Government was lacking, and that their institutions did not have incentives to prioritize the reforms implementation as there were no financial benefits for those entities grappling with multiple funding problems. (c) Justification of Rating for Overall Borrower Performance. Rating: Moderately Unsatisfactory 67. The government’s commitment to coordinating the implementation of the DPG-supported reform agenda was insufficient to accomplish the operation’s objectives. The failure to set up the supervision committee and the discontinuation of the 24 donors’ joint policy reforms matrix by the Government undermined the MEF’s ability to generate sufficient reform momentum. Both the MEF and the other implementing agencies were preoccupied with priorities in their respective sectors, and due to weak incentives they did not invest sufficient time and attention in supporting the reform agenda. As a result, borrower performance is assessed as Moderately Unsatisfactory. Rating: Moderately Unsatisfactory 6. Lessons Learned The ICR has identified 6 key lessons: • The design of a DPO in fragile environments requires (a) ownership and commitment by the agencies tasked with implementation of measures under the reform program, (b) strong mitigation measures and plans to address implementation failures due to very weak institutional capacity of the Borrower to carry out the reform agenda, and (c) dedicated resources to implement the reform program. To increase the chances of a DPO success in a fragile environment, Bank teams need to ensure that the reform program design is informed by good risk analysis on the sectors of interest. Similarly, the DPO design needs to be well- tailored to existing institutional capacities of the Borrower, and should factor in some kind of TA to help strengthening the Borrower’s institutional and technical capabilities. • Appropriate incentives to implement reforms, particularly at the level of line ministries in very financially constrained environments. To keep up the reform efforts, line ministries need to receive resources from MEF or from other sources to enable and incentivize the implementation of reforms. • Leveraging the support of multiple donors needs to be done carefully and in a sustained way. The reform agenda supported by the DPG was the result of understandings between the government, the World Bank, the Spanish Agency for International Development Cooperation and the French Development Agency, each of which was already implementing projects in the PFM, education, and water and sanitation sectors. Harmonizing TA and other donor’s support was expected to increase reform momentum, but in practice donor coordination was ineffective. Moreover, the agreed-upon reform agenda incorporated prior actions and outcome indicators in three large and diverse policy areas. The scope and ambition of the reform agenda was inconsistent with the unstable political context and the lack of implementation capacity of the public administration. Diffusing the reform efforts across multiple policy areas reduced the focus of the operation and complicated supervision. • The inability to align incentives over a long time period is a crucial drawback of standalone DPOs. The DPG timeframe was just 18 months, during what proved to be a tumultuous period for Haiti. Political instability continued to strain the normal functions of the public administration, and the MEF and other line 25 ministries remained focused on post-earthquake reconstruction and were not able to effectively coordinate the implementation of the DPG-supported reform program. While the government worked closely with the World Bank to ensure the completion of all prior actions, that commitment waned during implementation due to the authorities’ limited technical capacity, and lack of incentives. In similar fragile environments, programmatic DPOs have yielded better results. This conclusion was noted in the ICR for the World Bank’s previous standalone operation in Haiti. 4 • Delivering a policy lending operation in a fragile and high-risk environment requires close and sustained collaboration between the World Bank and the government in order to mitigate weaknesses in local technical capacity. The DPG was designed to be a relatively light operation with fairly simple outcome indicators. However, given the government’s limited technical capacity and the competing demands on its attention, the World Bank should have fielded a large and experienced staff and sent multiple supervision missions early in the implementation period. In retrospective, supervision was late and intermittent, and the conclusions of the supervision missions were not acted upon. • Providing TA to the agencies tasked with implementing the policy reforms could have generated better outcomes. Throughout the DPG’s implementation period the reform agenda for the MENFP and DINEPA could not gain traction due to inadequate technical capacity and limited direct collaboration with the World Bank. Other donors involved in the sector, including the IADB, have been confronted with the same issue. Close technical collaboration can help align incentives and build reform momentum in fragile environments. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies. 68. Underlying the issues raised by the Borrower affecting achievement of the program’s targets is the instability at the Cabinet level and staff turnover. For instance, there were three different Ministers of education and two different Director Generals of DINEPA from appraisal to closing date of the operation. The ensuing institutional memory loss from this instability endangered achievement of the program’s objectives. The appropriation of the DPG resources by the Treasury department left the sectors with little capacity to effectively implement the reform measures. (b) Cofinanciers: Not Applicable (c) Other partners and stakeholders: Not Applicable 4 ICR no. 3393 “Economic Reconstruction and Growth Development Policy Grant for Haiti,” June 2015. 26 Annex 1. Bank Lending and Implementation Support/Supervision Processes (a) Task Team Members Names Title Unit Appraisal and Supervision Teams Elizabeth Ruppert Bulmer Task Team Leader PRMTR Evans Jadotte Co-TTL LCS PE Patricia Holt Language Program Assistant LCSPE Kassia Antoine Research Analyst LCSPE Jean-Martin Brault Water & Sanitation Specialist LC SWS Patrick Ramanantoanina Senior Education Specialist LCSHE Fabienne Mroczka Senior Financial Management Specialist LCSFM Juan Baron Senior Economist- Education GED04 Victor Ordonez Senior Finance Officer CTRLN (b) Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks US$ Thousands (including travel and consultant costs) Lending Total: Supervision/ICR Total: 145,497.88 27 Annex 2: Performance on Development Policy Outcomes Expected outcome Baseline Value Original Target Values (from approval documents) Actual Value Achieved at Completion or Target Years The Grant’s Program Development Objective (PDO) is to strengthen transparency and build institutional capacity for budget management in the public financial management, education and wate r and sanitation sectors Policy Objective 1: Improve cash management by the Treasury Department, enabling better execution of s ectoral budgets. (i) The Treasury Department more effectively manages disbursement requests throughout the budget year and more closely aligns requests with revenue availability. The Treasury Department at the Ministry of Economy and Finance has little knowledge of how line ministries’ expenditure requests will be spread throughout the budget year. Line ministries measure payment arrears on a monthly basis. Line ministries did not measure and send payment arrears on a monthly basis to the Treasury Department on time. Monthly budget-execution reports sent to the Treasury Department, however, showed lower amounts than the effective budget commitments made by line ministries. Partially Achieved Policy Objective 2: Improve the MENFP’s regulation of non-public education services. (ii) A more effective accreditation system reflects increased school coverage, more decentralized processes and greater stakeholder oversight. 10,000 non-public schools remain unrecognized and non-accredited, and the MENFP and Departmental Directorates of Education (DDEs) have a backlog of 4,000 accreditation requests and lack the staffing to carry out accreditations; the Accreditation Commission does not exist. The current backlog of school accreditation requests is eliminated; accreditations are carried out at the regional DDE level; MENFP consults the public-private accreditation group. While the targets were missed, due to a significant change in Government policy on accreditation processes, the following achievements were made: (i ) the public-private accreditation group (ONAPE) was revived in order to accelerate the accreditation process, (ii) the identification of 17,191 schools to which a School Identity Card will be delivered once a set of agreed-upon criteria have been fulfilled, and (iii) the delivery of a portion of the School Ident ity Cards at end-2015. Partially Achieved Policy Objective 3: Increase transparency through m ore timely and widely accessible information on edu cation service delivery (iii) More timely data on the availability of education The school census is published irregularly and Annual information on education services, While the program targets were missed, important achievements were made towards meeting the target, 28 services across Haiti, disaggregated by department and gender, increases consumer access to information on the supply of education services. with delays of up to 8 years; the most recent census was published after a 2-year delay. including regional and gender-disaggregated data, is made available online and in flyers at regional DDE offices within 45 days of the end of the school year. including: (i) the creation of a database that can generate annual reports on the education sector, (i i) the digital mapping of 78.5% of all schools, (iii) the completion of computerized data treatment and Statistical Yearbook tables for 2012, and, (iv) the publication of annual reports on developments in th e education sector, including recent data, on the Ministry of Education’s website Partially Achieved Policy Objective 4: Improve the education budget’s alignment with sector priorities . (iv) A more comprehensive and prioritized budgeting process better integrates domestic and external resources to advance sector priorities. 46% of MENFP’s FY14 operating budget is allocated to primary education cycles 1 and 2, and non-salary expenditures account for 30.6%. 52% of the FY15 MENFP operating budget and 55% of the FY16 MENFP operating budget are allocated to primary education cycles 1 and 2; non-salary budget allocations in FY15 and FY16 represent at least 32% and 35%, respectively, of MENFP’s operating budget. At the end of the second quarter of FY15, budget allocations to the first two primary education cycl es reached 50% and 52% of the Ministry of Education’s operating budget for FY15 and FY16, respectively; non-salary budget allocations reached 22% and 26% of the Ministry’s operating budget for FY15 and FY16, respectively. Updated data on the above allocations were unavailable. Not Achieved Policy Objective 5: Improve oversight of the water and sanitation sector. (v) DINEPA’s institutional and oversight framework is appropriate to its core functions. DINEPA lacks a functioning oversight body. The DINEPA Executive Board approves DINEPA’s planned program of activities and annual budget. The names of DINEPA’s Executive Board members were sent to the Senate, but there was no Senate quorum to approve them. However, DINEPA’s operational budgets for FY15 and FY16 were approved. Partially Achieved 29 Policy Objective 6: Improve monitoring and transpar ency of water-service availability and quality at t he community level. (vi) A transparent monitoring framework generates performance information on service delivery, which can be used by water providers, consumers and other stakeholders to improve the quality of water services in rural communities. DINEPA, through the Rural Departmental Units (URDs) and Community-Level Water and Sanitation Technicians (TEPACs), does not collect performance information from its delegated operators, the water supply and sanitation committees (CAEPAs) and water system professional operator (OPs). DINEPA, through the URDs and TEPACs, has collected and published on a monthly basis, beginning in January 2015, performance information on its delegated operators, including at least 230 CAEPAs and OPs by December 2015. Out of 500 CAEPAS and OPs targeted in the program, only 163 have been reporting performance information to DINEPA. Further progress on this objective was hampered by the lack of funding for water-treatment chemicals which interrupted the activities being measured and the costs of communicating data from CAEPAs to DINEPA’s central database, which is managed by the “Observatoire.” As a result, this publication series was interrupted. Not Achieved Source: World Bank Official documents. 30 Annex 3: Summary of Borrower's ICR and/or Comments on Draft ICR Ministry of Economy and Finance « Memorandum on measures agreed upon and implemented under the program Strengthening Governance in Education, Water and Sanitation Sectors, supported by the Grant Agreement TF017656-HT » Introduction This note assesses the measures agreed upon and implemented under the Grant TF017656-HT. It examines the socio-political and institutional context overseeing the program’s progress in order to assist the Government of Haiti (GoH) in its efforts to lay the foundation for medium and long-term growth through the strengthening of public financial management and governance in the education, water and sanitation sectors. This report also highlights progress made and ongoing strategies and efforts being exerted to implement measures not yet finalized. It also emphasizes the lessons learned and experience acquired from this program to better channel and implement future programs. 1. Implementation of the Program The overall process of public financial management (PFM) reform. By the fiscal year 2013/14, Haiti reached a new milestone in the renovation process of public financial management through a three-pronged scheme, namely: development of a strategy and a three year action plan (2014/16) for PFM implemented through an appropriate governance framework comprised of a Strategic Steering Committee acting as a policy and decision-making entity; a Public Financial Management Reform Commission ensuring coordination and monitoring, and; Reform Sectoral Committees guaranteeing the interface between administrative entities implementing the reforms. Following two assessments carried out on the implementation of the reform, two important trends have been identified: • The increased performance level of the overall reform process has with the multiplicity areas of work where innovations were achieved or are being undertaken have been expanded. • The development of a differential process under a three-pronged approach covering 6 major areas of reform prioritized in the three year action plan 2014/16. However, some major structural problems due to an inappropriate organizational culture, lack of human capital and severe financial constraints continue to weigh heavily. The new results obtained are due to three factors: a new institutional environment, a strong political commitment, and above all an energized cooperation with technical assistance provided to the Haitian Administration, both from multilateral and bilateral partners. 31 Important progress was made, particularly in these two areas: Treasury and Public Accounting and the Financial Management Information System (FMIS). With respect to Treasury and Public Accounting, positive changes were observed in all sub-areas, but the major “success stories” are the implementation of a new accounting structure and the introduction of the Single Treasury Account (STA). The accounting centers, which are headed by public accountants, are rolled in all Ministries, with the exception of the Departmental Directorates. The Parliament and the Judicial Power have their respective accounting centers. However, the municipalities remain an important challenge. The STA implementation is successful with the following strength: • Operation of the Treasury Central Account. • Implementation of the STA structure with secondary revenue accounts (Customs and Internal Revenue Services) and expenditures sub-accounts (wages, operating expenses, subsidies, and investments). • In October 2015, a pilot experiment was launched in the Economic Sector I. • Implementation phase of the STA in all sectors started in December 2015 • Currently, only donors’ funds are not part of the STA. Education Sector Beyond the universal, free and mandatory schooling program (PSUGO) that allowed more than 90% of children to have access to primary education, the Ministry of Education and Vocational Training (MENFP) has adopted 12 policy measures in the summer of 2015. This program includes the following elements: a teachers and school principals census, the granting of a Provisional Teaching Permit (PPE); a school census to allow the authorities to deliver an Identification number to each school. To that we should add a continuing education program for teachers and a major school facilities construction program (almost 300). The implementation of the « twelve measures » was confronted with serious difficulties. The teacher’s census required a prior work to compare existing database to MENFP’s database because each unit (DRH, DAEP, DPCE etc.), and even certain departmental directorates, collected data for their own needs. However, no coordinated information storage system was put in place. The BUNEX (the National Examinations Office) database was selected to receive survey forms, for they provided the most comprehensive schools database qualified to perform exams. Water and sanitation sector Progress has been noted with regards to the institutional framework reform of the National Directorate of Water and Sanitation (DINEPA). In order to strengthen DINEPA’s structures and management system, it was important to establish a Board of Directors. The 32 Government has appointed Board members and submitted it to Parliament for approval. Another aspect of DINEPA’s action is the development of a transparent monitoring system to contribute to the improvement of services provided in rural areas. To achieve this goal, its monitoring structure has been reinforced taking into account, besides the water quality aspects, nutrition, assets, recovery, service quality based on CAEPA/OP monitoring framework. Performance indicators are published on a monthly basis on DINEPA’s website. Capacity building of monitoring structures will continue and performance indicators will be integrated in the documents for approximately 230 delegate operators. 2. Pillar 1: Improve Public Financial Management Policy objective 1.1 – Improve cash management by the Treasury, enabling a better execution of sectoral budgets. Unification of the Treasury accounts is a prerequisite to restore budgetary discipline, as it allows, inter alia, the improvement of cash management. Cash management’s goal is to ensure the optimal execution of the State budget through liquidity management (cash), bank balances, revenues, expenditures, surpluses and deficits. This consists of managing short term cash flows and cash balances in order to ensure expenditures payments « along the way». An effective cash management is based on a careful planning of resources and of future expenditures and also on disbursements control. An optimal cash flow execution would avoid accumulation of arrears or unexpected expenditure cuts, etc. In order to achieve this goal, a cash management structure must be put in place. It will be managed by DGTCP through a Treasury Committee tasked with the preparation of a treasury plan which has two objectives: • A forecast objective: anticipate cash flows for the period under review. • An operational objective for the budget implementation: facts finding for the period under review, variance analysis and update for the following period. The Treasury organized several training workshops for administrators and authorizing officers using the blueprint for cash flow plans. Several line ministries submitted their expenditure plans to the DTDCP. This represents significant progress, for it will help the institution to better plan for disbursement requests, and thereby better harmonize requests with available resources. 33 3. Pillar 2 : Strengthening governance and service delivery in the Education Sector Policy Objective 2.1 – Improve MENFP regulation of non-public education services Beyond the universal, free and mandatory schooling program (PSUGO) that allowed more than 90% of children to have access to primary education, the Ministry of Education and Vocational Training (MENFP) has adopted twelve policy measures in the summer of 2015. This program includes the following: a teachers and school principals census, the granting of a Provisional Teaching Permit (PPE); a school census to allow the authorities to deliver an Identification plate to each school. To that we should add a continuing education program for teachers and a major school facilities construction program (almost 300). The implementation of the « twelve measures » was confronted with serious difficulties. The teacher’s census required a prior work to compare existing database to MENFP’s database because each unit (DRH, DAEP, DPCE etc.), and even certain departmental directorates, collected data for their own needs. However, no coordinated information storage system was put in place. The BUNEX (the National Examinations Office) database was selected to receive survey forms, for they had the most comprehensive database of schools qualified to perform exams. The distribution of the PPEs to teachers is still ongoing and was a specific condition for the European Union’s (EU) budget support. But the MENFP decided to prioritize reliability rather than quantity, for many PPEs already printed were inaccurate or didn’t correspond to a teacher actually in service. MENFP is focusing on the preparation and distribution of PPEs and hasn’t yet decided on measures to be taken to allow teachers who meet the definitive Teaching Permit (PE) criteria to obtain it within two years. Policy objective 2.2 – Increase transparency through timely and more widely accessible information on education service delivery The school census is still ongoing but faces many difficulties, even though the 2012-2013 and 2014-2015 yearbook data have been published in MENFP Statistic Directory. As for the teaching permit, the measures envisages a two-step process: issuance of a school Identification number and a permit to operate if reception conditions and pedagogical qualification are met within two years. Until now, 26,000 school facilities requested an ID (CIE) and many of them are actually being distributed. In 2015, the National Geospatial Information Center (CNIGS) received financial assistance from the Interamerican Development Bank (IDB) to carry out on behalf of the MENFP a digital mapping of the 17,000 school facilities integrated in MENFP database. A database of schools benefiting from the PSUGO exists already and can be found on the MENFP’s website supported by a geographical information system. However, the information collected is incomplete and not always accurate nor easy to read. In a few 34 years, the MENPF hopes to become independent from the CNIGS for geographic data collection. From now on, first grade students benefiting from the PSUGO should only enroll in public schools. This points to a strategic reorientation of both the PSUGO and the obligation to ensure basic education to all Haitian children. The Ministry continues to make every effort to implement this measure. School census data are available on the Ministry’s website. The Ministry has carried out geo-referencing of schools in 2015 and also the 2016 school census. Finally, the Information System for Education Management (SIGE), aimed at facilitating the treatment of data produced, is being implemented with the technical assistance of the EU under of the State Building Contract (SBC) program. This system will be based on the installation of an Intranet for MENFP linking departmental directorates of education (DDE) with the central level. To this end, the MENFP has created an Information System Unit (USI), whose main tasks will be: on the one hand, ensure a regular and coordinated monitoring of existing database – without eliminating them at the beginning - and on the other hand, ensure SIGE the development and management of SIGE, in order to guarantee the a good monitoring. Objective 2.3 – Improve the education budget’s alignment with sector priorities These targets have not been reached due to the country’s economic situation. The national budget amount allocated to this sector did not allow the sector to arrive at such distribution. Furthermore, the Ministry is advocating for a new deal on education comprised of 7 points. Point 2 concerns the financing of the education sector, which should double to reach 35% of the national budget, or 8% of the country’s Gross Domestic Product (GDP). Whichever is greater of the two would be allocated to Haiti’s public education funding. 4. Pillar 3: Strengthening governance and service delivery in the Water and Sanitation sector. 3.1 – Improve the oversight of the water and sanitation sector Prior action No. 5: The Government of Haiti, through the Primature, has submitted to the Senate for approval thereby the nomination of DINEPA’s Executive Board members so as to establish the management structure of DINEPA . Target: The DINEPA Executive Board approves DINEPA’s planned program of activities and annual budget. Prior action No. 6: The Government of Haiti, through DINEPA, has expanded its water sector oversight structure beyond water quality to include service quality and coverage, asset monitoring and revenue tracking in the CAEPA/OP monitoring framework. 35 Target: DINEPA, through URD and TEPAC, has collecte d and published on a monthly basis and since January 2015 a news bulletin on delegate operators’ performance (at least 230 CAEPA and OP by December 2015). As indicated in this report, DINEPA failed to meet the two objectives agreed upon under the program for the Water and Sanitation sector, and the reasons and constraints DINEPA faced are presented in the report, while some progress made is highlighted as well. Progress made on target No.6 was not stressed enough in the World Bank report. Information submitted by DINEPA to the World Bank during the program’s implementation period was not fully exploited (Progress reports No.1, 2, and 3 and final progress implementation report). The reports provided the following information as of December 31 st 2015: - The number of rural SAEPs integrated in the monitoring system for chlorine residual reached 174 instead of the 230 targeted. The last bulletin published by the National Observatory for Potable Water and Sanitation (ONEPA) is of November 2015. Even though the program execution was analyzed in this report, it should be noted that progress analyses carried out during the program’s execution period have not allow the adjustment of target No.6. Furthermore, external factors over which DINEPA has no control as well as risk factors were much more important than what was initially forecast during the preparation of the programs. The World Bank noted in its report that the lack of funding to the sector compared to commitment made at during negotiation of this budget support was detrimental to the program. This concerns particularly the water and sanitation sector, since work that was to be performed to address technical problems of the system was not executed. According to the World Bank report, the supervision of the program was a challenge and many problems encountered were not anticipated. DINEPA hopes that lessons learned from this experience will help improve future programs. 3.2 – Increase monitoring and transparency of water service availability and quality at the community level. On June 8 2015, DINEPA has notified the World Bank on activities to be implemented in and the needed budget (USD 6, 311,000.00) in order to accomplish target No. 6 by December 2015. The activities are essentially: 36 • Activity 1 – Have 230 Potable Water Supply Systems (SAEP) providing on a regular basis residual chlorine test results through the SISKLOR mechanism; • Activity 2 – publish on a monthly basis the SISKLOR bulletin. This information should not only be accessible from DINEPA’s website but also published in the regions (OREPA, URD, CAEPA/OP offices). I. Activity 1: Increase of the number of SAEP providing residual chlorine test results through SISKLOR mechanism on a regular basis. a. Overview of the SISKLOR mechanism and its operating system i. Presentation of the SISKLOR mechanism SISKLOR is a system that enables DINEPA to assess the quality of water service delivery by the SAEPs. It is based on residual chlorine analysis usually carried out by the operators ensuring the functioning of the SAEPs. Results are transmitted through text messages (SMS) to DINEPA which governs the potable water and sanitation (EPA) sector. Regarding the urban SAEP, the measures taken are implemented by the Technical Operation Centre (CTE). For the rural SAEP, this procedure is handled by members of the Potable Water and Sanitation Supply Committees (CAEPA) or by Potable Water and Sanitation Technicians (TEPAC). ii. Reporting Frequency Water chlorination analysis is based on two main aspects: • SAEP must hand over the reports ; • Report frequency must be appropriate meaning that: ≥ At least 20 days of reporting in a month represents a « Good » reporting ≥ At least 10 days of reporting in a month represents an « average » reporting, ≥ Less than 10 days of reporting in a month represents a « Bad » reporting. Fréquence: X ≥ 20 jours - Résultat bon 10 ≤ X ≤ 19 jours - Résultat moyen X < 10 jours- Résultat mauvais 37 iii. Chlorination analysis Only the SAEPs having a Good or Average reporting frequency are considered in the chlorination analysis. SAEPs with a bad reporting frequency are not incorporated in the analysis due to lack of data. Water chlorine analysis is based on the percentage of data indicating an optimal residual chlorine content between 0.5 and 1 mg/L. Water chlorination during the period under review presents the following results: - « Good » : if at least 80% of the data received indicates an optimal residual chlorine content « Average » : if the data received indicates an optimal residual chlorine content between 50% to 79% - « Incorrect » : if at least 50% of the data received indicates an optimal residual chlorine content - « Void» : if 100% of the data received indicates an absence of residual chlorine Résultats SISKLOR: x > 1.0 mg/l - Surchloration 0.5 ≤ x ≤ 1.0 mg/l - Chloration optimale 0.3 < x < 0.5 mg/l - Chloration faible 0.1 ≤ x ≤0.3 mg/l - Chloration très faible x = 0.0 mg/l - Chloration nulle ou non-effective b. Progress of activity 1– SISKLOR CAEPA i. Increase in the number of SAEPs integrated in the SISKLOR system In January 2015, the number of SAEPs integrated in SISKLOR system went from 159 to 163. But their number has not increased since for the following reasons: ≥ Some SAEPs at the national level face technical problems which should be addressed and corrected in order for them operational ; ≥ Necessary funds to carry out interventions to address the technical problems observed in some SAEPs are not available; ≥ The drought plaguing the country for a number of years had a much more negative impact on water resources this year. This has affected several SAEPs that could have been otherwise integrated in the SISKLOR system. 38 In the September 2015 report, DINEPA indicated the possibility of integrating 24 SAEPs in the SISKLOR system. DINEPA has identified a number of SAEPs to could be integrated in the system in 2015. If the political situation remains stable, 3 missions will be deployed across the country and forty two (42) new SAEPs will be integrated to the SISKLOR. By the end of this year, the number of SAEP will go from 163 to 205. ii. Increase in the number of SAEPs reporting regularly to the SISKLOR system During the month of September 2015, the number of SAEPs that reported their data to the SISKLOR system has dropped from thirty-two (32) to twenty-seven (27). SAEP’s management structures have to step up to make reporting systematic. Likewise, follow up from the decentralized structures of DINEPA must be strengthened. Regarding the lack of reporting, the following causes can be mentioned (non- exhaustive list): • The volunteering aspect of CAEPA members in charge of carrying out residual chlorine tests at different points along the control system (for the sample to be representative, it is necessary to choose several points, and often those that are far from one another). This leads to a loss of motivation for many CEAPAs; • Several systems are not functioning due to technical problems; • CAEPAs’ staff in charge of the SISKLOR system should be trained on how to send text messages (SMS) and on how to use of the SISKLOR kit; • Supply sources that dried up; • HTH stock shortage. This affects greatly the reporting frequency. This is not actually an issue for DINEPA has just purchased twenty-four (24) tons of HTH for SAEP chlorination process. A certain quantity has been delivered to the DINEPA decentralized structures. This first chlorine distribution will be presented on SISKLOR bulletin of October 2015. DINEPA can address some of the issues, but certain actions to be taken do not only depend on the institution. Concerning the « volunteering » issue of the CAEPAs, DINEPA plans to organize a brainstorming session on this topic during the month of December 2016. DINEPA counts on the World Bank support to organize this event and on other donors intervening in the water and Sanitation sector. With regard to the problems with water resources (water flow reduction or the drying up of water sources), DINEPA carried out a survey during the month of September and the beginning of November 2015 to assess the extent of the problem. The findings indicated that sixteen (16) of SAEPs exploited and integrated in the SISKLOR system are facing 39 problems of water resources (reduction of water flow or the drying up of water sources). The table below presents a list of SAEPs concerned by this problem: Table A3.1 : List of SAEPs integrated in the SISKLOR and facing water resources problems No. OREPA Departement Commune SAEP 1 Nord Nord’est Cap-Haitien Tanga 2 Carice Centre ville de Carice 3 Mombin-Crochu Centre ville 4 Ouanaminthe Gens de Nantes 5 Nord’ouest Ile de la Tortue Mapou 6 Anse-à-Foleur Dubois 7 Chansolme Massonière 8 Centre Artibonite Saint-Marc Banique 9 Ouest Ouest Anse-à-Galets Les Etroits 10 Arcahaie Ti Bois 11 Sud Sud’est Grand Gosier Bodarie 12 Nippes Asile La Source 13 Petit-Trou-de-Nippes Batardo 14 Sud Saint-Jean du Sud Carrefour Joute 15 Chardonnières Centre ville 16 Maniche Centre ville (La Digue) II. Activity 2 : Monthly publication of the SISKLOR bulletin With regard to this activity, beside internal publications and the ones presented on DINEPA’s website, the World Bank has requested that results be published in other offices in the region (OREPA 5 , URD 6 , and CAEPA/OP 7 ). The preparation and publication of the SISKLOR bulletin under the responsibility of the National Observatory for Potable Water and Sanitation (ONEPA). Even though this structure is being restructured, the bulletin is still being produced. DINEPA continues to collect on a daily basis data on the performance several CAEPAs and OPs. However, some bulletins are still not published and users are not informed on the services offered by DINEPA’s delegate operators. From the 163 SAEP integrated in SISKLOR system during February 2015, only 46 rural SAEP submitted reports and provided information on the performance of CAEPAs and OPs. DINEPA’s 5 OREPA: Regional Office of Potable Water and Sanitation. 6 URD: Rural Departmental Unit. 7 OP Professional Operator. 40 General Directorate is faced with many constraints that hinder its objectives to improve the monitoring and availability of water delivery service to the whole community, the most important being: 1) Lack of funding to carry out interventions on SAEPs facing technical problems affecting their integration to the SYSKLOR system. 2) Inexistence of telephone cards to send text messages, and technicians carrying out residual chlorine texts do not have the means to travel. 3) The extended drought throughout the country in several departments had a more negative impact on water resources this year. Several SAEPs were not integrated into the SISKLOR system due to this situation. 4) HTH stock shortage for chlorination process affects the reporting frequency. 5. Lessons learned from the program’s implementation Definitely, the reform process is at full steam. The program helped focus on several reforms and it should be stated that turnovers at the lead team levels had no negative impact on the reforms. The implementation of this program allowed to identify and address several constraints, namely: - Certain preconditions are vital to the success of the reforms ; - Predictability on budget resources are necessary to put in place the reforms ; - Importance of training, access to appropriate information to prepare the reforms, particularly the mastery new concepts and appropriate tools; - Importance of establishing organizational basis underlying the achievement of an objective. It should be noted that the lack of communication between those involved in the reform program prevented the line Ministries to take ownership of the program. Delivery capacity of the institutions and ministries and their logistic needs should be taken into account. Therefore, envisaging a capacity building effort for these institutions and ministries is necessary if programs of this sort are to succeed. It is highly recommended that there exists high-level political dialogue to implement the reforms. This will help put it all into perspective, reinforce ownership and facilitate the respect of implementation timetables. The World Bank has to take into account long-term financial strategies for the sustainable development objectives, meet funding requirements and balance expenditures between sub-sectors. 41 The Haitian Government and the World Bank should pay special attention to the reforms’ implications and public expenditures for income growth and poverty, and see how the budget support can complement other forms of financing. In light of past experience, it is important not to overload budget support programs with unrealistic objectives or with too many reform projects. In the future, it will be necessary to limit the scope of activities linked to the budget support. Certain risks should be considered, including political ones threatening the institutions’ capacity to implement long-term reforms. Budget support sustainability depends on capacity building of line ministries. The budget support assessment should be based on the medium-term overall performance. 42 Annex 4. Beneficiary Survey in the Ministry of Economy and Finance, in the Ministry of Education and Technical Training, and in the National Directorate of Potable Water and Sanitation (a) Methodological Approach of the Beneficiary Survey Introduction: As part of the ICR exercise, the ICR field mission (March 27-April 2, 2016) sought to obtain some feedback from the beneficiaries of the DPG reforms through a light survey. The team surveyed a random sample of 18 senior officers using a questionnaire (Annex 3) that was distributed to the survey participants with the explicit indication that their responses will be kept confidential and anonymous. Sample Size: Considering the similarity of reforms in the three Government services, a maximum threshold and pessimistic risk of non-response error equal to 50 percent of respondents was assumed. This leads to a sample size equal to 18 individuals, corresponding to a margin of error of 5 percent. To select a representative random sample of operation beneficiaries, a stratified sampling method was applied to ensure representative samples from each of the three government services. Drawing of Beneficiaries to be surveyed: Because of the specialized nature of the reforms, we conducted a stratified random sampling in each beneficiary government service. The questionnaire was presented to senior officers who had a knowledge of and played a role in the DPG implementation, but, the questionnaire was not distributed to all senior officers (a population of about 100 senior officers). This generated the matrix below showing the number of surveyed representatives from each institution and for each questionnaire theme (see table below). Table A4.1: Aggregated Results of the Beneficiary Survey Theme 1: Beneficiary support to the operation objectives (Share in % of total) Questions Total number of responses Yes No Other Did you understand and support the objective supported by this operation? 35 94% 3% 3% Was the objective an integral part of the missions of the department for which you are in charge or to which you belong? 35 74% 23% 3% Total 70 84% 13% 3% Theme 2: Beneficiary perception of the quality of Government’s internal supervision (Share in % of total) Questions Total number of responses Yes No Other Have you been informed of this policy objective in 2014? 18 78% 17% 5% Did you receive the information from your administrative hierarchy in charge of the indicator? 18 83% 17% 0 Did you know that the implementation of this measure was part of a reform program supported by the World Bank? 18 78% 17% 5% Total 54 80% 17% 3% Theme 3: Rating of beneficiary perception of key obstacles to the achievement of the specific objective of the operation (Share in % of total) 43 Questions Number of responses 1 2 3 4 5 Other (a) Determine and circle a grade level on a maximum of 5, to qualify your satisfaction with regard to the adequacy of the number of employees tasked with the mission to achieve the objective of the reform. 34 9% 12% 59% 18% 0 2% (b) Determine and circle a grade level on a maximum of 5, to qualify your satisfaction with regard to the quality and work experience of employees tasked with the mission to achieve objective of the reform. 34 3% 27% 35% 29% 3% 3% (c) Determine and circle à grade level on a maximum of 5, to qualify your satisfaction with regard to the level of collaboration within the Department to support this policy measure. 34 0% 24% 46% 26% 2% 2% (d) Determine and circle a grade level on a maximum of 5, to qualify your satisfaction with regard to the level of support from your hierarchy in the implementation of the policy reform measures. 34 3% 6% 32% 44% 12% 3% (e) Determine and circle a grade on a maximum of 5, to qualify your satisfaction with regard to the level of collaboration with other public administrations involved in the realization of this policy measure 34 12% 56% 23% 9% 0% 0% Total 170 5% 25% 39% 25% 4% 2% Theme 4: Beneficiary perception of the Bank performance (Share in % of total) Total number of responses Highly Satis- factory Satis- factory Modera tely Satis- factory Modera tely Unsatis factory Unsatis factory Highly Unsatis factory (a) Determine and circle your overall assessment of the intervention of the World Bank. 18 0% 44% 50% 6% 0% 0% (b) Determine and circle your assessment of the quality of supervision of the World Bank of this operation of reform 18 6% 50% 44% 0% 0% 0% Total 36 3% 47% 47% 3% 0% 0% Source: Source: Bank ICR mission. 44 (b) Table A4.2: Sample Distribution of survey participants Questionnaire Themes Ministry of finance Ministry of Education DINEPA Total Beneficiary support to the operation objectives 5 6 7 18 Beneficiary perception of the quality of Government’s internal supervision 5 6 7 18 Beneficiary perception of key obstacles to the achievement of the specific objectives of the operation 5 6 7 18 Beneficiary perception of the Bank performance 5 6 7 18 Source: Bank ICR mission. (a) Survey Questionnaire: Theme A4.1: Beneficiary support to the operation objectives Operation objective Improve the management of liquidity by the Directorate of the Treasury and the public accounting and enable better execution of sectoral budgets Question Did you understand and support the objective set out above? Was the objective an integral part of the missions of the department for which you were in charge or to which you belong? Response YES NO YES NO Theme A4.2: Beneficiary perception of the quality of Government’s internal supervision Operation objective Improve the management of liquidity by the Directorate of the Treasury and the public accounting and enable better execution of sectoral budgets Question Have you been informed of this policy objective in 2014? Did you receive the information from your administrative hierarchy in charge of the indicator? Did you know that the implementation of this measure was part of a reform program supported by the World Bank? Response YES NO YES NO YES NO 45 Theme A4.3: Beneficiary perception of key obstacles to the achievement of the specific objectives of the operation 8 Policy Objective 1 : Improve the management of liquidity by the Directorate of the Treasury and the public accounting and enable better execution of sectoral budgets Policy Objective 2: Improve the Ministry of Education and Vocational Training’s Regulation of Non- Public Education Services Policy Objective 3: - Increase transparency through timely and more widely accessible information on education service delivery Policy Objective 4 - Improve the Education budget alignment with sector priorities. Policy Objective 5 - Improve oversight of the water and sanitation sector. Policy Objective 6 - Increase monitoring and transparency of water service availability and quality at the community level. (a) Determine and circle a grade level on a maximum of 5, to qualify your satisfaction with regard to the adequacy of the number of employees tasked with the mission to achieve the objective of the reform. 1 2 3 4 5 (b) Determine and circle a grade level on a maximum of 5, to qualify your satisfaction with regard to the quality and work experience of employees tasked with the mission to achieve objective of the reform. 1 2 3 4 5 (c) Determine and circle a grade level on a maximum of 5, to qualify your satisfaction with regard to the level of collaboration within the Department to support this policy measure. 1 2 3 4 5 (d) Determine and circle a grade level on a maximum of 5, to qualify your satisfaction with regard to the level of support from your hierarchy in the implementation of the policy reform measures. 1 2 3 4 5 (e) Determine and circle a grade on a maximum of 5, to qualify your satisfaction with regard to the level of collaboration with other public administrations involved in the realization of this policy measure 1 2 3 4 5 8 The questionnaire related to each specific objective was presented to senior officers in each of the appropriate three Government institutions as follows: (i) Objective 1 questionnaire was responded by those in the Ministry of finance, (ii) Objective 2, 3 and 4 questionnaire were responded by those in the Ministry of Education and Vocational Training, and (iii) Objective 5 and 6 questionnaire were responded by those in the National Directorate of Potable Water and Sanitation. 46 Theme A4.4: Beneficiary perception of World Bank performance (% of total) (a) Determine and circle your overall assessment of the intervention of the World Bank. Highly Satisfactory Satisfactory Moderately Satisfactory Moderately Unsatisfactory Unsatisfactory Highly Unsatisfactory (b) Determine and circle your assessment of the quality of supervision of the World Bank of this operation of reform Highly Satisfactory Satisfactory Moderately Satisfactory Moderately Unsatisfactory Unsatisfactory 47 Annex 5. List of Supporting Documents 1. Program Document for the DPG: 84957-HT (June 25, 2014) 2. Grant Agreement for the DPO: TF017657 of June 9, 2014 3. The World Bank Group’s Interim Strategy Note for Haiti for FY13-14 of September 27, 2012 4. The World Bank Group’s Strategy for Haiti for FY15-17 5. The series of the DPG’s ISRs 6. Aide-Memoires for the March and June 2015 supervision missions 7. International Monetary Fund: The 2015 Staff Report for the Article IV consultation and request for a three-year arrangement under the extended Credit Facility, June 2015. 8. International Monetary Fund: The 2015 Staff Report for the Article IV consultation and request for a three-year arrangement under the extended Credit Facility- The Debt Sustainability Analysis May 2015. 9. The Haiti PEFA Report, January 2012 10. Haiti: Systematic Country Diagnostic, June 2015 11. Haiti: The Country Partnership Framework, August 2015 12. National recovery and Development Action plan for Haiti (PARDH) 48 Map of Haiti