Cholera Emergency Response Project

Document of The World Bank Report No: 89201- HT IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H639- 0-HT) ON A GRANT IN THE AMOUNT OF SDR 9.9 MILLION (US$15 MILLION EQUIVALENT) TO THE REPUBLIC OF HAITI FOR A CHOLERA EMERGENCY RESPONSE PROJECT September 17, 2014 Health, Nutrition and Population Global Practice Haiti Country Management Unit Latin America and the Caribbean Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized CURRENCY EQUIVALENTS (Exchange Rate Effective: September 17, 2014) Currency Unit = Haitian Gourdes $1.00 = US$0.02 US$1.00 = $44.45 FISCAL YEAR October 1 – September 30 ABBREVIATIONS AND ACRONYMS CBR Cost-Benefit Ratio CDC Center for Disease Control CFR Case Fatality Rate CRD Center of Research and Development (Centre de Recherche et de Développement) DALY Disability-Adjusted Life Years Lost DELR Direction of Epidemiology, Laboratory and Research (Direction d’Epidémiologie, de Laboratoire et de Recherche) DINEPA National Direction of Water and Sanitation (Direction National de l’Eau et de l’Assainissement) FAES Economic and Social Assistance Funds (Fonds d’Assistance Economique et Sociale) FY Fiscal Year GDP Gross Domestic Product GOH Government of Haiti ICR Implementation Completion and Results Report IDA International Development Association IDB Inter-American Development Bank IFR Interim Financial Report ISR Implementation Status and Results Report LNSP National Laboratory of Public Health (Laboratoire National de Santé Publique) M&E Monitoring and Evaluation MSPP Ministry of Public Health and Population (Ministère de la Santé Publique et de la Population) NGOs Non-Governmental Organizations ORS Oral Rehydration Salt PAHO Pan-American Health Organization PDO Project Development Objectives PRUC Cholera Emergency Response Project (Project de Réponse D’Urgence Au Choléra) TEPAC Technicians on water and sanitation UNICEF United Nations Children's Fund UPMC University of Pittsburgh Medical Center WHO World Health Organization Vice President: Jorge Familiar Special Envoy for Haiti: Mary Barton-Dock Practice Manager: Enis Barış Project Team Leader: Eleonora Cavagnero ICR Team Leader: Claudia Macías ICR Primary Author: Claudia Macías and Voahirana Rajoela ii HAITI Cholera Emergency Response Project CONTENTS Data Sheet A. Basic Information .......................................................................................................... iv B. Key Dates ...................................................................................................................... iv C. Ratings Summary .......................................................................................................... iv D. Sector and Theme Codes ................................................................................................ v E. Bank Staff ....................................................................................................................... v F. Results Framework Analysis .......................................................................................... v G. Ratings of Project Performance in ISRs ....................................................................... ix H. Restructuring ................................................................................................................. ix I. Disbursement Profile ..................................................................................................... ix 1. Project Context, Development Objectives and Design ............................................ 1 2. Key Factors Affecting Implementation and Outcomes ........................................... 4 3. Assessment of Outcomes ....................................................................................... 12 4. Assessment of Risk to Development Outcome ...................................................... 17 5. Assessment of Bank and Borrower Performance .................................................. 18 6. Lessons Learned..................................................................................................... 20 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........ 21 Annex 1. Project Costs and Financing .............................................................................. 22 Annex 2. Outputs by Component...................................................................................... 23 Annex 3. Economic and Financial Analysis ..................................................................... 25 Annex 4. Bank Lending and Implementation Support/Supervision Processes ................. 32 Annex 5. Beneficiary Survey Results ............................................................................... 34 Annex 6. Stakeholder Workshop Report and Results ....................................................... 35 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 36 Comments from Recipient ................................................................................................ 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 48 Annex 9. List of Supporting Documents .......................................................................... 49 Annex 10. Project contribution to the reduction of cholera incidence and mortality ....... 51 Annex 11. MAP IBRD 33471R ....................................................................................... 53 iii Data Sheet A. Basic Information Country: Haiti Project Name: Cholera Emergency Response Project Project ID: P120110 L/C/TF Number(s): IDA-H6390 ICR Date: 09/04/2014 ICR Type: Core ICR Lending Instrument: ERL Recipient: HAITI Original Total Commitment: XDR 9.90M Disbursed Amount: XDR 9.23M Revised Amount: XDR 9.90M Environmental Category: B Implementing Agency: Economic and Social Assistance Funds (Fonds d’Assistance Economique et Sociale -- FAES) Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 11/19/2010 Effectiveness: 04/20/2011 02/07/2011 Appraisal: 12/03/2010 Restructuring(s): 06/04/2013 12/27/2013 Approval: 01/18/2011 Mid-term Review: Closing: 06/30/2013 12/31/2013 03/30/2014 C. Ratings Summary C.1 Performance Rating by Implementation Completion and Results Report (ICR) Outcomes: Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Quality of Supervision: Satisfactory Implementing Agency/Agencies: Moderately Satisfactory Overall Bank Performance: Moderately Satisfactory Overall Borrower Performance: Moderately Satisfactory iv C.3 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating Potential Problem Project at any time (Yes/No): Yes Quality at Entry (QEA): None Problem Project at any time (Yes/No): No Quality of Supervision (QSA): None DO rating before Closing/Inactive status: Moderately Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 73 73 Public administration- Health 27 27 Theme Code (as % of total Bank financing) Health system performance 13 13 Other communicable diseases 87 87 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Pamela Cox Country Director: Mary Barton-Dock Alexandre V. Abrantes Practice Manager: Enis Barış Joana Godinho Project Team Leader: Eleonora Del Valle Cavagnero Maryanne Sharp ICR Team Leader: Claudia Macias ICR Primary Author: Voahirana H. Rajoela F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the Project is to improve the health and hygiene practices in order to reduce the spread of cholera and strengthen the Recipient’s institutional capacity to respond to outbreaks. Revised Project Development Objectives (PDO) (as approved by original approving authority) The PDO remained unchanged during the Project life. v (a) PDO Indicator(s) Indicator Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Indicator 1 : Percentage of population in Project intervention areas who know A) the cholera early warning symptoms and B) prevention steps. Value (quantitative or qualitative) Rapid Survey to be done TBD by Department A): 69 B): 18 A): 82 B): 14 Date achieved 22-Dec-2010 30-June- 2013 30-June-2013 30-June-2013 Comments (incl. % achievement) Partially achieved. Baseline data was available for four Departments in Sept. 2011 and for three Departments in Aug. 2012. Final baseline, defined as Department average, was available in June 2013 - A) 64, B) 13. See section 2.3 for explanation. Indicator 2 : Percentage of population in Project intervention areas with access to improved water sources. Value quantitative or Qualitative) Rapid Survey to be done TBD by Department 78 79 Date achieved 22-Dec-2010 30-June-2013 30-June-2013 30-June-2013 Comments (incl. % achievement) Achieved. Baseline data was available for four Departments in Sept. 2011 and for three Departments in Aug. 2012. Final baseline, defined as Department average, was available in June 2013. See section 2.3 for explanation. Indicator 3 : Increased institutional capacity as measured by achievement of at least four out of five of Component 2 indicators on an annual basis. Value quantitative or Qualitative) N/A Four out of five Component 2 indicators achieved Four out of five Component 2 indicators achieved Date achieved 22-Dec-2010 30-June-2013 30-June-2013 Comments (incl. % achievement) Achieved. Source of data: Project Reports. See section 3.2 for further details. vi (b) Intermediate Outcome Indicator(s) Indicator Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Indicator 1 : Number of communities that have received targeted health education and/or prevention activities. Value quantitative or Qualitative) 0 2,214 4,447 Date achieved 22- Dec- 2010 30-June-2013 30-June-2013 Comments (incl. % achievement) Surpassed. "Localities" were used instead of communities, given the Haitian context. Source of data: Project Reports. Indicator 2 : Number of health centers and posts, treatment and chlorination centers and units, and oral rehydration posts supported with supplies. Value (quantitative or qualitative) 0 223 256 Date achieved 22- Dec- 2010 30-June- 2013 30-June-2013 Comments (incl. % achievement) Surpassed. Source of data: Project Reports. Indicator 3 : Number of people receiving water treatment products and soap. Value quantitative or Qualitative) 0 178,400 856,688 Date achieved 22- Dec- 2010 30-June- 2013 30-June-2013 Comments (incl. % achievement) Surpassed. Source of data: Project Reports. Indicator 4 : Number of personnel and community workers trained. Value quantitative or Qualitative) 0 2,633 5,571 Date achieved 22- Dec-2010 30-June- 2013 30-June-2013 Comments (incl. % achievement) Surpassed. Source of data: Project Reports. vii Indicator Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Indicator 5 : Departmental Cholera Management Plan developed (by MSPP and DINEPA) for at least five departments. Value quantitative or Qualitative) N/A 5 6 Date achieved 22-Dec-2010 30-June- 2013 30-June-2013 Comments (incl. % achievement) Surpassed. Source of data: Project Reports. Indicator 6 : Establishment and functioning of one national and five regional Cholera and Emergency Response Units. Value quantitative or Qualitative) N/A 6 7 Date achieved 22-Dec-2010 30-June- 2013 30-June-2013 Comments (incl. % achievement) Surpassed. This indicator reflects the number of coordinators recruited; one national cholera focal point and 6 departmental cholera coordinators. Source of data: Project Reports. Indicator 7 : Quarterly project progress reports submitted to the Cholera Coordination Committee and to the Bank. Value quantitative or Qualitative) N/A Reports submitted Reports submitted Date achieved 22-Dec-2010 30-June- 2013 30-June-2013 Comments (incl. % achievement) Achieved. Quarterly service providers (NGOs, MSPP and DINEPA) progress reports were considered as Project progress reports. At least, 10 quarterly Project progress reports should have been submitted during the Project implementation period. Indicator 8 : Good quality quarterly Interim Financial Report (IFRs) are submitted to the Bank in a timely manner. Value quantitative or Qualitative) N/A IFR submitted IFR submitted Date achieved 22-Dec-2010 30-June- 2013 30-June-2013 Comments (incl. % achievement) Not achieved. From Effectiveness to Project closing date, 10 IFRs should have been submitted. At the end of the Project, 8 IFRs were submitted. viii G. Ratings of Project Performance in ISRs No. Date ISR Archived Development Objective (DO) Implementation Progress (IP) Actual Disbursements (USD millions) 1 02/23/2011 Satisfactory Satisfactory 0.00 2 08/09/2011 Satisfactory Satisfactory 3.00 3 03/28/2012 Moderately Satisfactory Moderately Satisfactory 6.63 4 11/09/2012 Moderately Satisfactory Moderately Satisfactory 9.91 5 05/29/2013 Moderately Satisfactory Moderately Satisfactory 13.10 6 12/27/2013 Moderately Satisfactory Moderately Satisfactory 14.27 H. Restructuring Restructuring Date(s) Board Approved PDO Change ISR Ratings at Restructuring Amount Disbursed at Restructuring in USD millions Reason for Restructuring & Key Changes Made DO IP 06/04/2013 MS MS 13.10 Six-month extension of closing date to allow implementation of remaining activities. 12/27/2013 MS MS 14.27 Three-month extension of closing date to allow finalization of remaining activities. I. Disbursement Profile ix 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. At the time of appraisal Haiti remained the poorest country in the Latin America and the Caribbean region and one of the poorest in the world, with a GDP per capita of US$673. Half of its 10 million population lived in absolute poverty (less than US$1 per day); and four-fifths lived on less than US$2 per day. Multidimensional poverty was far-reaching as evidenced by poor social indicators such as literacy, life expectancy, infant and maternal mortality. Haiti ranked 148 out of 179 in the United Nations Human Development Index in 2008. 1 2. Haiti was strongly impacted by the January 12, 2010 earthquake that struck its capital and nearby towns. The earthquake was the largest urban disaster in recorded history. It killed over 220,000 people, injured another 300,000, and displaced approximately 1.5 million, and is estimated to have wiped out a decade in poverty gains. Housing, infrastructure, public buildings, and businesses sustained enormous damage. The March 2010 Post-Disaster Needs Assessment evaluated damages and losses at US$7.9 billion or 120 percent of GDP, and reconstruction needs at US$11.3 billion. On November 5, 2010, hurricane Thomas hit Haiti causing heavy flooding, landslides, infrastructure destruction and the loss of crops in west of the city, which worsened the already precarious living conditions. 3. Ten months after the devastating earthquake, a severe outbreak of cholera was confirmed in Haiti. Between October and December 2010, 2,481 deaths and 60,644 hospitalizations from cholera were confirmed nationwide . It was the first time cholera - a diarrheal disease associated with the consumption of food and water contaminated by feces infected with the bacterium vibrio cholerae - had been identified in the country in at least 100 years. 2 Within a month of the initial report, cholera had spread to all regions of Haiti and the Dominican Republic, with which Haiti shares the island of Hispaniola. The in-hospital case fatality rate 3 in Haiti was estimated at 3.2 percent and the cholera specific mortality rate at 14.16 per 100,000 inhabitants. All ten of Haiti’s departments had confirmed cases of cholera. Living conditions, particularly those in the camps and in slum communities, made the country extremely vulnerable to the spread of cholera: (i) more than 30 percent of the population lacked access to potable water; (ii) more than 80 percent had no access to latrines; 4 and (iii) limited health knowledge and often low access to health care had worsened the situation. The already fragile public health system was further weakened by the earthquake making it more challenging to treat patients with health providers who lacked experience in managing cholera cases. These conditions posed the risk of significant mortality and cholera becoming endemic. 1 Human Development Report, United Nations Development Program , 2009. 2 Katherine E. Bliss and Matt Fisher. Water and Sanitation in the T ime of Cholera. Center for Strategic and International Studies, September 2013. 3 [(Number of deaths/number of cases) x 100] 4 World Health Organization (WHO)/United Nations Children’s Fund (UNICEF) Joint Monitoring Program, 2012. 1 4. The Government of Haiti (GOH), in partnership with a number of agencies, created an Emergency Cholera Coordination Committee at the central level and corresponding committees at the departmental and commune levels to confront the situation. In addition, an Inter-Cluster coordination system was put in place (Health, Water Sanitation and Hygiene, Camp Coordination and Camp management and Logistics Clusters) to ensure a coordinated and rapid response on the ground. Moreover, the Government prepared a National Strategy for the Response to the Cholera Epidemic, including the health and water and sanitation response through the Cholera Inter-Sector Response Strategy estimated at US$164 million. Th e latter aimed at reducing avoidable mortality and morbidity by limiting the impact of the cholera outbreak. Improving awareness and hygiene practices were key to control and reduce cholera in Haiti. 5. As part of Haiti’s call to the international community for support, the GOH requested the World Bank’s assistance for its emergency response to the cholera epidemic. In just a few months, after the confirmation of the outbreak, the World Bank responded in record time by making a US$15 million equivalent grant available through a fast track Cholera Emergency Response Project aimed at supporting the GOH’s response to the outbreak and strengthening its capacity to manage this type of epidemics. This Project was the first World Bank operation in the health sector in over a decade. At the time, the World Bank was already supporting the GOH’s cholera prevention efforts through its Global Facility for Disaster Reduction and Recovery with a US$200,000 grant to assist the awareness raising and prevention program led by the Ministry of Public Health and Population (Ministère de la Santé Publique et de la Population - MSPP). Since the start of the cholera outbreak, the World Bank was also helping the Directorate of Civil Protection to coordinate, together with MSPP , the response of the Government and its partners. 6. The Project was fully consistent with the World Bank Group’s Country Assistance Strategy FY09- 12 (Report No. 48284- HT) discussed by the Executive Directors on June 2, 2009 and the 2012 Interim Strategy Note. The principal objectives of these were to invest in human capital and reduce vulnerability. Indeed, the four pillars laid out in the Interim Strategy Note are: (i) r educing vulnerability and increasing resilience; (ii) sustainable reconstruction in housing and electricity; (iii) b uilding human capital; and (iv) r evitalizing the economy through agriculture, community-driven development and private sector development. The pillars have a cross-cutting theme of strengthening governance and capacity. 1.2 Original PDO and Key Indicators (as approved) 7. The PDO was to improve the health and hygiene practices in order to reduce the spread of cholera and strengthen the institutional capacity to respond to outbreaks. Progress on achievement of the objectives of the Project was to be measured through three outcome indicators and eight intermediate result indicators. The Project’s outcome indicators were: (i) Percentage of population in Project intervention areas who know the cholera early warning symptoms and prevention steps; (ii) Percentage of population in P roject intervention areas with access to improved water sources; and (iii) Increased institutional capacity as measured by achievement of at least four out of five of Component 2 indicators on an annual basis. 2 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification. 8. The development objective was not revised. Although the formal restructuring processes did not include changes in the R esults Framework, clarifications on Project’s outcome indicators No. 1 and No. 2, and changes to their respective baselines and targets were agreed during Project implementation as documented in the Project’s supervision documents. Such implementation changes have been explained under Section 2.3. 1.4 Main Beneficiaries 9. The Project direct beneficiaries included over three million people in 4,447 localities across seven D epartments that received treatment, education, and training on proper hygiene and water treatment for the prevention of cholera. 5 The Project targeted populations affected by the cholera epidemic and vulnerable groups living in camps (homeless people from the earthquake that struck Haiti in January 2010). It targeted seven of the 10 departments in Haiti (Artibonite, Centre, Ouest - in selected parts of Port Au Prince, Sud- Est, Nippes, Sud, Grand- Anse). The seven departments of the Project represent 73 percent of the total population with an estimated number of 7.4 million inhabitants. 6 The Project provided oral rehydration salt (ORS) to 121,895 people and water treatment products and soap to 850,000 people. The Project’s health and hygiene and promotion campaigns reached around 80 percent of the population in 4,447 localities. Over 5,500 community health and hygiene agents traveled to over 1,600 communities across the seven D epartments, including the most remote areas. At the time of Project preparation, other World Bank partners were actively responding to emergency needs, including awareness-raising activities and distribution of chlorine tablets in the Nord and Ouest departments. 1.5 Original Components 10. The Project consisted of the following two components (see Annex 2). 11. Component 1: Support to the Government’s Response to Cholera at the Decentralized Level (appraisal: US$11.0 million; actual : US$11.0 million). The objective of this component was to carry out a program of activities for immediate response to cholera, as outlined in the National Strategy for the Response to the Cholera Epidemic. This component sought to respond to the needs at the departmental level through support to all service providers, whether public, mixed or private, undertaking interventions at all levels of the cholera response. This component included two sub- components: (i) Sub- component 1.1: Support to a Multi- sectoral Approach to Public Primary Health Care; and (ii) Sub-component 1.2: Support to the Health, Water and Sanitations Response of Non- Public Providers. 12. Component 2: Emergency Response Capacity Building (appraisal: US$4.0 million; actual: US$3.9 million). This component financed activities to support the successful implementation of Component 1 and to ensure that MSPP and National Directorate of Potable 5 See Table 3.1: Population at departmental level and in localities of intervention. 6 Haitian Institute of Statistics and Informatics, Population estimate in 2012. 3 water and sanitation (Direction Nationale de l’Eau Potable et de l’Assainissement – DINEPA) were prepared to tackle cholera and other possible water-borne diseases. As such, this component supported a number of emergency response capacity building activities for MSPP and DINEPA as well as overall Project management and supervision through two sub- components: (i) Sub- component 2.1: Strengthening Government’s capacity to manage and respond to outbreaks; and (ii) Sub-component 2.2: Carrying out a program of activities to support the management of Project, including monitoring, supervision and audits. 1.6 Revised Components 13. The Project’s components were not revised during implementation. 1.7 Other significant changes 14. The Project had two level II restructurings. The first restructuring , approved on June 4, 2013, extended the Project closing date from June 30, 2013 to December 31, 2013 to allow the GOH sufficient time to implement the remaining activities such as the rehabilitation of rooms for infectious and diarrheal diseases treatment; the installation of incinerators for proper waste management; and additional support for epidemiological surveillance. 15. The second restructuring, approved on December 27, 2013, extended the closing date for the second time from December 31, 2013 to March 30, 2014 to ensure the c ompletion of Project activities, specifically the construction of three ecological sanitary blocks and the delivery of one ambulance-boat. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 16. Project preparation and design were appropriately responsive to help the GOH respond quickly and effectively to damage caused by the cholera outbreak . The Project’s activities responded to the needs of the country and supported the Government’s National Response Strategy to the Cholera Epidemic. D espite the difficult conditions and because of an adequate commitment from the Government’s and the World Bank’s team, Project preparation was expeditious - about three months. 7 The preparation of the Project benefited from broad consultations with sector stakeholders such as MSPP, United Nations agencies, and selected implementing partners. 8 The GOH requested that the operation be financed from its International Development Association (IDA) allocation on grant terms. An alternative instrument was not considered given the crisis nature of the situation and the fact that the Project met the criteria defined under emergency operations. Funds were programmed based on the rapidly evolving situation and guided by the Government’s assessment of gaps and pressing needs. 7 The Concept Note review meeting was held in November 19, 2010, just a month after the cholera epidemic was declared, and the Project became effective on February 7, 2011. 8 Since then, the World Bank has built a strong partnership with key development partners in Haiti’s health, social protection, and water and sanitation sectors, including the US Government Center for Disease Control (CDC) and United States Agency for International Development (USAID), the Pan-American Health Organization (PAHO), the IDB, and Médecin Sans Frontiére (MSF). 4 17. Project design incorporated lessons learned from previous operations in Haiti, t he World Bank experience with emergency response operations and international good practices to cholera response. The most relevant were: (i) Since cholera has a "fecal-oral" transmission and 88 percent of cases of diarrhea worldwide are attributable to unsafe water, insufficient hygiene, and inadequate sanitation, 9 a multisectoral approach based on prevention, preparedness and response, along with an efficient surveillance system, is key to mitigating cholera outbreaks, controlling cholera in endemic areas and reducing deaths . (ii) Projects with simple project objectives and design are easier to manage in emergency operations. Complex multi-sector operations are difficult to implement in low capacity environments. (iii) Close coordination with government agencies and stakeholders is essential to manage well intentioned, yet often disparate efforts of these groups. (iv) Linking emergency operations with previous and future technical assistance is essential to increase, inter alia , their sustainability. (v) Capacity building activities are crucial to foster government’s emergency response to manage and respond to outbreaks. 18. Given the high cholera case fatality rate, the Project focused mainly on emergency response by improving: (i) the population’s access to ORS and health care services; (ii) the use of safe water sources; (iii) the hygiene practices, such as hand washing and removal of excreta and awareness of the early symptoms of the disease; (iv) epidemiological surveillance; and (v) coordination between DINEPA and MSPP to effectively coordinate the response. 19. A number of risks included in the risk matrix did materialize during Project implementation, while others that were omitted proved to be relevant later. Among the risks that materialized was the Government’s limited capacity to respond rapidly. As a result, initial roll-out of emergency activities was implemented by three Non -Governmental Organizations (NGO) - Partners in Health/Zanmi Lasante, World Vision, and Save the Children, in Port Au Prince, Artibonite, Centre, and Sud Es t Departments. The contracting of these NGOs to work at the community level was essential to the behavior change objectives and the expected uptake of prevention and treatment interventions. As the capacity of the Government was strengthened, departmental health authorities also contributed to the large-scale prevention and treatment efforts in these departments, while also expanding activities to include Grand Anse, Nippes, and Sud Departments. The severity of some risks was underestimated during preparation. For instance, although baseline data for two out of three outcome indicators were not available during preparation, the risk matrix did not include possible problems associated with this lack of information. 9 http://www.cdc.gov/healthywater/global/diarrhea-burden.html. 5 2.2 Implementation 20. Overall implementation was moderately satisfactory throughout Project life. Implementation progress was originally slow due to delays in the completion of preparatory activities with the Economic and Social Assistance Funds (Fonds d’Assistance Economique et Sociale – FAES), NGO service providers, and DINEPA. Intensive World Bank supervision and increased monitoring, coordination, and communication with all involved institutions helped to promote implementation progress. Diffi cult access to localities due to their remoteness and the lack of roads posed a considerable challenge for the implementation of Project activities. High rotation of staff (five different General Directors in FAES during Project implementation) and heavy administrative procedures at the management level of FAES delayed the implementation of Project activities. The regular update and submission of procurement plans helped to improve implementation of activities and disbursement rates , although extension of the closing date twice was needed to allow full completion of programmed activities. Despite the delays, most of planned activities were finalized. The fiduciary procedures of the GOH and the World Bank followed regular (non- emergency) processes, particularly on procurement and thus lacked the flexibility needed in emergency situations. 21. The Project was implemented by FAES, in close coordination with other development partners and NGOs at a decentralized level. At the time of preparation, FAES was considered the best agency for assuming overall Project coordination given its experience with Bank-financed projects. In addition, the Project was conceived to be managed by FAES on behalf of and in close partnership with MSPP and DINEPA. However, MSPP started activities at a later stage due to the fact that, at the time of Project preparation, its efforts were not only focused on the cholera epidemic but on the remaining work from the January 2010 earthquake. This fact made difficult the relationship between the MSPP and FAES and thus the implementation of Project activities. It was not until the end of the first year of the P roject that FAES was able to benefit from the collaboration of MSPP officials for the recruitment of institutional and community staff. Partners included mainly US Government Center for Disease Control (CDC), the Pan- American Health Organization (PAHO), United Nations Children's Fund (UNICEF), the Inter-American Development Bank (IDB), and Doctors without Frontiers (Médecins Sans Frontières) among others. As mentioned before, the initial phase of Project implementation was carried out by three NGOs hired to this end, which gave time for the Departmental health authorities to develop clear and prioritized plans on how to fight cholera at their level. 22. Project implementation at the community level was successful despite operational challenges. Community health and hygiene agents, including those working in very remote areas, were able to: (i) disseminate messages on proper hygiene and water treatment for cholera prevention; (ii) teach people to identify the cholera early symptoms; (iii) demonstrate the use of first-line cholera treatment products, such as O RS ; and (iv) distribute water treatment products like chlorine tablets. However, the delivery of services at the community level was negatively affected by several factors, namely: (i) limited availability of skilled community agents given the large amount of them required (more than 2,000); (ii) quality and duration of the training; (iii) difficulties in the coordination between FAES and MSPP - FAES was responsible for hiring the community health and hygiene agents and purchasing necessary goods, while MSPP was responsible for making sure that the plans and activities were executed timely and properly; and 6 (iv) difficulties in the communication between the central and decentralized levels (messages usually took a long time to descend from higher to lower levels and vice versa). 23. Frequent World Bank supervision and interaction with Project partners and stakeholders proved crucial for implementation progress. Problems encountered in Project implementation were addressed during regular implementation support meetings and bi -monthly supervision missions. Despite significant difficulties, the Project executed nearly US$14.9 million ( 99.6 percent of total grant amount), and achieved concrete and important results. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 24. The M&E framework was well fitted to measure the achievement of Project outcomes and was aligned with the GOH strategic priorities. The M&E framework included the following: (i) rapid baseline and ex -post surveys to be conducted (with representativeness at the departmental level) during Project implementation; (ii) routine data collection at the community level and monitored at the departmental level; and (iii) standard quarterly and annual reports. Due to the emergency nature of the operation, some definitions were delayed to Project implementation. Detailed monitoring and evaluation arrangements were outlined in the Project monitoring and evaluation Manual (December 2012), including roles and responsibilities, monitoring and supervision steps, timing and tools. Overall, the M&E framework allowed for the existing systems to be utilized and strengthened, however, it was much more difficult to implement than initially expected. 25. The M&E system was not sufficiently used as a management tool or to inform decision-making because reliable data to define baseline and ex -post target for two of the three PDO indicators was obtained at a very late stage of implementation. A two -phase rapid survey was planned for establishing baseline and ex -post for PDO indicator No. 1 (percentage of population in Project intervention areas who know cholera early warning symptoms and prevention steps), and for PDO indicator No. 2 (p ercentage of population in Project intervention areas with access to improved water sources). The first phase data was collected in four D epartments before planning and implementation of activities by NGOs service providers (in June 2011), 10 and the second phase before those by MSPP and DINEPA (in March 2012). 11 The baseline for the Project intervention by Department was first reflected in the Aide Memoire of August 2012 (18 months after Project effectiveness). Agreement was reached with FAES to set the ex -post target at five percent increase from baseline. 26. Although the survey design and sampling methodology were agreed with FAES and the World Bank, surveys had to be harmonized after the first report was available in June 2013. The sampling methodology adopted for the baseline survey was intended to provide sufficient information on the first two Project outcome indicators in terms of the robustness of the estimates both at an aggregate level, as well dis-aggregated by each D epartment of intervention. In addition, proposed sampling took into account the possibility to assess the impact of the Project through comparison with non- targeted departments. However, 10 The NGOs Partners in Health/Zanmi Lasante, Save the Children and World Vision collected the data in the departments of Centre, Artibonite, Ouest and Sud-est. 11 The Center of Research and Development (Centre de Recherche et de Développement -- CRD) collected data in Nippes, Sud and Grand Anse. 7 oversampling and weighting problems with phase 2 D epartments raised concerns regarding comparison across Departments and between baseline and ex -post data. Appropriate corrections were made to both baseline and ex-post data for phase 2 Departments. Consequently, although original baseline and ex -post target for PDO indicator No. 1 and PDO indicator N o. 2 were to be determined by Department, it was later decided to define them as an average of the seven Departments covered by the Project (Harmonization Report 2013) . This change was not made through a formal Project restructuring . 27. In addition to adjustments in the baseline and ex-post target of the above mentioned PDO indicators, clarifications on how to measure them were also made during supervision. “Percentage of population in Project intervention areas who know the cholera early warning symptoms and prevention steps” would mean observing and reporting diarrhea with rice water color and vomiting (as symptoms), and adopting and reporting four out of six means of prevention (as prevention steps). “Percentage of population in Project intervention areas with access to improved water sources” would mean regular access to clean chlorinated water. These clarifications and changes were documented through Aide Memoires (August 29 to September 9, 2011 and August 21 to 30, 2012), an Implementation Status and Results Report (ISR sequence 3). Table 1: Project Outcome Indicators – baselines and ex -posts Project Development Outcome Indicators Action taken during implementation Baseline (Original) Baseline (Defined during implementation) Cumulative Target Value (Original) Cumulative Target Value (Defined during implementation) Project Development Objective: To improve the health and hygiene practices in order to reduce the spread of cholera and strengthen the institutional capacity to respond to outbreaks. 1. Percentage of population in Project intervention areas who know A) the cholera early warning symptoms and B) prevention steps. Baseline and target established; Definition of indicator clarified Rapid Survey to be done 1 A: 64% 2 B: 13% 2 TBD by dept 1 A: 69% 2 B: 18% 2 2. Percentage of population in Project intervention areas with access to improved water sources. Baseline and target established; Definition of indicator clarified Rapid Survey to be done 1 78% 2 TBD by dept 1 78% 2 8 Project Development Outcome Indicators Action taken during implementation Baseline (Original) Baseline (Defined during implementation) Cumulative Target Value (Original) Cumulative Target Value (Defined during implementation) 3. Increased institutional capacity as measured by achievement of at least four out of five of Component 2 indicators on an annual basis. N/A N/A Four out of five indicators achieved Source: 1 Emergency Project Paper Results Framework, 2010; 2 Center of Research for Development harmonization report (average of seven departments covered by the Project), 2013. 28. Information for the intermediate outcome indicators was collected through the Project reports to allow the monitoring of the implementation progress in a quarterly basis. Baseline and ex-post target, set in December 2010 remained the same for five of the eight intermediate outcome indicators. Intermediate outcome indicator No. 2 (number of health centers and posts, treatment and chlorination centers and units, and oral rehydration posts supported with supplies) and No. 4 (number of personnel and community workers trained (by type of training)) and No. 6 (establishment and functioning of one national and five regional cholera and Emergency Response Units) were subject to some changes . In 2013, the cholera response strategy was changed by the MSPP. The new strategy no longer involved the use of permanent stand-alone cholera rehydration posts but rather the integration of the cholera treatment into the existing health system. Consequently, the target for these indicators was revised according to the implementation plan submitted by the Government and agreed with the World Bank team. 29. The Project’s activities contributed to achieving Project’s intermediate results effectively and in turn these contributed to the desired outcomes. Reducing the spread of cholera involves improved health and hygiene practices. This was a result of the implementation of community and sensitization and case management activities, carried out by the NGOs service providers and the decentralized departments of the MSPP under Component 1. The intermediate indicators of this component measured progress towards the Project’s first sub-objective, whereas the strengthening of institutional capacity was a result of the implementation of activities under Component 2. The latter w as directly related to the strengthening of DINEPA, Direction of Epidemiology, Laboratory and Research (Direction d’Epidémiologie, de Laboratoire et de Recherche - DELR), National Laboratory of Public Health (Laboratoire National de Santé Publique - LNSP) and FAES. The intermediate indicators of Component 2 measured effectively progress towards the second sub-objective. 9 2.4 Safeguard and Fiduciary Compliance 30. Environmental Safeguard. The P roject was classified as category B and triggered the Environmental Safeguard (OP 4.01) as a result of the increase of health care waste production due to the implementation of Component 1 and the inclusion of minor works. Component 1 included the implementation of activities such as management of healthcare waste within and from healthcare facilities, worker health and safety, and the inclusion of minor works. About 250 health facilities benefited from technical and financial support for waste management and environmental protection, including the installation of 12 incinerators acquired by the MSPP, out of which four were financed by the Project, the emptying of four septic tanks, and the construction of three ecological sanitary blocks. Over 50 latrines and seven water pumps in Truitier, an endemic area in Port Au Prince, were rehabilitated and are in use and managed by local community organizations. The Project contributed to the upgrade of the national Standards for Biomedical Waste Management and the Hygiene Promotion Plan. It also provided financial, technical and training support to NGO service providers and MSPP at all levels. The Environmental Unit of FAES followed an action plan including supervision and data collection in all Project sites. Despite these achievements, the implementation was hindered by frequent delays in the procurement of necessary safeguard equipment and in the disbursing of FAES funds for the recruitment of community agents. This is the reason for the Moderately Satisfactory rating in the last Project ISR. 31. Procurement. The procurement arrangements were performed Moderately Satisfactory in accordance with the last ISR. FAES internal procedures experienced some delays in the implementation of the plan. Nevertheless, most of the planned activities were completed. Despite the fact that FAES had previous experience with World Bank procurement rules, this was only limited to the procurement of works and the Project implied the procurement of medicines and medical equipment, among others. T he procurement of these goods posed challenges to FAES team since it required getting to know a new market, the creation of a whole new database of suppliers, and acquiring specific technical skills. Throughout the implementation of the Project, FAES counted on the support of the National Laboratory for the development of technical specifications of medical supplies and equipment. In addition, FAES found innovative ways to overcome procurement bottlenecks. For example, FAES team worked jointly with different bidders in the design of a new ambulance prototype boat that was required but did not exist in the country. On the other hand, there were lessons learned for improving procurement processes. For instance, the experience with the recruitment of service providers in charge of collecting the baseline data showed that there was a need to strengthen the procurement process through allowing sufficient time to collect expressions of interest, the use of a wider range of publishing tools (internet, etc.) and awareness of potential applicants to recruit high- quality firms to carry out accurate, exhaustive, and timely community surveys and data analysis. For three months (from May to July 2012), the Project benefited from the daily support of procurement specialists which resulted in a timely review of procurement activities. Overall, the close relationship between the FAES and World Bank procurement team allowed procurement bottlenecks to be overcome. 10 32. Financial Management arrangements in terms of accounting, budgeting, fund flow, internal control, external audit and financial reporting were performed Moderately Unsatisfactory in accordance with the last ISR. Significant shortcomings existed. Main issues encountered were that FAES, the Project Implementing Agency, was unable to provide current disbursement rates and accurately projections. There was a high financial management staff turnover in FAES and weaknesses in the withdrawal application submission throughout implementation. Delays in submitting withdrawal requests and/or supporting documents were in part due to problems encountered by FAES in accessing the World Bank´s client connection system. Despite support from the World Bank’s supervision team, withdrawal applications took a long time to be submitted and were often incomplete, delaying disbursements and implementation of Project activities. In addition, a qualified opinion was issued in the audit report covering fiscal year ending on September 30, 2012 and an ineligible expenditure was identified (absence of supporting documentation related to salary paid to the health department) although this issue was resolved later on. Weaknesses existed but did not prevent the managing and monitoring of information required for Project implementation. Due diligence was taken by FAES with the recruitment of a consultant who helped to find out the missing supporting documentation that was verified by the auditors during the audit ending on September 30, 2012. At the time of preparation of this ICR, the audit report covering the period October 1 st , 2012 to September 30, 2013 was overdue. The final audit covering the period October 1st, 2013 to July 30, 2014 is expected to be submitted to the World Bank no later than November 30, 2014. 2.5 Post-completion Operation/Next Phase 33. The GOH continues to prioritize the fight against cholera. Interventions to achieve this goal are included in the Operational Plan 2013- 2015 of the MSPP, which emanates from the National Plan for the Elimination of Cholera 2013- 2022. The ultimate goal of the Plan is to eliminate cholera from the island of Hispaniola through technical and financial support from the international community and binational coordination. 12 34. The World Bank remains strongly engaged in the fight against cholera in Haiti. In fiscal year 2013, the World Bank approved financing for a project aimed at Improving Maternal and Child Health through Integrated Social Services. This project supports the National Plan for 12 The objectives of the Plan are the following: (i) increase access to potable water to at least 85% of the population; (ii) increase access to improved sanitary and hygiene facilities to at least 90% of the population; (iii) increase collection of solid waste in the metropolitan area of Port- au-Prince to 90% and in secondary cities to 80%; (iv) strengthen the public health system to facilitate access to health care services for 80% of the population by increasing the number of physicians and nurses per 100,000 population; (v) strengthen epidemiological and laboratory surveillance for early detection of all cholera cases and other diseases under surveillance; (vi) ensure research on outbreaks and a response linked to surveillance activities; (vii) ensure a strong laboratory surveillance component to examine the possible serotypes and genotypes, as well as eventual changes in antimicrobial resistance among Vibrio cholera strains in Haiti; (viii) intensify education of the public about household hygiene and food hygiene to the extent that by 2022, 75% of the general population in Haiti will have knowledge of prevention measures for cholera and other diarrheal illnesses; and (ix) put in place an evaluation tool to measure the impact of activities related to cholera, water-borne diseases, and, more broadly, socioeconomic indicators such as absenteeism from schools and workplaces. 11 the Elimination of Cholera through a US$20 million component. The new operation was built on the lessons from this Project by utilizing identical implementation structures while further enhancing the focus on integrated approach of delivery of a cost-effective maternal and child health, nutrition and other social package of services at facility and community level as well as directly to vulnerable households, and continuing capacity building in Project implementation and fiduciary management aspects. The follow-up project considers contracting with public and non-public providers through a results-based financing model to provide the package of services. A network of community agents (Kore Fanmi) is expected to provide basic preventive services such as hygiene and cholera prevention practices. The new project seeks to continue strengthening the stewardship and management capacity of the GOH, donor coordination on implementation of the national health policy and the Cholera Elimination Plan, and the coordination with the Centers for Disease Control through use of joint implementing mechanisms. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 35. Project objectives, design and implementation remain highly relevant. These are aligned with GOH’s main strategy of implementing an integrated approach to prevent and stop the secondary transmission of cholera in Haiti. The Project supported directly the objectives of the National Strategy for the Response to the Cholera Epidemic namely (i) social mobilization, awareness raising on preventive measures and training of community agents; (ii) treatment of moderate cases through the strengthening of health centers and Cholera Treatment Units. The Project supported the National Response Strategy for the Water and Sanitation Sector through activities that limit the spread of cholera epidemic through hygiene promotion and health education campaigns and reinforcement of sanitation waste management. In addition, the Project supported, to a certain extent, all nine long term objectives of the National Plan for the Elimination of Cholera that became effective during Project implementation. 36. Project objectives support two pillars of the World’s Bank Country Strategy FY09- FY12 (Report No. 48284- HT), which are investing in human capital, with a focus on improving the delivery of basic services; and reducing vulnerability to disasters. As such the Project is consistent with the World Bank’s current Country Assistance Strategy. 37. In the medium term, the implementation of Project activities will likely help sustaining control of the epidemic by integrating cholera treatment into health services and reducing costs. The P roject began strengthening the public health system for the longer term. The phase-in of MSPP’s cholera prevention, control, and treatment efforts allowed sufficient time for defining clear priorities. These helped to focus efforts on integration of cholera treatment into healthcare services and the establishment of an effective system of epidemiological surveillance, ensuring smooth transition of previously stand- alone cholera treatment activities and sites into the existing health facilities. Finally, all activities of the P roject aimed to contribute to the containment of the epidemic in the country but also to initiate sustainable solutions while respecting the environment. These included among others, a better collaboration between the health (MSPP) and the water and sanitation sectors (DINEPA) and the 12 Project’s contribution to the implementation of the waste management and environmental protection plan. 3.2 Achievement of Project Development Objectives 38. Overall, the Project was successful in achieving the expected results based on the baseline and ex -post targets established during Project implementation. Two of the three outcome indicators were achieved (PDO outcome indicators No. 2 and No. 3). The remaining indicator (PDO outcome indicator No. 1) was partially achieved, but showed a very good progress. Seven of the eight intermediate outcome indicators were surpassed , and one was not achieved. (Annex 2). 39. The Project was Modest in achieving its first sub-objective of improving the health and hygiene practices in order to reduce the spread of cholera by nearly achieving its intended objective. Achievement of the health and hygiene practices objective was to be evaluated through two outcome indicators: percentage of the population in Project intervention areas who know (i) the cholera early warning symptoms and prevention steps (PDO outcome indicator No. 1), and (ii) percentage of population in Project intervention areas with access to improved water sources (PDO outcome indicator No. 2). 40. The Project contributed to intensify ing health, household and food hygiene education 13 of the people in the Project localities through community mobilization and prevention activities carried out by the three NGO service providers (PIH, World Vision, and Save the Children) and the MSPP. By Project completion, the percentage of the population knowledge on the cholera early warning symptoms (Part of PDO outcome indicator 1) increased from 64 percent in 2011/2012 to 82 percent in 2013, surpassing the 69 percent target. There was a slight increase in the percentage of the population’s knowledge on the prevention steps (also part of PDO outcome indicator 1) from 13 percent in 2011/2012 to 14 percent in 2013, albeit not achieving the 18 percent target. A modest rating was given due to the fact that despite that the population improved their knowledge on the cholera early warning symptoms, it did not improve their knowledge on prevention steps, which is crucial to break the transmission of cholera. 41. By Project completion, access to improved water sources had been slightly increased and sanitary and hygiene facilities as well as the collection of solid waste had been improved. The Project provided training to more than 4,000 health and hygiene agents and medical personnel. As a result, the Project populations have better knowledge on early detection and prevention of cholera, on treatment of drinking water and on seeking health- care. The distribution of water treatment products and soap expanded from less than 200,000 to 800,000 thanks to the Project. In addition, 115 latrines and 7 water pumps in Truitier were rehabilitated in the endemic area in Port Au Prince. The Project contributed to the waste management and environmental protection of the country through the installation of 12 incinerators, the emptying of four septic tanks, and the construction of three ecological sanitary blocs. By Project 13 Education activities focused on knowledge on symptoms (2 out of 2 symptoms: diarrhea rice color and vomiting) and means of prevention (4 out of 6 prevention steps: hand washing, water treatment practice, not eat fruits and vegetables not cleaned, eat hot meals, not defecating near water point source, household waste elimination far from home). 13 completion, the percentage of the population in Project intervention areas with access to improved water sources (PDO outcome indicator 2) increased from 78 percent in 2011 to 79 percent in 2013, achieving the 79 percent expected target, as shown in the CRD report. 42. The Project highly achieved its second sub- objective of strengthening the institutional capacity to respond to outbreaks with the full achievement of the related outcome indicator: Increased institutional capacity as measured by achievement of at least four out of five of Component 2 indicators on an annual basis (PDO outcome indicator No. 3). Three out of four indicators used to measure this PDO indicator were surpassed. (i) The number of personnel and community workers trained (intermediate outcome indicator 4) increased from 0 in 2010 to 5571 in 2013, surpassing the end of Project target of 2,633. (ii) The Departmental Cholera Management Plan developed (by MSPP and DINEPA) for at least five departments (intermediate outcome indicator 5) increased from 0 in 2010 to 6 in 2013, surpassing the end of Project target of 5. (iii) The establishment and functioning of one national and five regional Cholera and Emergency Response Units (intermediate outcome indicator 6) increased from 0 in 2010 to 7 in 2013, surpassing the end of Project target of 6. (iv) Quarterly Project progress reports submitted to Cholera Coordination Committee and to Bank (intermediate outcome indicator 7). At least 10 quarterly Project reports should be submitted during the period of the Project. In total, 17 quarterly (NGOs, MSPP, DINEPA) progress reports were developed and submitted, achieving its target. 43. The Project contributed to improving Haiti’s epidemiological surveillance. The revision of the National Epidemiological Surveillance Plan is now complete and the MSPP has the capacity to report on statistical epidemiological profile of any potential epidemic in a weekly basis through the support provided to the DELR and the National Laboratory of Public Health (LNSP) among other partners. Surveillance capacity was strengthened through the reinforcement of human and functioning resources. The process includes funding of a technical coordinator, two epidemiologists and a data entry operator for DELR. In addition, a coordinator, technical assistant, and field technicians were recruited, trained and are now in place to strengthen the LNSP personnel. Moreover, inputs for basic laboratory operations (syringes, gloves, lab coats, etc.) were procured by the Project to ensure infection prevention of the laboratory personnel. Implementation of DINEPA related work has improved : (i) the Coordinators of the Departments for Urgent Response and Information Management have been in pl ace since the end of 2012; (ii) 12 water focal points have been in place in all ten departments since January 2013; and (iii) 82 technicians on water and sanitation (TEPAC) have been in place in three departments since November 2012. 3.3 Efficiency 44. Project efficiency was Substantial , with the economic and social benefits of the Project outweighing the costs. The Project disbursed US$14.9 million (99.6% of the grant). Direct and indirect costs were calculated to evaluate the efficiency and economic impact of the Project. Direct costs are those associated with the provision of health care and stem from either outpatient clinic use or hospitalization. Indirect costs are those associated with loss in 14 productivity due to illness and death. Four scenarios were developed to take into account variation in the cost of treatment (at a low and an intermediate level) as well as the number people reached by the Project. In all four scenarios the Cost Benefit ratio is greater than one. The two intermediate scenarios, which can be seen as the most plausible ones, have an Internal Rate of Return (IRR) of 34 and 16 percent. Thus, health service delivery, health promotion and hygiene practices as well as improvement in access to water in most affected areas were a set of investments economically worthwhile in Haiti. 45. The Project contributed to the net decrease in the spread of cholera between 2011 and March 2014. The cholera situation has greatly improved in Haiti thanks to the efforts of the international community and the GOH. At Project closing, the cholera case fatality and incidence rates were at their lowest level since 2010 (see Annex 10). The design of the Project contributed to this important achievement through emergency health and water and sanitation interventions, such as the provision of proximity treatment to infected patients, the implementation of health and hygiene campaigns, community outreach activities and strengthening the Government’s capacity to respond rapidly to emergencies, among others. Table 2 and Figure 1 below illustrate the progress in the decrease of the cholera in Haiti. It should be noted that the 2012 increase in CFR coincided with the increase of rainfall. 14 Table 2 : Summary of cholera situation, Haiti, 2010 to 2014 Year Population Cases seen Hospitalized cases Institutional Deaths Community Deaths Total Deaths Hospitalized lethality Incidence rate per 1,000 2010* 10,085,214 185,351 103,728 2,521 1,580 4,101 2.43% 18.38 2011 10,248,306 352,033 186,673 1,950 977 2,927 1.04% 34.35 2012 10,413,211 101,503 61,877 597 311 908 0.96% 9.95 2013 10,579,230 58,574 37,649 403 184 587 1.07% 5.54 2014** 10,745,665 5,544 3,554 59 9 38 0.82% 0.52 *October to December 2010; ** May 31, 2014 Source: Statistic Profile of cholera epidemiologic week 22, National network surveillance report, MSPP, Direction of Epidemiology, Laboratory and Research (DELR) - 31 May 2014 14 Assistance Publique Hopitaux de Marseille (financed by UNICEF-Haiti) Report; Cholera Elimination in Haiti, Situation in May 31, 2014. 15 Fig 1. Trends in annual cholera case -fatality rate and number of cases of cholera in Haiti 2010 to 2014 Source: Statistic Profile of cholera epidemiologic week 22, National network surveillance report, MSPP, DELR – May 31, 2014 46. The Project contributed to the reduction of cholera incidence and mortality. Between 2010 and 2013, the cholera case fatality rate (CFR) in Project areas decreased by 1.8 percentage points, compared to the 1.2 percentage point decrease at the national level. This is a noteworthy progress considering the high CFR in 2010 in the Project areas (2.7 percent) as compared with the one at the national level (2.2 percent). This tends to confirm the effectiveness of access to adequate treatment and quality of case management which the P roject contributed to by establishing 174 oral rehydration points and 82 cholera treatment units at the community level. Please see Annex 10 for additional information. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 47. The overall outcome rating is Satisfactory. This rating is based on the following considerations: (i) the Project objectives, design and implementation are highly relevant to the Haitian country context and global priorities; (ii) Modest and High ratings in achieving its intended outcomes, as measured by both sub- objectives and (iii) Substantial efficiency. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 48. The strengthening of the epidemiological system and health system’s capacity to respond to outbreak as well as the implementation of community health and hygiene education likely benefitted the lower population deciles. Cholera epidemics are a marker for poverty and lack of basic sanitation. The rapid spread of the disease was mainly due to the lack of immunity and the very limited access of the population to safe water and basic sanitation in a context of internal migrations after the January 2010 earthquake. The Project addressed this challenge by strengthening the epidemiological surveillance system to better target interventions, improving the health system’s capacity to respond to emergencies, and implementing community education on health and hygiene practices toward the poorest populations. 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 20000 40000 60000 80000 100000 120000 140000 160000 20102011201220132014 case fatality rate number of cases number of cases 16 (b) Institutional Change/Strengthening 49. The Project helped to strengthen the capacity of FAES, MSPP and DINEPA in seven Departments to prevent and contain cholera and other water borne- diseases. Project execution resulted in the buildup of in- house capability. The Project supported DINEPA and MSPP through the development and finalization of the National Plan of Epidemiological Surveillance, capacity building activities and the upgrading of national standards for bi omedical waste management and the Hygiene Promotion Plan. Institutional capacity at the departmental level increased through the development of Departmental Cholera Management Plans supported by the P roject and training to institutional personnel. As a res ult of the Project, previous stand– alone cholera treatment activities and sites are now part of the existing health systems at the departmental level. In addition, the implementation of Project activities promoted the collaboration at the departmental level authorities, particularly between the health and water authorities as well as local FAES. (c) Other Unintended Outcomes and Impacts (positive or negative): N/A 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops: N/A 4. Assessment of Risk to Development Outcome Rating: Significant 50. Technically, improvements in approaches to community services are likely to be sustained. There are high levels of political support at the central level (FAES, MSPP and DINEPA) and an ever increasing recognition of the importance of reducing cholera epidemic. This is further substantiated through the declared interests of the GOH to invest further in the elimination of cholera and the fact that a follow-on investment with the World Bank (US$90 million Maternal and Child Health Project) allocated US$20 million to cholera related interventions. This support is important for the country as the cholera outbreak has evolved since 2010 and is now endemic not only to Haiti, but in the whole island of Hispaniola. Evidence from other outbreaks in Latin America indicates that cholera will likely remain a serious public health challenge for several more years, because the bacteria is in the environment and sporadic cases will always be detected. Nevertheless, the Project suffered to some exten t from limited ownership by local government and communities, indicating a risk that the momentum gained and efforts achieved could be lost if efforts are not sustained. 51. One of the main risks to the development outcome relates to the institutional setting in Haiti. The contributions of the Project to the DINEPA, MSPP national strategy to prevent cholera and Project annual plans, among others, have reduced the risk of the spread of cholera. Nevertheless, the volatility of the political context, high staff rotation/turnover constitutes substantial risk. The new Maternal and Child Health Project cited above is facilitating the development of a new Result s-based Financing program that represents an important milestone towards more sustainable development results. 52. The lack of a dedicated multi-sectoral governance structure accountable for preventing and containing Cholera , and of coordinating government bodies and donor intervention poses a risk to the development outcome. Given the diverse set of determinants of cholera, there may be uncertainty as to which sector or entity should lead the implementation 17 of a comprehensive package of community services. The importance of a multisectoral action to prevent cholera is critical, and since cholera has "fecal-oral" transmission and 88 percent of cases of diarrhea worldwide are attributable to unsafe water, insufficient hygiene and inadequate sanitation, the emphasis should be put on water, sanitation and hygiene interventions. Government leadership is essential to ensure that sectoral roles and responsibilities are well defined from central to community levels. 53. The lack of sustained funding for cholera poses a risk to maintaining the momentum. As stated in the United Nations plan to support the implementation of the GOH’s two-year plan on cholera elimination (January 2013), the lack of sustained funding has dramatically decreased capacity to respond effectively. In addition, Haiti has received less than a quarter of the US$40 million required for immediate humanitarian needs. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 54. World Bank performance on ensuring Quality at Entry is rated Moderately Satisfactory. The Project objectives were relevant and firmly supported GOH and global priorities. Project design capitalized on the relevant knowledge, lessons learned on previous World Bank experiences in dealing with cholera epidemics and international experience and discussions with relevant stakeholders. The World Bank team preparing the Project comprised a range of skills, knowledge and operational experiences, which allowed a quick preparation. The Project was jointly prepared with key stakeholders and the World Bank team was able to closely coordinate with strategic partners since then. However, human resource needs and constraints to establish inter-ministerial or inter-organizational teams were underestimated at preparation stage, which had an impact on the implementation of Project activities. In addition, t he likelihood of having M&E constraints during project implementation was underestimated during Project preparation and M&E risks were not included in the risk matrix. Weaknesses in the M&E framework negatively weight on the rating because although it was methodological sound and provided elements to measure Project achievements, it suffered from substantial delays on data availability, which may have impacted on implementation of Project activities and its results . (b) Quality of Supervision Rating: Satisfactory 55. The Project benefited from a steady and committed World Bank team, which successfully identified and resolved Project obstacles along the way. World Bank team worked very closely with Project counterparts. Through intense supervision, the team was able to provide extensive support and timely input to the client. The team worked to resolve issues delaying implementation, urging intensive remedial action on the part of the Government Project team. The team’s strategy of holding shorter but more frequent missions, as well as a focus on short-term achievements provided easier implementation objectives. Agreeing on specific short- term action plans with the client, setting specific goals and deadlines proved crucial in keeping Project momentum. The Bank team hired a field based implementation consultant to provide intensive support to FAES on a daily basis and the World Bank team conducted monthly follow 18 up meetings with the client beyond the regular scheduled missions until the Project closed. The World Bank team, however, should have provided better support to FAES in terms of financial management to ensure prompt disbursements. World Bank team ensured adequate transition arrangements for the implementation of the new Maternal and Child Health Project, including the appointment of financial management staff for Bank -financed projects and the assessment of the public financial management systems of the MSPP . Based on this evaluation, a capacity building program has been developed and a MSPP Accounting, Financial and Administrative Manual produced with Bank support. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 56. Given the Moderately Satisfactory Bank performance at Quality at Entry and the Satisfactory performance during Project supervision, the overall rating for the Bank performance is Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 57. Government performance is rated Moderately Satisfactory. There were shortcomings in Government performance considering the slow speed of Project implementation. Government involvement and political commitment to the Project was extremely good during Project preparation. Project implementation was slow in spite of the adoption of various measures, including increased frequency of technical and management support meetings. Frequent changes in Government staff, both due to administration changes and general turnover, bureaucratic process and financing delays during the life of the Project contributed to a challenging implementation environment. Staff changes occurred at the central and Departmental levels. During the course of the Project, there were five different Director General at FAES and most of the Departmental Directors also changed. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 58. The rating for FAES performance is similar to that of the overall Government since similar issues apply. Performance of the implementing agencies under the Project was in general influenced by great dedication of the core team, but constrained by insufficient human resources and capacities, especially within the area of procurement and financial management. Throughout implementation, FAES was unable to provide current disbursement rates and accurate projections. Some disbursement requests or direct payment from the World Bank, which would have alleviated some of the implementation problems, did not materialize. Requests for disbursement were either not properly submitted or submitted with significant delay. This resulted in delays in delivering planned actions and activities. Delays in obtaining FAES approval on all fiduciary activities by the management level delayed the implementation of the plan and completion of activities. The task team had to address the problem of quality of data to prepare the current ICR. The lack of a robust health management information system within the MSPP remains one of the Project's weakest areas. 19 (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 59. For reasons outlined above, the overall borrower performance is rated Moderately Satisfactory. 6. Lessons Learned 60. The following lessons learned can be mentioned as key in particular. (i) Project design in an emergency situation needs to consider the balance of responding to urgent versus long term needs. In an emergency situation, the usual intervention strategy is to reduce deaths by ensuring prompt access to treatment and responding to shorter term needs. It might have been a good practice to evaluate the Project at the level of outputs (affected population knowledge, use of preventive products or services) specifically for the two- year period of the Project. Indeed, behavior change and the access to adequate sanitation infrastructure need longer period of time to be expected to change. (ii) Institutional capacity and readiness. The implementation agencies' institutional capacity and readiness should not be underestimated or considered less important in emergency situations. The World Bank's strategy of involving experienced NGOs in Project implementation due to the need of a rapid response to the epidemic and to the initial lack of capacity of the Government may have contributed to diminish the MSPP's stewardship role. (iii) In countries where cholera outbreaks follow seasonal cycles, training of health care workers prior to the epidemic may help to increase awareness. Especially relevant is providing information on standard case definition15 and case management. This course of action combined with the informative watch campaign to the most at risk population in any cholera red zone can further increase awareness, ensure adequate diagnosis and ultimately prevent the epidemic from occurring again. (iv) Coordination of service delivery at decentralized level and better communication of the Project activities ensure community participation and ownership of interventions. The decentralized decision making process helped the implementation of the safeguard management/mitigation as an example, to ensure appropriate elimination, after emptying septic tanks, the local authorities designed a landfill site for the community, in compliance with environmental standards, in order to protect the coastline and avoid the contamination of the groundwater. 15 WHO case definition: a case of cholera should be suspected when: (i) in an area where the disease is not known to be present, a patient aged 5 years or more develops severe dehydration or dies from acute watery diarrhea; or (ii) in an area where there is a cholera epidemic, a patient aged 5 years or more develops acute watery diarrhea, with or without vomiting. A case of cholera is confirmed when Vibrio cholera O1 or o139 is isolated from any patient with diarrhea. 20 (v) Consistency of emergency policies (OP/BP 8.00) with other fiduciary policies. Even though OP/BP 8.0 operational guiding principles have been developed bearing in mind the need to respond with speed, simplicity and flexibility in emergency situations, the policy is not fully aligned with other fiduciary policies and guidelines, such as procurement. The implementation of the Project demonstrated that World Bank procurement processes need to be more flexible in emergency situations. (vi) Projects with weak counterpart administrative capacity require innovative approaches to supervision and strong stakeholder commitment. Continued capacity building of implementing agency in all aspects of Project implementation, procurement and financial management needs to be emphasized. In a situation of limited capacity and a variety of actors involved in reviewing and financing approval, more thought should be given to how to more effectively strengthen existing structure (public and non- public). (vii) Monitoring and Evaluation - Tracking results in real time is crucial for controlling cholera epidemics. Collecting and analyzing quality data on the dynamics of the epidemic and on the behaviors of the population is crucial. This enables strategy discussions and shaping the relevant response to the epidemic. Specifically, baseline data collection should be conducted earlier in the Project implementation and by a third party rather than the same NGO service providers implementing the activities to avoid conflict of interest. Strong health information system should be putted in place to ensure better monitoring and evaluation of cholera response Projects. Project sta ff and partners need to be trained in a common methodology for gathering, recording, and documenting indicator data. To do this effectively, department offices need to have M&E specialists. Data audits, carried out by an internal or third party auditor, are as important as financial audits in verifying that a project is on track in achieving project results transparently. (viii) World Bank financial and human resources in responding to public health emergencies. There are a number of aspects that the World Bank needs to consider when responding to public health emergencies. First, the preparation of epidemic-related operations requires even greater urgency in response, since the emergency is ongoing and possibly expanding exponentially. Second, the preparation of these operations requires sufficient budget allocation to put together strong teams to avoid flaws in terms of design, analysis of implementation capacity and assessments of risks. In this case, the P roject preparation team could have benefited from having a monitoring and evaluation expert to help detect possible risks and include the necessary mitigation measures. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies: 61. FAES, DINEPA and MSPP reviewed a draft of this Implementation Completion and Results Report. C omments were provided by FAES on the following sections: (i) Safeguard and Fiduciary Compliance; (ii) Achievement of Project Development Objectives; (iii) Borrower Performance; and (iv) Annex 7. (b) Cofinanciers: N/A (c) Other partners and stakeholders: N/A 21 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Components Appraisal Estimate (USD millions) Actual/Latest Estimate (USD millions) Percentage of Appraisal 1. Support to the Government’s Response to Cholera at the Decentralized Level 11.00 11.00 100.00 2. Emergency Response Capacity Building 4.00 3.95 98.75 Total Baseline Cost 15.00 14.95 99.67 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 0.00 0.00 Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00 Total Financing Required 15.00 14.95 99.67 (b) Financing Source of Funds Type of Cofinancing Appraisal Estimate (USD millions) Actual/Latest Estimate (USD millions) Percentage of Appraisal Borrower 0.00 0.00 0.00 IDA Grant 15.00 14.95 99.67 22 Annex 2. Outputs by Component 1. The Project had three Project outcome i ndicators and eight intermediate outcome indicators. Two out of three Project outcome indicators and seven out of eight intermediate outcome indicators were fully achieved. Only one Project outcome indicator was partially achieved and one intermediate outcome indicator was not achieved. Despite significant issues with implementation delays, the Project utilized nearly the totality of the grant. 2. The Project was organized around two components and the following presents the outputs from each component. Component 1: Support to the Government’s Response to Cholera at the Decentralized Level 3. The Component 1 was successful in supporting the Government’s Response to Cholera at the Decentralized Level. The objective of this c omponent was to carry out a program of activities for immediate response to cholera, as outlined in the National Strategy for the Response to the Cholera Epidemic and defined at the departmental level to support Government’s emergency response to cholera. This component sought to respond to the needs of the departmental level through support to all service providers, whether public, mixed or private, undertaking interventions at all levels of the cholera response. This component financed two sub- components: (i) Sub- component 1.1: Support to a Multi-sectoral Approach to Public Primary Health Care; and (ii) Sub-component 1.2: Support to the Health, Water and Sanitations Response of Non- Public Providers. 4. The activities under this component contributed to the achievement of the first and second PDO indicators. The results achieved are as follows: (i) The percentage of population in interventions areas knowing the cholera early warning symptoms increased from 64 to 82 percent surpassing its target of 69 percent; (ii) The percentage of population knowing cholera prevention steps slightly improved from 13 to 14 percent but did not reach the target of 18 percent; (iii) Almost 4,500 localities received health education and/or prevention activities, surpassing the target of 2,214; (iv) Cholera treatment has been integrated into the existing health facilities through the creation of 16 rooms for diarrheal diseases treatment across the six departments of intervention; (v) More than 250 health facilities were supported. This includes 82 health centers and posts, treatment centers and units surpassing the target of 42; (vi) The percentage of population in intervention areas with access to improved water sources or treated water slightly improved from 78 to 79 percent ; (vii) The number of people receiving water treatment products and soap increased from less than 200,000 to almost 800,000; (viii) Over 50 latrines and seven water pumps in Truitier, an endemic area in Port Au Prince, were rehabilitated and are in use and managed by local community organizations; and 23 (ix) All facilities benefited from technical and financial support for waste management and environmental protection, including the installation of eight incinerators acquired by the MSPP, four incinerators acquired and installed by the Project, the emptying of four septic tanks, and the construction of three ecological sanitary blocs. Component 2: Emergency Response Capacity Building 5. The Project was successful in building capacity to respond to the emergency. This component financed activities to support the successful implementation of Component 1 and ensure that the MSPP and DINEPA were prepared to tackle cholera and other possible water- borne diseases. As such, this c omponent supported a number of emergency response capacity building activities for MSPP and DINEPA as well as overall Project management and supervision through two sub- components: (i) Sub- component 2.1: Strengthening Government’s capacity to manage and respond to outbreaks; and (ii) Sub-component 2.2: Carrying out a program of activities to support the management of Project, including monitoring, supervision and audits. 6. The activities under this component contributed to the achievement of the third PDO indicator. The results achieved under this component are as follows: (i) All six intervention Departments have a Departmental Cholera Management Plan; (ii) A national cholera focal point and six coordinators at the Departmental level are in their position; (iii) The Project supported the DELR and the LNSP through the field technicians collecting specimen to detect cholera cases as well as the development and finalization (including recent validation by the Minister) of the National Plan of Epidemiological Surveillance; (iv) The Project supported the DINEPA through the TEPACs and the rehabilitation of at least 20 water points; (v) The Project contributed to the upgrade of the national Standards for Biomedical Waste Management and the Hygiene Promotion Plan. It also provided financial, technical and training support at all levels; and (vi) More than 5,500 community agents, supervisors, and institutional personnel throughout the 6 departments were recruited and trained, far exceeding the Project target of 2,633. 24 Annex 3. Economic and Financial Analysis 1. The present Annex presents the results of the economic evaluation of the Project. The analysis considered direct and indirect costs saved (benefits) and the cost of investment to calculate the Cost-Benefit Ratio (CBR) of the Project. This analysis aims to evaluate the economic impact of the Project whose objective was to improve the health and hygiene practices in order to reduce the spread of cholera and strengthen the institutional capacity to respond to outbreaks. The main outcome for this evaluation were the number of cases averted (hospitalized and outpatient) and deaths. The direct costs were based on the cost of treatment due to the lower number of cholera cases and the indirect costs were calculated as the loss in productivity due to illness and premature mortality due to cholera. 2. The analysis considered only a proportion of the targeted population that could be reached by the Project. Though the Project targeted seven departments, activities were carried out in some localities within those departments. The population in each of those departments and in the localities of intervention is presented in Table 3.1. The MSPP and DINEPA coordinated all cholera response activities with international partners. Under their coordination, a project financed by the IDB and implemented by UNICEF was also approved at the end of 2010 to carry out activities to control cholera in the Great North (North-West, North and North- East departments). This explains why, in spite of the high disease prevalence, the World Bank- financed Project has not intervened in the North. Table 3.1: Population at departmental level and in localities of intervention Department Departmental Population Population in localities of intervention % of Population in Departments of Intervention Artibonite 1,571,020 1,378,570 88% Centre 678,626 531,534 78% Grande-Anse 425,878 265,051 62% Nippes 311,497 196,744 63% West 3,664,620 524,137 14% South 704,760 354,703 50% South- East 575,293 425,352 74% Total 7,356,401 3,250,739 44% Source: Own calculations based on data from the Haitian Institute of Statistics and Informatics 2009 (1) 3. The analysis considered two scenarios to calculate the impact of the P roject on the population reached (on cholera cases and deaths averted) in the localities of intervention. In both scenarios, the underlying principle was that the percentage of all cases/deaths averted that was attributable to the World Bank P roject should be equal to the percentage of total new financing for anti-cholera activities that came from the World Bank P roject (within the localities covered by the World Bank P roject) – we refer to this percentage as X percent. 25 4. While data constraints make it impossible to know the value of X with precision, it should not be far from 100 percent, because MSPP and DINEPA made explicit efforts to coordinate the response and ensure that partners’ support was distributed meaningfully across the country. (At the beginning of the epidemic cholera spread quickly to all ten departments.) In the localities covered by the World Bank Project, this Project was thus the main financing source for new activities against cholera, in line with the principle – adopted by MSPP and DINEPA – that each major partner should be “assigned” particular localities where they should be the main partner financing new anti-cholera activities. The World Bank Project was thus assigned localities with a combined population of 3.2 million (as in Table 3.1), which is around 31 percent of the total population in Haiti of 10.3 million. This is in line with the available total financing for cholera of US $14.9 million disbursed for the World Bank Project, which was more than 30 percent of the total financing for cholera from major partners, 16 according to a mapping of funding carried out by the Cholera Coalition in 2012 (2). 5. Although X should be close to 100 percent in practice, it is possible that the fall in cases and deaths were partly due to other unobserved or unknown factors that are not apparent from the available information. Although each major partner in the anti-cholera effort was assigned some localities, the incidence of cholera in any one locality is certainly also affected by activities in other localities, given the contagious nature of cholera and its ease of spread between different localities. This means that the falls in cholera cases/deaths observed in the World Bank Project localities could have been due partly to interventions elsewhere, although similarly the interventions of the Project in its localities could also have had a positive impact elsewhere. 6. To be conservative, we thus produced estimates for two Scenarios – one where X was 33 percent (one third) and another where X was 66 percent (two thirds). It should be noted that both of these (especially the 33 percent estimate) are likely to be underestimates of the true value of X. 7. For each of these two scenarios, number of cases (hospitalized and non- hospitalized) and deaths averted in the localities of intervention were calculated. The fall in the number of new cases and the fall in the number of new deaths in Period 1 (2011 - 2012) and Period 2 (2012- 2013) in the localities of intervention – based on actual recorded data – were used as the basis for the calculations as shown in Table 3.3. The total number of averted cases and averted deaths was assumed to be equal to the fall in the total number of cases and of deaths, respectively, in the localities of intervention. 8. In scenario 1, the number of averted outpatient cholera cases attributable to the World Bank Project for both periods combined was found to be 17,491 (15,743 + 1,748 in Table 3.2) and the number of averted hospitalized cases attributable to the World Bank Project was 15,024 (13,159 + 1865). As for averted deaths, scenario 1 found a decrease of 353 (311 + 42) attributable to the Project. In turn, Scenario 2 found a decrease of 34,982 (31,487 + 3,495) and 30,048 cases (26,318 + 3,729) for outpatient and hospitalized cases respectively that were 16 This excludes financing from IDB and AECID/FCAS for water infrastructure projects focusing mainly on urban areas. Their operations started much before the cholera outbreak and did not explicitly target areas with high cholera incidence. 26 attributable to the Project, over the entire period. In Scenario 2, a decrease of 705 (621 + 84 in Table 3.2) deaths was found to be a ttributable to the Project in the localities of intervention. Table 3.2: Number of averted cases and deaths in the localities of intervention Department Outpatient cases Hospitalized cases Deaths 2011-2012 2012-2013 2011-2012 2012-2013 2011-2012 2012-2013 Artibonite 22,908 1,591 12,535 542 264 1 Centre 7,206 378 5,497 1,801 119 37 Grande-Anse* 2,019 495 4,208 (1,529) 203 (4) Nippes 1,277 92 1,671 609 29 21 West 7,635 2,410 7,887 2,492 165 30 South 5,834 54 4,778 1,381 50 18 South-East 351 223 2,901 298 102 23 Total 47,230 5,243 39,477 5,594 932 126 Total attributable to Project (scenario 1) 15,743 1,748 13,159 1,865 311 42 Total attributable to Project (scenario 2) 31,487 3,495 26,318 3,729 621 84 Source: Own calculations based on information from MSPP database 9. Note: Negative numbers of Grande Anse are due to two effects. The first is the cholera spike in the rainy season of 2013, and the second the improvements in cases recoded by the system of epidemiological surveillance. The analysis takes into account these negative values reducing the amount saved due to averted cases. 10. To assess the efficiency and economic impact of the Project, the benefits were estimated as the direct and indirect costs saved from the averted cases and deaths. The benefits stemmed from the direct costs were the cost of treatment saved due to the number of averted cases. Direct costs are those associated with the provision of health care. It is assumed that the only direct costs for cholera stem from either outpatient clinic use or hospitalization. Calculations were based on the costs proposed by University of Pittsburgh Medical Center (UPMC) for Health Security based on World Health -Organization- Choosing Interventions that are Cost-Effective (WHO-CHOICE) data. The Infectious Disease Cost Calculator (elaborated by the UPMC) presents an estimate of the cost of treatment that a patient could have in the Americas’ Region per day (1). It differentiates the costs of treatment according to whether the country has a very low, low or high mortality rate in children and adults. For this analysis we have taken into account two different cost estimates. Scenario A assumes a low cost of treatment (for a country with high mortality) and scenario B assumes an intermediate cost of treatment (for a country with low mortality). Although cholera continues to afflict Haiti, between 2011 and 2013 there was a reduction of more than 80 percent in the new number of cases and deaths and the fatality decreased at less than 1 percent. Therefore, the scenario B with an intermediate cost alternative for countries with low mortality also seems appropriate. The information on the daily cost of treatment for a cholera patient in the Americas Region is presented in Table 3.3. It is assumed that that the normal period of illness of cholera is 7 days if the patient does not attend a health facility, and 5 days of illness if the patient is treated by health professionals (3). 27 Table 3.3: Daily costs of treatment of Cholera in the America’s Region Cholera Treatment Cost Inpatient (US$) Clinic (US$) High mortality for children and adults (low cost, Scenario A) 50.68 9.66 Low mortality for children and adults (intermediate cost, Scenario B ) 162.96 22.83 Very low mortality for children and adults (high cost, was not used) 665.45 65.5 Source: University of Pittsburgh Medical Center UPMC, 2010 (4) 11. The benefits derived from the indirect costs are those associated with the loss in productivity avoided due to illness and death averted. This analysis calculates benefits derived from indirect cost in two ways. The first is the value of lost productivity due to illness, which is assumed to be the labor days lost to illness multiplied by the average daily per capita gross domestic product (GDP). To account for productivity losses from a caregiver, an additional 0.5 days of per capita GDP is assumed for every day a person was ill. The second indirect cost is calculated as the value of the Years of Life Lost (YLL) due to premature mortality because of cholera (5, 6). This value is assumed to be the discounted stream of foregone earnings due to a person that dies due to cholera, and the earnings in each year are assumed to be equal to the per- capita GDP (for each person), taking into account the growth in per-capita GDP over time. The calculation differentiates between under -five children and those older than five (adults and children), since there is some difference between these two groups in terms of the impact of cholera. Specifically, it is assumed that: (i) 8 percent of cholera cases in Haiti are accounted for by under-five children (calculation based on MSPP surveillance data); (ii) life expectancy is 62 years; (iii) the mean age of older cholera patients (aged over 5) is 30 years old and is 3 years old for the under-5 patients; (iv) the age when people start to work is 18 years old; (v) the estimations used the discount rate of 3 percent, as recommended by the World Health Organization for all diarrheal diseases, and which is used in several analyses of cholera (7, 8). The calculations incorporate US$819.9 as the value for the GDP per capita in 2013 (9). This methodology produced estimates of US$26,210 for the value of an adult life lost (or anyone aged over 5), and US$35,874 for an under-five child. The years of life lost due to Disability were not calculated. Therefore, Disability-Adjusted Life Years Lost (DALYs) analysis is not presented, and is not taken into account in the calculations. The figures presented are thus underestimates; the estimated indirect costs would be much higher if DALYs were incorporated in the calculations. The indirect impact of the program on the non- targeted population is not included in the analysis either. 12. Due to the lack of consideration of other direct and indirect costs, the Project's impact is underestimated and thus, the net benefit would in practice be much higher. The analysis only took into account the direct cost of cholera treatment and no other indirect costs such as transport cost, social prejudice of being sick and the discomfort caused by cholera among others. In addition, the benefits do not consider the positive externalities that the Project may have had outside the areas of intervention, nor reductions in the incidence of other diseases (e.g. waterborne diseases other than cholera). In addition, the low cost treatment option revealed an 28 average cost per case (outpatient and hospitalized) of US$144.77 which is much lower than the average cost per case of US$327 estimated with the Infectious Disease Cost Calculator when using data for Haiti (1). That may indicate that the low cost treatment option may indeed underestimate the benefits of saved costs. Regarding the number of cases and deaths, the reduction of number of cases/deaths between 2010 and 2011 was not taken into account (only the reduction between 2011- 2012 and 2012-2013). Since the Project started in February 2011, the analysis could have considered that impact. However, since the epidemic started in October, it was difficult to find a comparable timeframe for October- December and therefore we compared the changes that occurred during the three full years for which data were available. This might also underestimate the number of cases. 13. Four scenarios were developed to account for the variation in the extent of attribution in intervention areas to the Project (Scenarios 1 and 2) and for the cost of treatment at a low and an intermediate level (Scenario A and B). In all four scenarios the CBR is greater than one. Table 3.4 shows the breakdown in the benefits derived from the direct and indirect costs (from illness and deaths) saved due to the averted number of cases and deaths. The total benefits are the result of summing up the benefits from direct and indirect costs averted in each scenario. The CBR is calculated as the ratio of total benefits to total investment costs of more than 14.9 million disbursed by the Project. Given the Haitian context and taking into account the impact of fewer cases and lower cholera fatality rates on the costs of treatment, Scenario 1B is probably the most plausible scenario followed by Scenario 2A. The Internal Rate of Return 17 (IRR) was also calculated for each scenario, being 34 and 16 percent for Scenarios 1B and 2A, respectively. To sum up, the present analysis reveals that the health service delivery, health promotion and hygiene practices as well as improvement in access to clean water in most affected areas were a set of investments economically worthwhile in Haiti. 17 The Internal Rate of Return is the discount rate that m akes the Net Present Value of all cash flows (both positive and negative, i.e. benefits as well as costs) from a particular investment equal to zero. 29 Table 3.4: Total benefits and benefit-cost ratio in different scenarios in US$ Source: Own calculations Note: Scenario 1A: Low Reach (one third of cases/deaths in intervention areas attributable to Project) & Low treatment cost. Scenario 1B: Low Reach (one third of cases/deaths in intervention areas attributable to Project) & Intermediate treatment cost. Scenario 2A: Higher Reach (two thirds of cases/deaths in intervention areas attributable to Project) & Low treatment cost. Scenario 2B: Higher Reach (two thirds of cases/deaths in intervention areas attributable to Project) & Intermediate treatment cost . Scenarios Benefits from direct cost saved Benefit from indirect cost saved from productivity loss saved Total Benefits (US$) Benefit- Cost Ratio Treatment cost Due to illness Due to mortality Scenario 1A 4,780,971 781,061 9,542,576 15,104,607 1.01 Scenario 1B 14,653,180 781,061 9,542,576 4,976,817 1.67 Scenario 2A 9,561,941 1,562,122 19,085,152 30,209,215 2.02 Scenario 2B 29,306,360 1,562,122 19,085,152 49,953,633 3.35 30 Annex 3. References (1) Institute Haitïen de Statistique et d'Informatique. Population totale, population de 18 ans et plus ménages et densités estimés en 2009. (2) PAHO. Mapping of Coalition Contributions to the Cholera Elimination Initiative in Haiti. Summary report, 2012. 23 pages. Mimeo. (3) UPMC Center for Health Security. Infectious Disease Cost Calculator. Cholera Cost Model Methods. [Online] 2012. [Cited: April 30, 2014.] http://www.idcostcalc.org/contents/cholera/cost- model.html. (4) University of Guelph. Food Safety Network. Vidrio Cholerae. [Online] June 31, 2012. [Cited: April 29, 2014.] https://www.uoguelph.ca/foodsafetynetwork/vibrio- cholerae. (5) World Health Organization. Metrics: Disability-Adjusted Life Year (DALY). Health statistics and information systems. [Online] 2014. [Cited: May 24, 2014.] http://www.who.int/healthinfo/global_burden_disease/metrics_daly/en/. (6) Ali, Mohammad, et al. The global burden of cholera. Bulletin of the World Health Organization. [Online] World Health Organization, March 2012. http://www.who.int/bulletin/volumes/90/3/BLT-11-093427- table-T3.html. (7) Costs of illness due to Cholera, costs of immunization , and Cost-Effectiveness of an Oral Cholera Mass Vaccination Campaign in Zanzibar. Schaetti, Christian, et al. 10, Geneva: PLOS Neglected Tropical Disease, 2012, Vol. 6. doi:10.1371/journal.pntd.0001844. (8) Economic burden of cholera in the WHO African region. Kirigia, Joses, et al. 8, 2009, BMC International Health and Human Rights, Vol. 9. (9) World Development Indicators. World Bank. [accessed: June 29, 2014] 31 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Maryanne Sharp Former Task Team Leader LCC2C Eleonora Del Valle Cavagnero Task Team Leader GHNDR Francesca Lamanna Co-Task Team Leader GSPDR Hassine Hedda Senior Finance Officer CTRLA Prosper Nindorera Senior Procurement Specialist GGODR Franck Bessette Senior Financial Management Specialist GGODR Josue Akre Financial Management Specialist GGODR Helene Bertaud Senior Counsel LEGAM Valerie Hickey Senior Biodiversity Specialist GENDR Miguel-Santiago da Silva Oliveira Senior Finance Officer CTRLN Alois Ndorere Consultant AFTPW Kathleen E. Krackenberger Consultant GHNDR Jimena Jesus Mejia Consultant GHNDR Emmanuel Ngollo Consultant GENDR Viviana Gonzalez Team Assistant GHNDR Supervision/ICR Claudia Macias ICR Task Team Leader and Primary Author GHNDR Voahirana Rajoela ICR Primary Author GHNDR Eleonora Del Valle Cavagnero Economist/Economic Analysis GHNDR Andrew Sunil Rajkumar Senior Economist/Economic Analysis GHNDR Daniela Pena de Lima Senior Operations Officer/Quality Review GHNDR Viviana A. Gonzalez Program Assistant GHNDR (b) Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands (including travel and consultant costs) Lending FY2011 29 128,811 Sub-total: 29 128,811 Total: Supervision/ICR Staff Weeks US Thousands (including travel and consultants costs) FY 2011 25 164,376 FY 2012 52 277,263 32 Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands (including travel and consultant costs) FY 2013 23 165,526 FY 2014 25 114,238 Sub-total: 125 Total: 154 721,403 33 Annex 5. Beneficiary Survey Results N/A 34 Annex 6. Stakeholder Workshop Report and Results N/A 35 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR COMPLETION REPORT CHOLERA EMERGENCY RESPONSE PROJECT 7.1 Context of the Project, Development Objectives, and Design 1. Cholera did not exist in Haiti 18. It appeared in the country in October 2010, putting the entire country at risk because the population had no immunity against the disease. In addition, over the past five years, Haiti has demonstrated its vulnerability to a wide range of economic and social crises as well as numerous shocks such as natural disasters (four consecutive hurricanes in one year in 2008: Faye, Gustave, Hannah and Ike). The precarious humanitarian situation that followed the earthquake of January 12, 2010 also exacerbated the epidemic. This disaster has stripped the areas of Port-au-Prince (West), Léogane (West) and Jacmel (South-East) of their most important infrastructure, including schools, hospitals, housing, electricity, water and telecommunications, and caused enormous loss of life. Estimates indicate that nearly 3 million people were affected by the earthquake and more than 200,000 people were killed, while 300,000 people were injured and 1 million people were left homeless and living in camps throughout Port-au-Prince and other cities. Hurricane Thomas hit Haiti on November 5, 2010 and caused flooding, landslides, loss of harvests in some regions of the country, which aggravated the already precarious living conditions. This was compounded by the cholera epidemic that caused 2,193 deaths and 46,749 hospitalizations as of December 6, 2010, and put 200,000 people at risk of infection and 10,000 at risk of death in the six months that followed 19. 2. The Government of the country, in partnership with a number of organizations, established an Emergency Cholera Coordination Committee that was quickly put in place by the Department of Civil Protection, in collaboration with MSPP at the central level and the corresponding committees at the departmental and municipal levels. MSPP led the humanitarian response, with technical support from PAHO/WHO, mobilizing partners from all sectors including health, education and communication to ensure a rapid and synchronized response on the ground. 3. MSPP developed the National Response Strategy to the Cholera Epidemic, which describes essential activities at two levels of intervention, namely at community level, and at institutional level, ensuring support of referrals, mild cases as well as the most serious cases. 4. DINEPA also developed a National Water and Sanitation Strategy with similar objectives, supporting activities limiting the cholera epidemic through short- term improvements in the provision of water, an increase in hygiene promotion and health education campaigns, the strengthening of sanitation and waste management as well as effective coordination with local authorities. 18 JENSON D., SZABO V., and the Duke FHI Haiti Humanities Laboratory student Research Team University, November 2011, Emerging infectious diseases: Cholera in Haiti and other Caribbean Regions, 19 th Century. 19 MSPP/OPS-OMS 36 5. To meet the challenges brought by the cholera epidemic, the Bank made a USD 15 million equivalent grant available to the Government of Haiti through a fast-tracked Cholera Emergency Response Project that became effective on February 7, 2011, and whose ending date was set to March 30, 2014. The project aimed at improving the health and hygiene practices in order to reduce the spread of cholera and strengthen the institutional capacity to respond to outbreaks. 6. As expected, the project was executed by MSPP, DINEPA and three (3) NGOs that were already implementing project activities and were selected based on their ability to expand operations and/or absorb additional funding in a short period of time for the completion of emergency activities. This way of recruiting NGOs aptly answered FAES’s strategy of always intervening through service provider operators (OPS), which in this specific case were international non- governmental organizations: Partners in Health, Save the Children and World Vision. 7. The project was deployed in six (6) health departments in the country: Artibonite, Centre, Grande Anse, Nippes, South and South- East and implemented through two components: (i) Support to the Government’s Response to Cholera at the Decentralized Level; and (ii) Emergency Response to outbreak Capacity Building. The objective of the first component was to carry out a program of activities for immediate response to cholera. This Component responded to departmental level needs through support to all service providers, whether public, mixed or private, and to interventions at all levels of the cholera response, as well as through community participation and social mobilization, and support to community and institutional staff for an effective response to cholera, not to mention the water and sanitation sector. In addition to deployment and remuneration of human resources as well as technical assistance, equipment, vehicles, medical supplies, goods and materials for medical waste management, were purchased. 8. In addition, the project rehabilitated health facility structures for the management of acute diarrhea that were prioritized in the implementation of departmental plans. Such plans constituted the cholera response at departmental level. They were developed with the support of the project and finalized at a workshop held in Nippes on March 2- 5, 2011. The activities undertaken under this component allowed the people affected as well as vulnerable groups immediate access to basic health services and strengthened health promotion, and improved hygiene awareness and access to safe water and sanitation. 9. As for the second component, it allowed MSPP and DINEPA to build capacity, ensuring that they were prepared to tackle cholera and other possible waterborne diseases not only in the short term but also in the longer term. It also enabled close supervision of all stakeholders involved in the project on the ground: MSPP (central and departmental directorates), DINEPA (central, regional, departmental and municipal directorates), WB and FAES, and helped strengthen the Government’s capacity to manage and respond to outbreaks through the two institutions playing the most important role in the fight against cholera. 10. The second component also supported activities to strengthen epidemiological surveillance through the Division of Epidemiology, Laboratory and Research (DELR) by paying a coordinator of activities, two medical technologists for quality control of tests at central level and technicians for the epidemiological surveillance at departmental level. This helped improve data management by the DELR (including the collection, analysis and dissemination of data) as 37 well as develop the National Epidemiological Surveillance Plan, which is now available for all actors in the health system. 11. As for the DINEPA, 82 potable water and sanitation technicians for the communes (TEPAC) were set up to monitor the water quality and take part in early warning activities to minimize the impact of a cholera outbreak. DINEPA also established the Emergency Response Department (DRU) to steer emergency operations at the institutional level. 12. In addition, supervision and monitoring of NGO activities were made possible by the introduction of six monitoring officers at the level of FAES’s regional offices, who maintained permanent contact with the Departmental Health Directorates. Regular financial audits were made for these NGOs by FAES’s internal audit team. 7.2 Key Factors Affecting Implementation and Results 13. MSPP was not very involved in the design phase of the project. The Bank solely considered the emergency and did not see fit to engage MSPP in the design of the project and in the selection of NGOs. The criteria of the World Bank were to provide grants to a small number of non- public experienced providers who were already implementing activities within the context of the cholera epidemic at the level of the country’s health departments. The World Bank underestimated the complexity of the project and institutional procedures in the public administration in Haiti. MSPP was faced with a fait accompli. However, during the negotiation phase, it insisted for some activities that were not supported by NGOs to be taken into account and integrated into the implementation plan. It was not until the end of the first year of the project that FAES was able to benefit from the collaboration of MSPP officials for the recruitment of institutional and community staff. 14. Besides, various obstacles delayed the launch of NGO activities: ( i) the contracts were signed three (3) months after the effectiveness of the project; (ii) two (2) NGOs did not follow the guidelines for submission of the project document according to the framework proposed by FAES; and there were delays by NGOs to finalize the survey which needed to precede the start of activities. 15. The services provided by each of the actors revolved around two (2) key actions: community mobilization and support. Each of these providers had to develop a detailed action plan around these strategic areas of focus for the geographical areas they covered. NGO strategy was similar to MSPP’s with some variations depending on the needs identified. 16. Access to localities was a considerable challenge. Localities that were difficult to access because of their remoteness from any urbanization as well as the poor state (or complete absence) of roads were more difficult to reach in case of outbreaks of cholera. 17. Finally, the duration of NGO contracts was a challenge. It did not allow NGOs to finalize the activities as planned, which resulted in the need to extend these contracts through additional clauses (as NGOs had to carry out too many activities in a very short time). 38 18. There was little ability to pursue community interventions after the end of the project, leaving communities once again vulnerable to continued outbreaks, given that MSPP has not yet taken the necessary steps to keep the community and/or institutional teams operating. 19. With regard to all partners in the project, it is appropriate to emphasize the difficulties encountered during implementation to receive timely responses and reactions from partners: MSPP, DINEPA, Partners in Health/Zanmi Lasante, Save the Children and World Vision Haiti. 20. Changes in interlocutors and decision makers in MSPP (department managers) and FAES (general managers and assistants) resulted in planned activities being constantly called into question, slowing down implementation and thus progress towards the project’s goals. 21. The delivery of inputs and essential goods for adequate care were adversely affected, among others, by the country’s heavy administrative procedures, the limited involvement of the FAES team in the financial management of the project, the lack of networks to transport goods, the insufficient storage capacity of Health Directorates, and the lack of availability of products in sufficient numbers at the supplier level in the country. 22. Regarding Monitoring & Evaluation, the project is being continuously and rigorously monitored and supervised. Data is collected at the level of MSPP departmental directorates, who transmit it to MSPP’s Division of Epidemiology, Laboratory and Research, which is responsible for analyzing and disseminating information on diseases under surveillance, notably cholera. The data is regularly available on the MSPP website. 7.3 Evaluation of Results 23. With respect to the relevance of the objectives, the project was designed taking into account the Cholera Response Strategic Plan, as well as supporting MSPP and DINEPA activities. The WB considered the wishes of MSPP to integrate the additional activities in the project document so that most of the determinants of the epidemic be taken into account, always doing so with the officials concerned. Various MSPP documents on community mobilization and the integration of cholera response at health facility level were consulted. Activities were therefore in line with MSPP expectations. 24. In the short term, the project contributed to the immediate treatment of patients with cholera, and to the deceleration of the cholera outbreak through health and hygiene promotion campaigns and improved water and sanitation. In the medium term, the implementation of project activities helped meet needs while avoiding fragmentation of services and an even greater increase in costs, and eventually helped strengthen the foundations of a sustained control of the epidemic. The project thus initiated the transition to a stronger public health system in the long term. Finally, all project activities helped not only reduce the incidence of the disease in the country but also engage in sustainable solutions while respecting the environment. 25. The community mobilization activities conducted by the network of brigadiers deployed by MSPP’s Departmental Health Directorates to improve hygiene practices have reached seven hundred forty-six thousand three hundred ninety-one (746,391) persons and those carried by the NGO network have reached six million five hundred forty-one thousand nine hundred twenty - 39 one (6,541,921) persons, totaling seven million two hundred eighty-eight thousand three hundred twelve (7,288,312) persons across four thousand four hundred forty-seven (4447) localities in the six (6) geographic departments involved in the project. In addition to awareness activities, a total of eight hundred fifty-six thousand six hundred eighty-eight (856,688) persons have received distributions of water treatment products and/or soap. 26. Over time, NGO partners worked to constantly improve the implementation of activities under the continuous supervision of FAES and did so with great success. Despite the difficulties of implementation, NGO partners were able to reach a large number of target populations with prevention/education messages, providing them with water treatment and hygiene products. 27. From the outset, the availability of a Project Operations Manual along with FAES’s manual, made it possible to align the implementation with the standards and procedures of these two institutions. In addition, a Monitoring Manual was developed and served as a guide for both technical and financial supervision. 28. Regarding project implementation, FAES always aligned itself with MSPP, embracing the latter’s pace regarding the assimilation of procedures to be followed. 29. Project results and development objectives are reflected in the following tables: Project Indicators Baseline 2011 Targeted Result Result achieved Percentage of population in project intervention areas who know the cholera early warning symptoms and prevention steps. 64% 13% 69% 18% 82% 14% Percentage of population in project intervention areas with access to improved water sources. 78% 78% 79% Increased institutional capacity as measured by achievement of at least four out of five of Component 2 indicators on an annual basis. 0 4 4 Component 1 Indicators Baseline 2011 Targeted Result Result achieved Number of communities that have received targeted health education and/or prevention activities. 0 2213 4447 Oral rehydration posts in service and supported with supplies. 0 223 256 Cholera Treatment Units (CTU) in service and supported with supplies. 0 81 Number of people receiving water treatment products and soap 0 178 400 856 688 Component 2 Indicators Baseline Targeted Result Result achieved Number of institutional personnel in place and trained. 0 2063 4325 Departmental Cholera Management Plan developed (by MSPP and DINEPA) for at least five departments. 0 5 6 Establishment of a national coordinator and five regional coordinators to monitor the response to cholera. 0 6 7 Good quality quarterly IFRs are submitted to the Bank. 0 10 13 Source: Baseline inquiry 40 30. The implementation plan calculated the provisional budget for the activities. It was revised as needed and according to the course of activities keeping the same budget. This flexibility allowed the MSPP to carry out significant activities. 31. To achieve these objectives, FAES supported each partner in the development of their strategic action plan, including the need for human, material and financial resources, which were clearly inventoried around the two strategic areas of focus mentioned earlier: community mobilization and support. These plans were revised and activities were consistently and thoroughly monitored by both central and regional teams. Project provisional financing table by component and implementation year Component and/or activity Year 1 2011 Year 2 2012 Year 3 2013 Total (millions USD) Component 1: Support to the Government’s Response to Cholera at the Decentralized Level. 9.0 2.0 0.0 11.0 Component 2: Emergency Response Capacity Building. 1.5 2.0 0.5 4.0 Total for the project 10.5 4.0 0.5 15.0 Table of actual expenditures by budget year Component and/or activity Year 1 2011 Year 2 2012 Year 3 2013 Year 4 2014 Total (USD) Component 1: Support to the Government’s Response to Cholera at the Decentralized Level. 3,277,748.97 4,644,135.70 2,729,736.75 357,762.89 11,009,384.31 Component 2: Emergency Response Capacity Building. 214,944.17 776,939.15 2,343,327.21 612,143.65 3,947,354.18 Total for the project 3,492,693.14 5,421,074.85 5,073,063.96 969,906.54 14,956,738.49 Note: This report is not exhaustive because some payments already committed by March 31, 2014 were not included at that date. 32. The project was financed by an IDA Grant through an Emergency Recovery Grant of SDR 9.9 million (USD 15 million equivalent) over an implementation period of two and a half years. Project financing by Component is outlined in the Table below. The Closing date was June 30, 2013. An extension was granted until December 31, 2013 and was revised to March 30, 2014. 7.4 Evaluation of the Performance of the Bank and the Borrower 33. FAES succeeded in managing this complex project with different partners. Despite delays, very complex procurement processes turned out well and inputs planned for by the project could be acquired and handed over to those concerned. 41 34. The number of projects (about twenty) managed by FAES exceeded the capacity of the administrative and financial teams. This prevented them from playing their full role and from carrying out procedures within the agreed timeframe. The PRUC did not really benefit from all the attention needed. This led to extending the closing date 3 times. 35. Open and continuous dialogue with the Bank throughout project implementation, prioritization of a certain flexibility in solving the problems encountered during the project, support in the implementation of complex procedures and close monitoring for implementation during the different missions, allowed to overcome difficulties in order to achieve the intended objectives. 36. From the outset, the project design took into account lessons learned from previous operations in Haiti and from World Bank experience with emergency response operations, including keeping project objectives simple, coordinating closely with all partners, using existing implementing agencies, and limiting the number of actors. Lessons learned from previous projects indicate that it is important to link the emergency projects with previous and future technical assistance to increase, inter alia, its sustainability. Thus, Component 2 of the project included emergency response capacity building for MSPP and DINEPA. 7.5 Risk Assessment of Development Outcomes 37. The main risks identified were the following: generally weak financial management capacity in Haiti, complex control of activities as project activities were carried out by public and non-public service providers, as well as the complexity arising from the need to coordinate public and non- public service providers, potentially leading to poor supervision and monitoring of the use of project funds for intended purposes. 38. However, adaptation measures were put in place: as planned, FAES maintained internal control procedures, incorporated service provider obligations and possible solutions regarding the financial management aspects of their operations into the contracts signed by non-public service providers, and ensured that partner revised their implementation plans to adapt them according to the needs. 39. FAES and its regional teams were able to carry out technical and financial supervision of departmental level teams as well as day-to-day management of public partners’ activities, non- public service provider contracts, and the monitoring of technical assistance activities under Component 2. FAES is well versed in preparing quarterly reports on the progress of the project to be submitted to the Bank team, monitoring compliance with World Bank environmental safeguard policies, maintaining records and separate accounts for all transactions related to the project, and preparing, consolidating and producing the project’s financial statements and other financial information. Regarding implementation of the project, FAES always aligned itself on MSPP and DINEPA adopting the latter’s pace regarding the integration of procedures to be followed. 40. To mitigate risks, open and ongoing dialogue and listening, as well as close technical assistance were established by the donor. Moreover, the project used a tried and tested approach in an emergency context in which the potential incremental scale of environmental impacts was 42 minor and easily localized, while impact mitigation measures were already a standard practice among the service providers. However, given the sensitivity of some of the interventions, particularly in terms of managing healthcare waste within and from healthcare facilities, and the inclusion of minor works, the project supported multiple, small-scale activities to prevent the spread of cholera through enhancing access to hygiene information and clean water, and facilitating treatment for infected individuals as well as waste management. At the request of MSPP, eight incinerators that had been donated to MSPP were thus installed and rendered functional. Potential adverse impacts (direct, indirect, and cumulative) related to waste management from healthcare facilities, worker health and safety and minor works were identified, minimized, mitigated and avoided under the guidelines provided in the Environmental and Social Management Framework (ESMF). The ESMF is based on good practices identified in the SFI Environmental, Health and Safety Guidelines for Health Care Facilities. 7.6 Lessons learned 41. During the implementation of this project, lessons were learned and notable successes were achieved. One such notable success was the project’s contribution to the reduction of all cholera cases during the intervention period. Although this result cannot solely be attributed to the achievements of this Fund (as other factors and other partners also influenced it) it is a positive result the project largely contributed to achieving. 42. In addition, NGO partners noted that this project enabled them to reinforce/create community networks, and to better understand the challenges of budgeting for other similar projects, as well as the mechanisms of the administration of a government agency with which they had never worked before. 43. At project inception, a deficiency of information was noted from the Departmental Health Authority toward the institutions with which FAES was to implement activities. Project information should therefore be transmitted at all levels of implementation as early as implantation and throughout the project on a regular basis for all partners. This deficiency of information did not allow Departmental Health Authorities officers to participate in audits of NGO expenditures. 44. Baseline and ex-post surveys in areas where NGOs were involved were carried out by these NGOs for the departments of the Centre and the South- East. NGOs should not be the judge and jury in surveys or studies aimed at providing project baseline or impact information used to measure their performance. 45. FAES minimized the involvement of the project management team in the execution of activities. A lesson learned from this project to take into account for the prepar ation and implementation of other emergency projects relates to human resources. Staffing needs for this operation were not properly assessed, which resulted in the carrying out of financial management related tasks being very cumbersome. World Bank presented a proje ct budget that included the provision of a restricted list of personnel at FAES’s level. It would have been advisable to discuss human resources needs before implementation, while taking into account the reality of Haiti’s existing structures and/or constraints to establish inter-ministerial or inter-organizational teams that could eventually expedite decision making as well as execution of procedures. FAES 43 should therefore revise its institutional procedures to allow for more timely records processing, and upon signing the Subsidiary Agreement prepare the directorates concerned to take ownership of the project, which would also reduce the time needed to monitor records. This should not only be done at the level of FAES but also with partners. 46. Before implementation, the collaborative relationship with MSPP should be improved. FAES is used to launching projects over two or three days in the presence of all stakeholders in order to identify the criteria for success. This was not done in the case of this emergency project. Initially the only meeting with MSPP was to finalize cholera response departmental plans for the six departments targeted. A constructive and regular dialogue must be maintained with all concerned. 47. In order to achieve expected results, open and continuous listening and dialogue with the World Bank was necessary. This gave the project its flexibility, which played a role in its success. 7.7 Conclusion and Recommendations 48. With the idea of ensuring the durability of certain achievements of the Proj ect de Réponse D’Urgence Au Choléra (PRUC), an Interventions Monitoring Plan would be needed in order to keep strengthening waste management activities with MSPP. FAES’s Environmental Unit team, which was very involved, should intervene in the PASMISSI to continue to assist MSPP with this issue. 7.8 Reference Documents 1. Grant Agreement 2. Subsidiary Agreement 3. Signed Memoranda 4. Project Document 5. Operations Manuals 6. PRUC Monitoring and Evaluation Manual 7. Final Report of the activities of the various partners 8. Final Monitoring and Evaluation Report 9. Environmental and Social Management Report 44 Comments from Recipient Port-au-Prince, September 12, 2014 Mrs. Mary A. Barton- Dock Special Envoy to Haiti, Latin America and the Caribbean 7, Rue Ogé Pétion-Ville, Haiti Ref.: Commentary to Report No 89201- HT dated September 4, 2014: IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA- H639- 0-HT) ON A GRANT IN THE AMOUNT OF SDR 9.9 MILLION (US$15 MILLION EQUIVALENT) TO THE REPUBLIC OF HAITI FOR A CHOLERA EMERGENCY RESPONSE PROJECT Dear Mrs. Barton-Dock, The Government of Haiti and the implementing agency, FAES, would like to thank you for the opportunity to review the project report “Implementation Completion and Results Report (Ida- H639- 0-Ht) on a grant in the amount Of SDR 9.9 Million (US$15 Million equivalent) to the Republic of Haiti for a Cholera Emergency Response Project”. Please find enclosed the comments we would like to include in above mentioned report at the time of its publication. The Government of Haiti and the People of Haiti express their gratitude for this grant, which has proven to be a very helpful contribution at aiming to eradicate Cholera in Haiti, and is looking forward to a continued and fruitful relationship with the World Bank. Marie Carmelle Jean -Marie Minister of Economy and Finance 45 Fonds d’Assistance Économique et Sociale (FAES) Commentary to Report No: 89201- HT dated September 4, 2014 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA- H639- 0-HT) ON A GRANT IN THE AMOUNT OF SDR 9.9 MILLION (US$15 MILLION EQUIVALENT) TO THE REPUBLIC OF HAITI FOR A CHOLERA EMERGENCY RESPONSE PROJECT (PRUC) September 12, 2014 The Government of Haiti would like to express its gratitude towards the lender to have been trusted with the above-mentioned project aiming to support the Government of Haiti and build Emergency Response Capacity to respond to Cholera. The Completion Report was read and analyzed by the PRUC project teams at FAES, DINEPA and MSPP. Whereas DINEPA and MSPP have no further comments, FAES submits following commentary pertaining to the respective chapters described below: 2.4 Safeguard and Fiduciary Compliance 32: Financial Management arrangements in terms of accounting, budgeting, fund flow, internal control, external audit and financial reporting were performed Moderately Unsatisfactory in accordance with the last ISR. While recognizing weaknesses in the withdrawal applications, FAES would like to stress the fact that access to the World Bank’s financial management system “CLIENT CONNECTION” was met with recurrent problems, making it at times impossible for FAES to enter the system. Several attempts, over a period of time, by the World Bank’s team to remediate the difficulties to access the system resulted in delays in submitting the withdrawal requests and/or supporting materials. Thus, the delays in disbursements were in part due to the problems World Bank was having with their ill functioning CLIENT CONNECTION system. 3.2 Achievement of Project Development Objectives 41. By Project completion, access to improved water sources had been slightly increased and sanitary and hygiene facilities as well as the collection of solid waste had been improved. “The Project contributed to the waste management and environmental protection of the country through the installation of four incinerators, the emptying of four septic tanks, and the construction of three ecological sanitary blocs.” FAES would like to point out that four (4) incinerators were installed by the NGO, and in addition eight (8) incinerators were installed for MSPP (totaling 12 incinerators installed). 5.2 Borrower Performance (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 46 52: The rating for FAES performance is similar to that of the overall Government since similar issues apply. 1. With reference to “insufficient human resources and capacities” , FAES would like to point out that during the initial project design World Bank had not taken into account the actual HR situation in Haiti. The complexity of components of this project, involving various government entities, each with structures and procedures of their own, would have required a larger project team with a variety of skills. Although FAES pointed this out to the World Bank team, taking it into account seemed not have been possible. 2. “The lack of a robust health management information system at all levels remains one of the Project's weakest areas.” This remark is less pertinent to FAES than to MSPP. Annex 7. Summary of Borrower’s ICR and/or Comments on Draft ICR Completion Report Cholera Emergency Response Project Section 4: Evaluation of the Performance of the Bank and the Borrower 6. Lessons learned With reference to “Staffing needs were not properly assessed”, FAES points out that World Bank presented a project budget that included the provision of a restricted list of personnel at FAES’s level. It would have been advisable to discuss HR needs before implementation, while taking into account the reality of Haiti’s existing structures and/or constraints to establish inter-ministerial or inter-organizational teams that could eventually expedite decision making as well as execution of procedures. 47 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A 48 Annex 9. List of Supporting Documents Key Project documents: Emergency Project Paper: World Bank. 2010. Haiti - Cholera Emergency Response Project Restructuring Paper: World Bank. June 4, 2013. World Bank. December 27, 2013. Financing agreements: Ronald Baudin, Alexandre V. Abrantes. 2011. Financing Agreement, Grant Number H639-0-HT. Conformed Copy. Other Project documents: Manuel de Suivi du Projet de Réponse Urgente au Choléra. Décembre 2012. Direction Suivi et Evaluation – DES – FAES. Accord de Rétrocession entre MEF et FAES. Janvier 2011. WBDocs. Rapport d’analyse des données de l’enquête finale. Projet de Réponse d’Urgence au Choléra. Avril 2013. Centre de Recherches et de Développement. Manuel d’Opération du Projet de Réponse d’Urgence au Choléra. Juillet 2012. FAES Rapport de fin de participation. Janvier 2014. Direction d’Epidémiologie, de Laboratoires et de Recherche. Rapport final de Projet. Avril 2013. Direction Sanitaire de l’Artibonite. Rapport final de Projet. Mai 2013. Direction Sanitaire du Centre. Rapport final de Projet. Juin 2013. Direction Sanitaire du Grand Anse. Rapport final de Projet. Avril 2013. Direction Sanitaire du Nippes Rapport final de Projet. Avril 2013. Direction Sanitaire du Sud Est Rapport final de Projet. Avril 2013. Direction Sanitaire du Sud Rapport final de Prestation. Mai 2012. PIH/ZL. Rapport final OPS PRUC. Mai 2012. SV. Rapport final de Prestation. Decembre 2012. WV. 49 In addition: v arious Aide Memoires, Implementation Supervision Reports (ISRs), Procurement reports, Financial Management reports. Policy, Strategy, and Program Documents from the Government of Haiti Growth and Poverty Reduction Strategy Paper 2008- 2010. November 2007. Ministry of Planning and External Cooperation Cadre de Gestion Environnementale et Sociale. Fevrier 2011. FAES. National Plan for the Elimination of Cholera in Haiti 2013- 2022. February 2013. Ministry of Public Health and Population; National Directorate for Water Supply and Sanitation Population and Health Data Global Water, Sanitation and Hygiene Strategy – CDC. Jan 14 2014. http://www.cdc.gov/healthywater/global/index.html Prevention and control of cholera outbreaks: WHO policy and recommendations. Jan 12 2014. http://www.who.int/cholera/technical/prevention/control/en/ World Bank. Open Data. http://data.worldbank.org/country/haiti Stratégie de réponse pour la prise en charge du choléra. December 2010. Save the Children 50 Annex 10. Project contribution to the reduction of cholera incidence and mortality 1. Between 2010 and 2013, t he cholera case fatality rate (CFR) in Project areas decreased by 1.8 percentage points, compared with 1.2 percentage points decrease at the national level. This is a notable progress considering the high CFR in 2010 in the Project areas as compared with the one at the national level (Figure 2). In other words, the decline is more noticeable in the Project areas than at the national level. This tends to confirm the effectiveness of access to adequate treatment and quality of case management which the P roject contributed to establishing 174 oral rehydration points and 82 cholera treatment units at the community level. Actually, the number of cases and the CFR at national and at the Project level have the same trend. There was an increase in the number of cases and CFR between 2010 and 2011 and a decrease in 2012 and 2013. T he increase in the number of cases and CFR more or less overlapped with the increase of rainfall . 20 In addition, P roject intervention areas were those most affected by the cholera outbreak. In the Project areas, there was a relative increase of the CFR in 2012 compared with the 2011 (from 0.8 to 1.2 percent) and with the national level (0.9 percent). However, due to the high number of the detected cases, the CFR actually decreased by more than half in absolute terms (1,109 to 451). The Table 10.1 clearly shows the situation. Figure 2. Comparison of number of cases of cholera in Haiti (National and Project), from 2010 to 2014 Source: Profil Statistique du Cholera Semaine 22 epidemiologique, Rapport du réseau national de surveillance, MSPP, DELR - 31 May 2014 FAES - M&E Directorate March 2014; WHO/PAHO Epidemiological update - Cholera, March 20, 2014 20 Report of Public Assistance Hopital of Marseille on the situation of the elimination of the cholera in May 31, 2014. - 50,000 100,000 150,000 200,000 250,000 300,000 350,000 400,000 2010 2011 2012 2013 2014 number of cases number of cases Project number of cases 51 Table 10.1. Comparison of cholera CFR in Haiti (National and Project), from 2010 to 2014 2010 2011 2012 2013 2014 Case-Fatality Rate (%) 2.2 0.8 0.9 1.0 0.7 Project Case-Fatality Rate (%) 2.7 0.8 1.2 0.9 0.5 Project Death number 1,986 1,109 451 177 18 Source: Profil Statistique du Cholera Semaine 22 epidemiologique, Rapport du réseau national de surveillance, MSPP, DELR - 31 May 2014 FAES - M&E Directorate March 2014; WHO/PAHO Epidemiological update - Cholera, March 20, 2014 and authors' calculation. 2. While the CFR at the national and the Project level are below one percent in 2013 and 2014, this should be held steady in a sustainable way to prevent a new cholera outbreak, particularly during rainy seasons and hurricane periods. Consequently, the international community and the GOH should remain vigilant in implementing appropriate programs relevant to the current situation in which epidemiological surveillance is critical. 52 Annex 11. MAP IBRD 33471R 53