Rapport de fin d'exécution et de résultats - Améliorer la santé maternelle et infantile grâce aux services sociaux intégrés

Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005403 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON GRANT S H864 - 0 - HT, D203 - 0 - HT AND TF014474 IN THE AMOUNT OF SDR 46.70 MILLION (US$ 70 MILLION EQUIVALENT) IN THE AMOUNT OF SDR 18.3 MILLION ( US$ 25 MILLION EQUIVALENT) AND A HEALTH RESULTS INNOVATION TRUST FUND GRANT IN THE AMOUNT OF US$20 MILLION TO THE REPUBLIC OF HAITI FOR THE IMPROVING MATERNAL AND CHILD HEALTH THROUGH INTEGRATED SOCIAL SERVICES May 28, 202 1 Health, Nutrition & Population Global Practice Latin America a nd Caribbean Regio n Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized CURRENCY EQUIVALENTS (Exchange Rate Effective January 27, 2021 ) Currency Unit = Haitian Gourde - HTG HTG1 = US$ .0014 US$ 1 = 75.5 HTH FISCAL YEAR October 1 - September 3 0 Regional Vice President: Carlos Felipe Jaramillo Country Director: Tahseen Sayed Khan Regional Director: Luis Benveniste Practice Manager: Michele Gragnolati Task Team Leader(s): Andrew Sunil Rajkumar, Briana N. Wilson, Nicolas Collin Dit De Montesson ICR Main Contributor : Maria Cecilia Zanetta ABBREVIATIONS AND ACRONYMS AF Additional Financing ASPIRE The Adaptive Social Protection for Increased Resilience Project BCR Benefit - Cost Ratio BSEIPH Bureau of the Secretary of State for the Integration of Persons with Disabilit ies ( Bureau de la Secrétaire d’Etat a l’Intégration des Personnes Handicapées) CDAI Departmental Supply Center ( Centre Départemental d’Approvisionnement en Intrants ) CDC Center for Disease Control CDP Departmental Steering Committee ( Comité Départemental de Pilotage ) CU Contracting Unit CPF Country Partnership Framework DDS Departmental Health Directorate (Direction Départementale de la Santé) DELR Epidemiology, Laboratory, and Research Directorate (Direction d’Epidémiologie, de Laboratoires, de Recherches ) DHS Demographic and Health Survey DINEPA National Water and Sanitation Directorate ( Direction Nationale de l'Eau Potable et de l'Assainissement ) DPSPE Health and Environmental Protection Directorate (Direction de Promotion de la Santé et de Protection de l´Environnement ) EMIRA Mobile Rapid Response Teams (Équipes Mobiles d'intervention Rapide) ESMF Environmental and Social Management Framework FAES Economic and Social Assistance Fund ( Fonds d'Assistance Economique et Sociale) FM Financial Management GDP Gross Domestic Product GoH Government of Haiti GRM Grievance Redress Mechanism ICR Implementation Completion and Results Report IDA International Development Association IDB Inter - American Development Bank IHSI Haitian Institute of Statistics and Informatics ( Institute Haitien de Statistique et d’Informatique ) IMF International Monetary Fund IRI Intermediate Results Indicator ISR Implementation Status Report KF Kore Fanmi LNSP National Laboratory for Public Health M&E Monitoring and E valuation MAST Ministry of Labor and Social Affairs ( Ministère des Affaires Sociales et du Travail) MCH Maternal and Child Health MSPP Ministry of Public Health and Population ( Ministère de la Santé Publique et de la Population ) NGO Non - government O rganization NPV Net Present Value PAHO Pan - American Health Organization PASMISSI Improving Maternal and Child Health Through Integrated Social Services ( Projet d'Amélioration de la Santé Maternelle et Infantile à travers des Services Sociaux Intégrés) PDI Project Development Indicator PDO Project Development Objective PIU Project Im plementation Unit PLR Performance and Learning Review PMCHNS Package of Maternal and Child Health and Nutrition Services PNPPS National Policy on Social Protection and Promotion (Politique Nationale de P rotection et de P romotion S ociales) PRF Project Results Framework PROSYS Strengthening Primary Health Care and Surveillance in Haiti Project ( Projet de Renforcement des Soins de Santé Primaire et de la Surveillance en Haïti ) PwDs Persons with Disabilities RBF Results - Based Financing SBR Single Beneficiary Registry SIMAST Integrated Beneficiary Registry ( Système d’Information du MAST) SISNU Single Sanitary Information System ( Système d'Information Sanitaire Unique ) SP Social Protection SPST Social Protection Sectoral Table SSR Single Social Registry TA Technical Assistance TTL Task Team Leader UN United Nations UNICEF United Nations Children's Fund UN FPA United Nations ’ Population Fund USAID United States Agency for International Development WB World Bank WFP U nited N ations' World Food Programme WHO World Health Organizatio n TABLE OF CONTENTS DATA SHEET ................................ ................................ ................................ ................................ 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ................................ ......................... 6 A. CONTEXT AT APPRAISAL ................................ ................................ ................................ .......... 6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................ ............................. 10 II. OUTCOME ................................ ................................ ................................ ......................... 13 A. RELEVANCE OF PDOs ................................ ................................ ................................ ............. 13 B. ACHIEVEMENT OF PDOs (EFFICACY) ................................ ................................ ...................... 14 C. EFFICIENCY ................................ ................................ ................................ ............................ 22 D. JUSTIFICATION OF OVERALL OUTCOME RATING ................................ ................................ ... 23 E. OTHER OUTCOMES AND IMPACTS ................................ ................................ ......................... 24 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ .. 25 A. KEY FACTORS DURING PREPARATION ................................ ................................ ................... 25 B. KEY FACTORS DURING IMPLEMENTATION ................................ ................................ ............ 26 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ... 28 A. QUALITY OF MONITORING AND EVALUATION (M&E) ................................ ........................... 28 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ................................ ................... 29 C. BANK PERFORMANCE ................................ ................................ ................................ ............ 30 D. RISK TO DEVELOPMENT OUTCOME ................................ ................................ ....................... 32 V. LESSONS AND RECOMMENDATIONS ................................ ................................ ................. 33 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ................................ .............................. 35 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ........................... 52 ANNEX 3. PROJECT COST BY COMPONENT ................................ ................................ ............... 54 ANNEX 4. EFFICIENCY ANALYSIS ................................ ................................ ............................... 55 ANNEX 5. RECIPIENT, CO - FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ....... 57 ANNEX 6. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................ ................ 85 The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 1 of 87 DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P123706 Improving Maternal and Child Health through Integrated Social Services Country Financing Instrument Haiti Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of Haiti Fonds d'Assistance Economique et Sociale (FAES), Ministry of Public Health and Population (MSPP), Institut Haïtien de Statistique et d’Informatique Project Development Objective (PDO) Original PDO The objective of the proposed Project is to increase the access and use of maternal and child health, nutrition and other social services in the Recipient’s territory. The Project will support services in at least three Departments with a total catchment population of around 1.8 million people, targeting pregnant women,chi ldren under five and vulnerable families.Progress on the objectives of the Project will be measured by the following:(i) percent of children under five immunized; (ii) percent of institutional deliveries; (iii) contraceptive prevalence rate; and (iv) decr ease in percentage of families categorized as extremely vulnerable. Revised PDO To increase the access and use of maternal and child health services, strengthen cholera control, and improve targeting of social services in the Recipientâ €™ s territory, with a particular focus on areas affected by Hurricane Matthew. The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 2 of 87 FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing TF - 13431 850,000 818,530 818,530 IDA - H8640 70,000,000 70,000,000 65,129,356 TF - 14474 20,000,000 20,000,000 16,235,000 IDA - D2030 25,000,000 25,000,000 24,906,415 Total 115,850,000 115,818,530 107,089,301 Non - World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 115,850,000 115,818,530 107,089,302 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 21 - May - 2013 20 - Apr - 2013 31 - Oct - 2018 31 - Dec - 2018 30 - Sep - 2020 The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 3 of 87 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 29 - Mar - 2017 30.95 Change in Implementing Agency Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Disbursements Arrangements Change in Legal Covenants Change in Institutional Arrangements Change in Financial Management Change in Procurement Change in Implementation Schedule 14 - Jun - 2017 30.95 Additional Financing Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Disbursements Arrangements Change in Safeguard Policies Triggered Change in Procurement 19 - Dec - 2019 97.05 Change in Loan Closing Date(s) Reallocation between Disbursement Categories 24 - Mar - 2020 103.95 Change in Loan Closing Date(s) Reallocation between Disbursement Categories KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Moderately Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs No. Date ISR Archived DO Rating IP Rating Actual Disbursements (US$M) 01 02 - Sep - 2013 Satisfactory Satisfactory .40 02 28 - Apr - 2014 Satisfactory Moderately Satisfactory 1.85 03 14 - Nov - 2014 Satisfactory Moderately Satisfactory 3.85 The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 4 of 87 04 16 - May - 2015 Moderately Satisfactory Moderately Satisfactory 10.00 05 28 - Dec - 2015 Moderately Satisfactory Moderately Satisfactory 15.04 06 30 - Jun - 2016 Moderately Satisfactory Moderately Satisfactory 19.48 07 28 - Dec - 2016 Moderately Satisfactory Moderately Satisfactory 26.56 08 30 - Jun - 2017 Moderately Satisfactory Moderately Satisfactory 33.70 09 28 - Dec - 2017 Satisfactory Moderately Satisfactory 46.11 10 29 - Jun - 2018 Satisfactory Moderately Satisfactory 56.42 11 31 - Dec - 2018 Satisfactory Moderately Satisfactory 71.93 12 28 - Jun - 2019 Satisfactory Moderately Satisfactory 89.30 13 25 - Dec - 2019 Satisfactory Moderately Satisfactory 97.04 14 30 - Jun - 2020 Satisfactory Moderately Satisfactory 105.89 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 13 Sub - National Government 13 Health 59 Public Administration - Health 9 Health 50 Social Protection 50 Social Protection 50 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 5 of 87 Social Development and Protection 20 Social Protection 20 Social Safety Nets 20 Human Development and Gender 0 Disease Control 0 Pandemic Response 1 Health Systems and Policies 60 Reproductive and Maternal Health 30 Child Health 30 Nutrition and Food Security 20 Nutrition 10 Food Security 10 ADM STAFF Role At Approval At ICR Regional Vice President: Hasan A. Tuluy Carlos Felipe Jaramillo Country Director: Alexandre V. Abrantes Tahseen Sayed Khan Director: Keith E. Hansen Luis Benveniste Practice Manager: Joana Godinho Michele Gragnolati Task Team Leader(s): Maryanne Sharp, Francesca Lamanna Andrew Sunil Rajkumar, Briana N. Wilson, Nicolas Antoine Robert Collin Dit De Montesson ICR Contributing Author: Maria Cecilia Zanetta The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 6 of 87 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Country Context 1. Three years after being hit by a devastating earthquake, 1 the Government of Haiti (GoH) had begun to emerge from the catastrophic aftermath and return its attention to the country's structural problems. With a G ross Domestic Product (G DP ) per capita of US$726 in 2011 and a GINI coefficient of 0.59, Haiti was one of the poorest, most unequal countries in the world. Over half of its population of 10 million was estimated to live on less than US$1 per day, 78 percent on less than US$2 per day, and 40 percent were categorized as food insecure. The country also performed poorly on the non - income dimensions of poverty, ranking 158th out of 187 in the 2011 Human Development Index. 2 Sectoral and Institutional Context 2. Ten months after the earthquake hit, a severe cholera outbreak placed additional pressure on the already fragile health system, further compromising the welfare and health status of the population. As of January 2013, nearly 650,000 cases of cholera had been reported with almost 8,000 attributable deaths, mak ing it the largest epidemic ever recorded in a single country. Haiti's already fragile public health system was confronted with the challenge of treating patients with health workers who lacked experience in managing cholera cases. Moreover, due to separat e funds for cholera prevention and treatment, parallel emergency responses systems were put into place in an unstructured manner. In response, the Ministry of Public Health and Population ( Ministère de la Sante Publique et de la Population - MSPP) launched the National Plan for the Elimination of Cholera ( Plan d' Elimination du Cholera ) , which sought to integrate cholera response activities back into the public health system to improve efficiency and sustainability and ultimately stop the secondary transmis sion of cholera in Haiti. 3. Haiti also faced serious challenges ensuring access to basic health services, as reflected in its high child mortality and malnutrition rates. While under - five mortality had decreased from 152 per 1,000 live births in the 1990s, as of 2012 the rate remained high at 87 per 1,000 live births -- three times the regional average. 3 Moreover, children from the poorest households faced a mortality rate more than double that of children from the richest households . Malnutrition rates had also stagnated since 2000, with one quarter of newborns exhibiting low birth weight, nearly one - third of children under five suffering from stunted growth and three - quarters of children 6 - 24 months being anemic. T he incidence of di arrheal diseases – a key contributor to child mortality – was high among children, especially those between six months and two years old (39 percent), and in rural areas (25 percent). 4 4. Despite the efforts of the Haitian authorities, maternal mortality w as still the highest in the region at 630 per 100,000 live births (six times the regional average) , and access to family planning services was low . During childbearing years, a Haitian woman had a 1 in 37 probability of dying from maternal causes. 5 The nutritional status of women of childbearing age was of particular concern, since underweight and anemia contributed to the cycle of 1 In January 12, 2010 , a 7.0 Mw earthquake hit 25 miles of Port - au - Prince, the capital of Haiti. It caused great devastation , with over 300,000 officially reported deaths and 1.5 million people displaced. 2 World Bank (2013), Haiti - I mproving Maternal and Child Health through Integrated Social Services ; Project Appraisal Document ; Report No. 67945 - HT ; April 22, 2013; Washington, D.C. 3 MSPP Haiti D HS 2012. Preliminary Report. 4 DHS 2005/06 was the latest source of data on MCH indicators at A ppraisal. 5 DHS 2005/06. The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 7 of 87 inter - generational under - nutrition and the risk of maternal death during childbirth . Although on the decline, Haiti als o had the highest fertility rate in the Americas ; access to family planning services remained low. 5. Low cov erage rates of key maternal and child health (MCH) interventions played a key role in Haiti's poor MCH outcomes. On the supply side, key challenges included low access and quality of health services as well as the GoH's difficulties in coordinating service providers. The latter challenge was compounded by the large array of organizations involved in health service provision, which resulted in a fragmented health and social system with a myriad of standards and implementation mechanisms. Within the G o H , the MSPP was the primary institutional actor in the provision of maternal and child services. Concomitantly, the Economic and Social Assistance Fund ( Fonds d'Assistance Economique et Sociale - FAES) was implementing the Kore Fanmi (KF) pilot initiative, 6 which sought to improve service delivery -- maternal and child care, nutrition and social services in general -- through the use of polyvalent household agents. T his initiative was conceived in the aftermath of the earthquake as an attempt to link households with the emergenc y health and social services being provided a significant number of external agencies, bilateral aid and non - government organizations (NGOs) to Haiti. While NGOs helped ensure the continuation of service delivery, they did not necessarily increase access o r address existing barriers. Overcoming these issues required improving the quality and coverage of services at the institutional level, bridging the gap between families and service providers, and strengthening the GoH's stewardship through a focus on res ults and the efficient use of resources to improve health outcomes. 6. On the demand side, financial constraints were one of the most important barrier s to service utilization across socio - economic quintiles and particularly among women. Of those who were seriously sick and did not seek treatment (24 percent of all those who reported being sick) in the 30 days preceding the 2005/06 Demographic Health Surv ey (DHS), almost half cited financial reasons and 20 percent, physical accessibility. Financial barriers posed a greater hurdle for poor women in rural areas, with eight out of ten women citing financial difficulties when seeking health care, with the prop ortion increasing even more among the poorest (92 percent). These demand - side barriers and social determinants of health had to be addressed at community and household level to help improve MCH outcomes, particularly for the poor. Theory of Change (Result s Chain) 7. The operation aimed to support the GoH's efforts to reduce maternal and child mortality by improving the supply of MCH and other essential social services while simultaneously stimulating the demand. The Project supported the delivery of a package of preventative MCH and nutrition services proven to have an impact on maternal and child mortality by establishing results - based payment agreements with eligible public and non - public providers (the "health stream"). Concomitantly, the Project provided support to polyvalent community agents, called KF agents, for the delivery of health, nutrition services and other social services at the community and household levels. In addition, KF agents were expected to strengthen linkages between households (particularly the most vulnerable ones) and health and social services in order to stimulate the demand (the "social protection (SP) stream") and to provide conditional cash transfer grants to eligible beneficiaries. In addition, the operation provided support for the institutional development of the two main implementing agencies -- MSPP and FAES. A more detailed mapping of the Results Chain underlying the operation is shown on Figure 1. 6 The K F initiative ( "family support" in Creole ) was implemented under FAES under the supervision of the supervision of Haiti's Ministry of Economy and Finance. The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 8 of 87 Figure 1. Original Project Results Chain Note: Project Outputs are indicated in bullet points under the corresponding Intermediate Outcomes. Please see below for the underlying critical assumptions. 8. There were several critical assumptions underlying the Results Chain , notably : i) the effectiveness of results - based payments to incentiv ize the provision of quality MCH , nutrition and social services by both health providers and K F agents; ii) adequate implementation and M onitoring and Evaluat ion (M &E ) capacity on the part of both MSPP and FAES when complemented with the additional support envisioned under the operation; and iii) sustained commitment toward the coordinated provision and reporting of health and nutrition services on the part of MSPP and FAES. The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 9 of 87 Project Development Objectives (PDOs) 9. The Project 's development objective was to increase the access and use of MCH , nutrition and other social services in the Recipient's territory (defined as at least three Departments with a total catchment population of around 1.8 million people, targeting pregnant women, children under five and vulnerable families). Key Expected Outcomes and Outcome Indicators 10. PDO achievement was to be captured by the following Project Development Indicators (PDIs): • PDI 1 - Children under five immunized (Percentage) - Increase in the percentage of children under five immunized from 46.22 in 2012 to 49 percent in 2018. • PDI 2 - Institutional deliveries (Percentage) - Increase in the percentage of institutional deliveries from 20.38 in 2012 to 22 percent in 2018. • PDI 3 - Contraceptive prevalence rate (Percentage) - Increase in the percentage of contraceptive prevalence from 21.98 in 2012 to 24 percent in 2018. • PDI 4 - Decrease in percentage of families categorized as extremely vulnerable (Percentage) - Eight percent decrease in the percentage of families categorized as extremely vulnerable betwee n 2012 and 2018. Components 11. Component 1: Providing MCH Health, Nutrition and Social Services (US$81 million, equivalent to 90 percent of total grant proceeds). This Component included two subcomponents: • Subcomponent 1.1: Performance - based MCH and Nutrition Service Delivery (US$64 million, equivalent to 71 percent of total grant proceeds). This subcomponent provided financial support to the MSPP to carry out three sets of activities aimed at: i) improving the quality and supply of MCH services of selected public health providers; ii) maintaining and strengthening external controls (i.e., third - party verification) in terms of quantity and quality of Packages of M CH and Nutrition Services (PMCHNSs) being provided under iii); and Results - Based Financing (RBF) for the delivery of PMCHNSs 7 and supporting monitoring and supervision by departmental health authorities. • Subcomponent 1.2: Results - oriented Family Suppo rt for Poor and Vulnerable Families (US$17 million, equivalent to 19 percent of total grant proceeds). This subcomponent provided financial support to FAES for the delivery of social services through family support to poor and vulnerable families by KF agents , including : i) goods, consultant services, training and operating costs to support the provision of basic social services at the household level; and ii) conditional cash transfer grants to eligible beneficiaries. 12. Component 2: Strengthening the Stewardship and Management Capacity of Government (US$9 million, equivalent to 10 percent of total grant proceeds). This Component included two subcomponents: • Subcomponent 2.1: Strengthening MSPP's Stewardship and Management Capacity (US$5 million, equivalent to 5.6 percent of total grant proceeds). This subcomponent financed goods, TA and training with the aim of strengthening the MSPP's stewardship and management capacity. 7 The PMCHNSs included: (i) preventative services, such as immunization, micronutrient supplementation, cholera prevention and promotion of insecticide - treated bed - nets; (ii) promotion of health services, such as increasing prevalence of exclusive breast - feeding and use of family pla nning; (iii) basic curative services, such as treatment of acute respiratory infections, cholera and other diarrheal diseases, other child hood illnesses, and tuberculosis; and (iv) reproductive health services, such as family planning, prenatal care, emerg ency obstetrical care, and post - partum care. The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 10 of 87 • Subcomponent 2.2: Strengthening S P Coordination and Management Cap acity (US$4 million, equivalent to 4.4 percent of total grant proceeds). This subcomponent financed small works, goods, technical assistance ( TA ) and training with the aim of strengthening FAES's institutional capacity at the central, municipal, and commun ity levels to enhance coordination, organization, management and social service delivery to vulnerable families . B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 13. There were significant changes introduced during the operation's lifetime in response to the changing implementation environment due to Hurricane Matthew, the continuous threat posed by cholera , changes in certain Go H’s policies and significant fluctuations in donors' contributions. The main changes can be summarized as follows (see also Annex 6 a nd Tables 1 and 2): 14. Level 2 restructuring - March 2017 - Although the PDO remained unchanged, this restructuring introduced considerable modifications to the operation's design and Project Results Framework (PRF) to reflect the country's evolving needs an d priorities. As shown in Table 2, this restructuring included significant changes to the Project components: i) the elimination of SP activities provided through the KF network under the original Subcomponent 1.2 (including cash transfers) as a result of several factors, including FAES' inadequate implementation capacity vis - à - vis the by - then obvious complexity of the integrated, cross - sectoral approach to the delivery of services envisioned under the Project; the lack of sustained commitment to cross - sect oral a ctivities on the part of the MSPP 8 ; the drastic reduction in the number of SP programs with external funding that reduced the need to coordinate social services at the household level ; and reduced Government support for the KF program ; ii) SP activities under Subcomponent 2.2 were modified to focus solely on the development of a Single Social Registry (SSR) 9 (w hich had increasing Government support) ; iii) activities to combat cholera previously included under Subcomponent 1.1 were given higher priority under a new Subcomponent 1.2 and strengthened in response to the heightened risk of outbreaks as a result of th e flooding and destruction caused by Hurricane Matthew; iv) a new Subcomponent 1.3 was added (Contingent Emergency Response) to ensure the immediate availability of funds in the event of an emergency; and iv) a new Component 3 (Piloting Vulnerability Indic ators for More Targeted Social Service Delivery) was added aimed at piloting the calculation of vulnerability indicators, a key step in targeting vulnerable households for the delivery of social services. In addition, among other changes, grant proceeds we re reallocated and the PRF was significantly revised to reflect the changing priorities, including changes in PDIs (see Table 1 and Annex 6 for a more detailed description). 15. Additional Financing (AF) and Level 1 restructuring - June 2017 - An AF for US$2 5 million was approved on June 14, 2017 as part of a package of World Bank (WB) support to help the GoH recover and rebuild after Hurricane Matthew. Concomitantly, the original operation was restructured and the PDO was revised to better respond to Haiti’s evolving needs . 10 Specifically, t he AF provided financial support for response activities in the affected areas, mainly: i) to restore the quality and supply of health services via rehabilitation and re - equipping of health facilities damaged by the hurricane; and ii) to scale - up cholera prevention and response activities to help address the new front in the fight against cholera that had opened up in hurricane - affected areas (see Table 2 for 8 In particular, the MSPP favored its own Community Health Agent model (with agents focusing only on health sector activities) rather than the multi - sectoral K F agents. 9 Also called the Integrated Beneficiary Registry or Information System of the Ministry of Labor and Social Affairs (MAST) or Système d’Information du MAST ( SIMAST). 10 The WB's response to Hurricane Matthew was financed under IDA’s Crisis Response Window. The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 11 of 87 the allocation of AF funds). The AF also filled the financing gap created to support emergency response activities under the original grant immediately after Hurricane Matthew. At the same time , a Level 1 restruct uring was carried out to reflect the changes made under the AF, including: i) a revision of the PDO and the PRF to reflect the expanded geographical coverage and emergency response objectives (see Section IV.A); and ii) the activation of new safeguards pol icies (see Section IV.B and Annex 6 for a more detailed description). Revised PDOs and Outcome Targets 16. As noted earlier, the operation's PDO was modified in June 2017 under a Level 1 restructuring that was carried out as part of an AF. 17. The original PDO: “ to increase the access and use of MCH , nutrition and other social services in the Recipient's territory (defined as at least three Departments with a total catchment population of around 1.8 million people, targeting pregnant women, children under five and vulnerable families) ” was revised in June 2017 to : “ increase access and use of MCH services, strengthen cholera control, and improve targeting of social services in the Recipient’s territory, with a particular focus on areas affected by Hur ricane Matthew. ” Revised PDO Indicators 18. PDO indicators (PDIs) were modified to reflect the changes introduced under the two restructurings that took place in March and June 2017 (see Table 1). The main modifications under the March 2017 restructuring included: i) elimination of the original PDI 4 (i.e., families categorized as extremely vulnerable); and ii) the addition of a new PDI 4 (i.e., cholera fatality rate). The main modi fications under the June 2017 AF and restructuring included: i) the addition of PDI 5 (i.e., pilot census carried out); ii) the upward revision of end targets for PDIs 2 and 3; and iii) extensions to end dates and minor modifications to the baselines for P IDs 1 through 3 (see also Annex 6). Table 1. Changes in PDOs and PDO Indicators (PDIs) Original PDIs Revised PDIs Restructuring March 2017 Revised PDIs AF and Restructuring June 2017 PDI 1. Children under five immunized PDI 1. Children under five immunized No change PDI 1. Children under five immunized Slight adjustment to the baseline; end date extended. PDI 2. Institutional deliveries PDI 2. Institutional deliveries No change PDI 2. Institut i onal deliveries Slight adjustment to the baseline; end date extended; end target revised upward PDI 3. Contraceptive prevalence rate PDI 3. Contraceptive prevalence rate No change PDI 3. Contraceptive prevalence rate Slight adjustment to the baseline; end date extended; end target revised upward PDI 4. Families categorized as extremely vulnerable -- Dropped -- -- PDI 4. Cholera fatality rate Added PDI 4. Cholera fatality rate End date extended PDI 5. Pilot census carried out Added The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 12 of 87 Revised Components 19. As described earlier, Project components were also modified under the two 2017 restructurings. The specific changes that were introduced are summarized on Table 2 (see also Annex 6). Table 2. Changes in Project Components Restructuring March 2017 AF and Restructuring June 2017 ▪ Subcomponent 1.2 (Results - oriented support for poor and vulnerable families) was eliminated due to termination of SP activities under the KF network . ▪ Subcomponent 2.2 (Strengthening SP coordination and management) was modified to focus solely on the development of a S SR . ▪ New Subcomponent 1.2 (Prevention and treatment of cholera) was added to include cholera - related activities previously under Subcomponent 1.1 . ▪ New Subcomponent 1.3 (Co ntingent emergency response) was added to ensure the availability of contingency financing in case of emergency . ▪ New Component 3 (Piloting vulnerability Indicators) was added to support the piloting of the Fifth Housing and Population Census and vulnerabil ity indicators . ▪ Subcomponent 1.1 (Performance - based MCH and Nutrition Service Delivery) received an additional allocation of US$9.5 million under the AF to expand coverage to areas affected by Matthew . ▪ Subcomponent 1.2 (Prevention and treatment of cholera and other diarrheal diseases) received an additional allocation of US$13.5 million under the AF to expand activities in areas affected by Matthew and fill financing gap from emergency response in its immediate aftermath . ▪ Subcomponent 2.1 (Strengthening MSP P's Stewardship and Management Capacity) received an additional allocation of US$2 million under the AF for M &E of Project activities in areas affected by Matthew . Other Changes 20. Level 2 Restructuring - December 2019 - This restructuring addressed implementation delays caused by growing social and political unrest and ensure the full utilization of the grant proceeds as well as a smooth transition to a follow - on health Project ( P167512 - Strengthening Primary Health C are and Surveillance in Haiti - ( Projet de Renforcement des Soins de Santé Primaire et de la Surveillance en Haïti , PROSYS ) . The specific modifications included: i) a three - month extension of the closing date from December 31, 2019 to March 31, 2020; ii) the transfer of the key health activities with continuous support under PROSYS; and iii) the reallocation of funds between dis bursement categories under the Project's original International Development Association ( IDA ) grant (IDA - H8640) to ensure the completion of pending Project activities managed by FAES (i.e., the finalization of the National S P Strategy and the S SR ) and the financing of civil works and activities supported by this operation and implemented by United Nations (UN) agencies. 21. Level 2 Restructuring - March 2020 - This restructuring aimed to support the MSPP's efforts to address the COVID - 19 pandemic. Specific modifications included: i) a six - month extension of the closing date from March 31, 2020 to September 30, 2020; and ii) the reallocation of funds between disbursement categories to support the MSPP’s capacity to address the COVID - 19 pandemic. Rationale for Changes and t heir Implication on the Original Theory of Change 22. The changes introduced in 2017 implicitly altered the original theory of change. The operation was streamlined to focus solely on the supply side by expanding access to health se rvices at the facility level whereas the activities intended to stimulate demand at the household and community level through the KF network were eliminated. The theory of change underlying the health stream remained largely unchanged. In response to signi ficant shortcomings identified in the implementation of the KF initiative, the SP stream was considerably reduced, focusing solely on the development of a SSR and the development of vulnerability indicators to improve beneficiary targeting for delivery of social services (the “vulnerability indicators stream”). Finally, cholera control The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 13 of 87 gained additional importance in response to the renewed cholera outbreaks in the aftermath of Hurricane Matthew (the "cholera control stream"). Figure 2. Revised Project Results Chain Note: Project O utputs are indicated in bullet points under the corresponding Intermediate Outcomes. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: High (Pre - AF); High (Post - AF periods) The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 14 of 87 23. The relevance of the operation's PDOs, both original and revised, is deemed High. They were fully aligned with the WB’s Country Partnership Framework (CPF) for the Republic of Haiti FY16 - 19 (Report No. 98132 - HT) discussed by the Board of Directors on September 29, 2015 and the Haiti Performance and Learning Review (PLR) of the CPF (Report No. 124812 - HT). 11 Specifically, the Project provided support the CPF Area of Focus 2 (Human Capital), in particular Objectives 6 (Increase Access to Health Services for Mothers and Children) and 7 (Control Cholera i n Priority Communes). In addition, the revised PDO was in alignment with priorities for Haiti identified by the Systematic Country Diagnostic conducted as part of the CPF FY16 - 19 (i.e., better targeting in SP as a priority to protect households and individ ual livelihoods vulnerable to external climate and other emergencies; and better targeting of policies and programs in a context of limited resources). The Project was also consistent with the findings of the WB 's 2017 Health Financing Assessment, as it fo cused on improving the organization of the health sector and the efficiency of the service delivery system, while increasing access and use of health care services with particular attention to women and children. 24. Both the original and revised PDOs were also highly consistent with domestic priorities. Specifically, they were fully aligned with the MSPP’s Health Sector Development Plan 2012 - 2020 ( Plan Directeur de Sant é 2012 - 2022) and contributed toward the achievement of Sustainable Development Goal s 3.1, 3.2, 3.3 and 5 (maternal mortality, child mortality, communicable diseases, and gender equality, respectively). The Project provided direct support to two components of the MSPP’s Health Sector Development Plan 2012 - 2022 (i.e., “Organizational and O perational Strengthening of the Health System” and “Provision of Health Services and Care"). Similarly, the revised PDO was fully aligned with the Go H ’s 2013 - 2022 National Plan for the Elimination of Cholera . Finally, the revised PDO is central to the impl ementation of the National Policy on Social Protection and Promotion (Politique Nationale de Protection et de Promotion Sociales, PNPPS ) that was adopted by the Council of Ministers in June 2020 and has the full support of the donor community. Specifically , better targeting of social services is a precondition for achieving the PNPPS goals for 2040: i) a reduction of poverty and inequality, ii) a reduction of economic, social, and institutional injustices, and iii) giving citizens the right to access SP and promotion as mechanisms to enhance their capacity to live better lives. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of E ach Objective/Outcome 25. The following considerations shou ld be noted regarding the methodological approach adopted for this I mplementation Completion and Results Report (I CR ) : • The changes introduced under the restructurings that took place in March and June 2017, respectively, called for the utilization of the s plit methodology. 12 However, given that there were no disbursements between March and June, for the purpose of this ICR, the evaluation is split into only two (as opposed to three) periods: the pre - AF period (effectiveness through June 2017); and post - AF (June 2017 through c losing). Their relative weights reflect the disbursements during each period as a proportion of total disbursements (28.5 and 71.5 percent for the pre - and post - AF periods respectively). • Efficacy for the pre - AF implementation period is assessed based on t he original PDO, which has been "unpacked" into three different PDOs, focusing separately on MCH services (PDO 1); nutrition services 11 The CPF period was origin ally set from 2015 to 2019. However, after completion of the PLR in 2018, CPF milestones were extended until 2021 while a new CPF is prepared. 12 The May 2017 restructuring included changes in PDO indicators, and the June 2017 restructuring that was carri ed out in conjunction with the AF included changes in the PDO, PDO indicators, and end - project targets. The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 15 of 87 (PDO 2); and other social services (PDO 3). Efficacy performance for each of these PDOs is assessed against the original P RF targets. • Efficacy for the post - AF implementation period is assessed based on the revised PDO, which has also been "unpacked" into three different PDOs, focusing separately on: MCH health services under the RBF model (PDO 4); cholera control (PDO 5); an d targeting of social services (PDO 6). With regard to PDO 6, it is important to note that it encompasses actions undertaken under both the S P and Targeting streams (Subcomponent 2.2 and Component 3, respectively). Efficacy performance for each of these PDOs is assessed against the revised PRF targets. • Thus, the operation's overall efficacy is assessed against three PDOs for the pre - AF period (i.e., PDOs 1, 2, 3) and three PDOs for the post - AF period (i.e., PDOs 4, 5, 6), each of them being assigned the same relative weight. • Finally, although the PDO definitions refers to both "access" and "utilization" of services, further "unpacking" these two dimensions presents methodological difficulties, since in practice they are simultaneously incorporated in ea ch of the PDIs supporting the original and revised PDOs. Hence, the two dimensions are considered to be the same for the purpose of this evaluation. PDO 1 - To increase access and use of maternal and child health services. Rating: Substantial (Pre - AF implementation period) 26. Efficacy for PDO 1 is deemed Substantial . This PDO applies to pre - AF period, when MCH services were to be provided under a two - pronged strategy: i) at the facility level under the MSPP's RBF Model; and ii) at the community a nd household levels by the KF network under FAES. Despite the limited contribution of the KF agents to the delivery of maternal and child services 13 , there was a significant expansion in access to cost - effective preventative MCH services under the MSPP's RBF Model. One of the main innovations of the RBF Model was its reliance on results - based payments to service providers for the delivery of a package of selected health care services following pre - defined clinical and reporting protocols. Both the quantity and the quality of the services were externally verified, and payments were adjusted according. After the successful implementation of a pilot in s even health facilities in 2014, the RBF model, together with its rules and regulations, clinical and reporting protocols and M&E system, was expanded to 135 health facilities in the four target Departments 14 . The successful implementation of the RBF Model reflects the close coordination with other donors, in particular Canada and the United States Agency for International Development (USAID) and strong stewardship on the part of the MSPP. 27. Overall, the operation made a significant contribution in terms of e xpanding and enhancing the capacity for the delivery of health care services under the RBF model, resulting not only in access to "more" but also "better" and "more homogeneous" MCH care services. 15 Even though direct attribution cannot be readily establis hed given the available information, these outcomes can be expected to have contributed to the decrease in neonatal, infant and child mortality during the past decade. The specific achievements under this PDO are reflected in two of three PDIs (i.e., PDI 2 - Increase in institutional deliveries; and PDI 3 - Increase in contraceptive prevalence) that amply 13 Largely because ultimately, the MSPP did not support the K F program, preferring instead its own Community Health Agent program with agents dedicated to the health sector only rather than the multi - sectoral K F agents. 14 At the design stage, the Project was to cover three Departments (Northeast, Northwest and Center ), but this was expanded later to also include the South Department. The number of health facilities included when the RBF program was scaled up in 2014 (135) was l ater expanded to 188, out of a total of 276 facilities in the four Departments. The excluded facilities consist mostly of large hospitals, private for - profit facilities and facilities with low functionality. 15 As seen from the significant increases over time in utilization of these services, as well as significant increases in the qu ality score , for the health facilities under the RBF program (based on data verified by the independent external verification agents). The World Bank Improving Maternal and Child Health through Integrated Social Services (P123706) Page 16 of 87 exceeded the original targets, while the third one (PDI 1 - Increase in vaccination coverage) lagged behind original expectations (see Annex 1.A and B). T hese and other achievements under PDO 1 can be summarized as follows: PDO 1 - To increase access and use to maternal and child health services Level of Achievement: Substantial (pre - AF implementation period; Original targets) Outputs ▪ RBF for the delivery of selected MCH health care services according to predefined clinical protocols, accompanied by purchase of key vaccines 16 under the Project, resulting in, among others, over 3.5 million children 17 immunized and over 47,000 deliveries attended by skilled health personnel between 2013 and 2019, amply exceeding the 660,000 and 21,000 targets, respectively. ▪ Upgraded health infrastructure at the primary level (e.g., small - scale rehabilitation, equipm ent, medical supplies, essential health commodities and training of, and TA to health personnel). Intermediate Results Enhanced MSPP's stewardship, institutional and technical capacity for the implementation of the RBF Model ▪ The percentage of contracted health providers supervised at least quarterly increased from 0 to 100 percent between 2012 and 2019, exceeding the 95 percent target. ▪ The percentage of contracted service providers achieving the minimum quality score (60 perc ent) increased from 0 percent to 93 percent between 2012 and 2019, more than twice the original target of 40 percent. ▪ The percentage of providers utilizing the contracting model increased from 0 to 62 percent between 2012 and 2019, exceeding the original target of 50 percent. Enhanced supply (quality and supply) of MCH services under the RBF Model ▪ The number of children aged under 12 months that were completely vaccinated at health facilities under the RBF program increased by 51 percent between 2012 and 2019, amply exceeding the 13 percent target. ▪ The number of births at health facilities under the RBF program increased by 64 percent between 2012 and 2019, amply exceeding the 10 percent target. Outcomes Increased access and use of MCH services ▪ PDI 1. The percentage of children under five immunized in the four target Departments increased -- albeit slightly