Pwojè Ranfòse Sèvis Idwo-meteyo ann Ayiti

Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005439 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF - A0226 ON A GRANT IN THE AMOUNT OF SDR 3.5 MILLION (US$ 5 MILLION EQUIVALENT) TO THE REPUBLIC OF HAITI FOR THE HT Strengthening Hydro - Met Services Project April 13 , 2021 Urban, Resilience And Land Global Practice Latin America And Caribbean Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized CURRENCY EQUIVALENTS (Exchange Rate Effective March 15, 2021) Currency Unit = Haitian Gourdes (HTG) HTG78.993 = US$1 US$1.431 = SDR 1 FISCAL YEAR October 1 - September 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Tahseen Sayed Khan Regional Director: Anna Wellenstein Practice Manager: David N. Sislen Task Team Leader(s): Claudia Ruth Soto Orozco ICR Main Contributor : Naraya Carrasco ABBREVIATIONS AND ACRONYMS ACCPAC Account Package CARICOM Caribbean Community CCPC Municipal Civil Protection Committee (Comité Communal de Protection Civile) CIAT Comité Interministériel d’Aménagement du Territoire (Inter - ministerial Committee for Territorial Planning) CIF Climate Investment Funds CNIGS Centre National de l'Information Géo - Spatiale (National Center for Geo - Spatial Information) CNM Centre National de Météorologie (National Center for Meteorology) CNSA Coordination Nationale de la Sécurité Alimentaire (National Coordination for Food Security) COOPI Cooperazione Internazionale (Italy’s International Cooperation) CPF C ountry P artnership F ramework DCC Division Changement Climatique (Climate Change Directorate) DDA Direction Départementale de l’Agriculture (Departmental Directorate for Agriculture) DG Director General DIA Direction des Infrastructures Agricoles (Agriculture Infrastructure Directorate) DPC Direction de la Protection Civile (Civil Protection Directorate) DRE Direction des Ressources en Eau (Water Resource Directorate) EIRR Economic Internal Rate of Return EMP Environment al Management Plan ESMF Environmental and Social Management Framework EU European Union EWS Early Warning System FCV Fragile, Conflict and Violence GDP Gross Domestic Product GFDRR Global Facility for Disaster Reduction and Recovery GIS Geographical Information System GoH Government of Haiti HT Haiti IDB Inter - American Development Bank INSMET Cuban National Institute for Meteorology (Instituto de Meteorología de la República de Cuba) ISN Interim Strategy Note ISR Implementation Status and Results MARNDR Ministère de l’Agriculture, Ressources Naturelles et Développement Rural (Ministry of Agriculture, Natural Resources and Rural Development) MCH Meteorological, Climatological and Hydrological Database Management System MDE Ministère de l'Environnement (M inistry of the Environment) MICT Ministère de l’Intérieur et des Collectivités Territoriales (Ministry of Interior and Territorial Communities) MIS Management Information System MPCE Ministère du Plan et de la Coopération Externe (Ministry of Planning and External Cooperation) MTPTC Ministère des Travaux Publics, Transports et Communications (Ministry of Public Works, Transport and Communications) NAPA National Action Plan for Adapta tion NHP National Hydro - met Platform NPV Net Present Value OFNAC Office National de l’Aviation Civile (National Civil Aviation Agency) OM Operations Manual ONEV Observatoire National de l’Environnement et de la Vulnérabilité (National Observatory on Environment and Vulnerability) OP Operational Policy PAD Project Appraisal Document PDO Project Development Objective PGRAC Strengthening Disaster Risk Management and Climate Resilience Project ( Projet d e Gestion d es Risques e t d e Résilience a ux Aléas Climatique ) PIU Project Implementation Unit PNAP Programme National d’Alerte Précoce (National Early Warning Project) PPCR Pilot Program for Climate Resilience RESEPAG II Relaunching Agriculture: Strengthening Agriculture Public Services Project (Projet de Renforcement des Services Publics Agricoles II) RF Results Framework SCF Strategic Climate Fund SDGs S ustainable D evelopment G oals SDR Special Drawing Rights SEMANAH Service Maritime et de Navigation d'Haïti (Haiti National Maritime and Navigation Service) SNRE Service National des Ressources en Eau (National Service for Water Resources) SOP Standard operating procedures SORT Systematic Operations Risk - rating Tool SPCR Strategic Program for Climate Resilience STEP Systematic Tracking o f Exchanges i n Procurement TA T echnical A ssistance T o C Theory of Change UEP Unité d’Études et de Programmation (Study and Planning Unit, MARNDR) UHM Unité Hydrométéorologique d’Haïti (Hydrometeorological Unit of Haiti) UIS Unité Informatique et Statistique (Statistics and IT Unit, MARNDR) UPMP Unité de Passation de Marchés Publics (Centralized P rocurement U nit, MARNDR) USAID United States Agency for International Development WMO World Meteorological Organization TABLE OF CONTENTS DATA SHEET ................................ ................................ ................................ ................................ 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ................................ ......................... 5 A. CONTEXT AT APPRAISAL ................................ ................................ ................................ ............. 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ................................ ......... 8 II. OUTCOME ................................ ................................ ................................ ........................... 9 A. RELEVANCE OF PDOs ................................ ................................ ................................ .................. 9 B. ACHIEVEMENT OF PDOs (EFFICACY) ................................ ................................ .......................... 10 C. EFFICIENCY ................................ ................................ ................................ ................................ 13 D. JUSTIFICATION OF OVERALL OUTCOME RATING ................................ ................................ ....... 14 E. OTHER OUTCOMES AND IMPACTS ................................ ................................ ............................ 14 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ .. 15 A. KEY FACTORS DURING PREPARATION ................................ ................................ ....................... 15 B. KEY FACTORS DURING IMPLEMENTATION ................................ ................................ ................ 16 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ... 18 A. QUALITY OF MONITORING AND EVALUATION (M&E) ................................ .............................. 18 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMP LIANCE ................................ ....................... 19 C. BANK PERFORMANCE ................................ ................................ ................................ ............... 20 D. RISK TO DEVELOPMENT OUTCOME ................................ ................................ .......................... 21 V. LESSONS AND RECOMMENDATIONS ................................ ................................ ................. 22 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ................................ .............................. 24 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ........................... 32 ANNEX 3. PROJECT COST BY COMPONENT ................................ ................................ ............... 35 ANNEX 4. EFFICIENCY ANALYSIS ................................ ................................ ............................... 36 ANNEX 5. BORROWER, CO - FINA NCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 37 ANNEX 6. SUPPORTING DOCUMENTS ................................ ................................ ...................... 42 ANNEX 7. PROJECT I MPLEMENTATION TIMELINE: EXOGENOUS FACTORS ............................... 43 ANNEX 8. PROJECT RESULTS FRAMEWORK AND PPCR CORE INDICATORS ............................... 44 ANNEX 9. PROJECT COMPONENT ACHIEVEMENTS AND LINK TO CREWS ................................ . 45 The World Bank HT Strengthening Hydro - Met Services (P148259) Page 1 of 46 DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P148259 HT Strengthening Hydro - Met Services Country Financing Instrument Haiti Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of Haiti Ministry of Agriculture, Natural Resources and Rural Development (MARNDR) Project Development Objective (PDO) Original PDO The Project Development Objective (PDO) is to strengthen the Republic of Haiti's institutional capacity to providehydro - meteorological and climate information services customized to the needs of the civil protection and agriculture sectors,which contribute s to increasing disaster and climate resilience. The World Bank HT Strengthening Hydro - Met Services (P148259) Page 2 of 46 FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing TF - A0226 5,000,000 5,000,000 3,742,007 Total 5,000,000 5,000,000 3,742,007 Non - World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 5,000,000 5,000,000 3,742,007 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 26 - Jun - 2015 19 - Oct - 2015 12 - Nov - 2018 30 - Jun - 2020 30 - Oct - 2020 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 26 - Jun - 2020 3.70 Change in Loan Closing Date(s) Change in Implementation Schedule 30 - Jun - 2020 3.70 KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Unsatisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs No. Date ISR Archived DO Rating IP Rating Actual Disbursements (US$M) 01 24 - Nov - 2015 Satisfactory Satisfactory 0 The World Bank HT Strengthening Hydro - Met Services (P148259) Page 3 of 46 02 08 - Jun - 2016 Satisfactory Moderately Satisfactory .50 03 27 - Dec - 2016 Moderately Satisfactory Moderately Satisfactory .50 04 18 - May - 2017 Moderately Satisfactory Moderately Unsatisfactory .51 05 05 - Dec - 2017 Moderately Satisfactory Moderately Unsatisfactory .52 06 18 - May - 2018 Moderately Satisfactory Moderately Satisfactory .52 07 31 - Dec - 2018 Moderately Satisfactory Moderately Satisfactory .85 08 26 - Jun - 2019 Moderately Unsatisfactory Moderately Unsatisfactory .85 09 31 - Dec - 2019 Moderately Unsatisfactory Moderately Unsatisfactory 1.29 10 30 - Oct - 2020 Moderately Unsatisfactory Moderately Unsatisfactory 3.70 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 14 Other Agriculture, Fishing and Forestry 14 Water, Sanitation and Waste Management 86 Public Administration - Water, Sanitation and Waste Management 72 Other Water Supply, Sanitation and Waste Management 14 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Finance 16 Finance for Development 16 Disaster Risk Finance 16 The World Bank HT Strengthening Hydro - Met Services (P148259) Page 4 of 46 Public Sector Management 36 Public Administration 36 E - Government, incl. e - services 36 Urban and Rural Development 48 Disaster Risk Management 48 Disaster Response and Recovery 16 Disaster Risk Reduction 16 Disaster Preparedness 16 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Carlos Felipe Jaramillo Country Director: Mary A. Barton - Dock Tahseen Sayed Khan Director: Ede Jorge Ijjasz - Vasquez Anna Wellenstein Practice Manager: Anna Wellenstein David N. Sislen Task Team Leader(s): Gaetano Vivo Claudia Ruth Soto Orozco ICR Contributing Author: Naraya Carrasco The World Bank HT Strengthening Hydro - Met Services (P148259) Page 5 of 46 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At appraisal in May 2015, Haiti was still confronting considerable challenges due to exogenous and endogenous factors. Haiti continued to be severely affected by the aftermath of the catastrophic 2010 earthquake that led to a toll of 220,000 casualties, including one in three civil servants, 1.5 million people displaced, and the equivalent of 120 percent of gross domestic product (GDP) destroyed, while facing a growing political and socioeconomic crisis. Climate change was projected to heighten the risk of hydrometeorological (hydromet) hazards, with an increased frequency and intensity of extreme events such as Hurricane Sandy in 2012 — about 75 deaths, 200,000 affected and US$0.5 billion in damages. The poverty rate was 58.5 percent, with almost 6.3 million Haitians unable to meet their basic needs and one million Haitians living slightly above the poverty level, risking falling back into poverty because of external shocks, including climatic - induced shocks. 2. Haiti’s economic recovery and growth potential could not be sustained without improving the country’s meteorological data to support early warning systems to enhance its resilience as a whole and informing the agricultural sector. A World Bank global study 1 ranked Haiti fifth in exposure to the risk of two or more hazards, with more than 93 percent of its territory and 96 percent of its population at risk of two or more hazards and 56 percent of its GDP linked to an area exposed to risk stemming from two or more hazards. High coastal population density, informal settlements, weak infrastructure and flood - prone agricultu ral land render Haiti’s population and the agricultural sector particularly vulnerable. Over the past decade, hydromet hazards have claimed lives, affected livelihoods, and have taken a heavy toll on all sectors of the Haitian economy, especially in agricu lture and food security. Rationale for World Bank Engagement 3. Four years after the 2010 Earthquake, the Government of Haiti (GoH ) had embarked on a structural reform of the country where catastrophic events and climate change were high on national priority. In this context, a World Bank recipient - executed trust fund grant was provided to Haiti under the Pilot Program for Climate Re silience (PPCR) under the Strategic Climate Fund. At appraisal, the objectives of the project aligned with the World Bank Group’s Haiti Interim Strategy Note (ISN) FY13 – FY14 (Report #71885 - HT) and supported two of the four objectives of the ISN: (i) reduci ng vulnerability and increasing resilience, and (ii) promoting inclusive growth. It also contributed to the ISN crosscutting theme of strengthening governance. Moreover, the World Bank had a comparative advantage in hydromet operations and experience in se veral countries, in conjunction with the Global Facility for Disaster Reduction and Recovery (GFDRR) Hydromet Program, to better predict, prepare, and respond to disastrous events, manage natural resources — water in particular — and better understand climate change effects, as well as support transport activities in aviation and navigation. The project aligned with the GoH 2013 National Strategic Program for Climate Resilience (SPCR), which included “Strengthening Knowledge Management of Hydro - met, Water Resou rces, and Climate Data to Inform Decision Making and Policy Dialogue”. It also contributed to the implementation of Haiti’s 2006 National Action Plan for Adaptation (NAPA). The project design responded to challenges and opportunities highlighted in various studies, such as 1 World Bank, Natural disaster hotspots: A global risk analysis, 2005. The World Bank HT Strengthening Hydro - Met Services (P148259) Page 6 of 46 those of the World Meteorological Organization (WMO 2010 and 2012), 2 the Interamerican Development Bank (2014) 3 and the World Bank (2014). 4 The 2012 Haiti Strategic Program for Climate Resilience (SPCR) especially pointed to a new inst itutional framework for the hydromet services with the following priorities: (i) rationalizing and streamlining the national data collection and management system; (ii) establishing a mechanism to bring together data producers and main end users — aviation, civil protection, water utility — and providing feedback, and increasing the socioeconomic value of data; and (iii) ensuring a sustainable financing model based on a cost - recovery principle. Theory of Change (Results Chain) 4. The p roject a ppraisal d ocument (PAD) did not include a t heory of c hange (ToC). The inferred ToC (Figure 1) , including the critical assumptions, are presented below: Figure 1 : Theory of Change - Hydromet data accessible through a centralized platform will support decision - making processes and therefore increase preparedness and overall resilience of the Haitian population , as well as agricultural production . 2 The re - establishment of meteorological services in Haiti in 2010 with a website ( www.meteo - haiti.gouv.ht ) to disseminate information and warnings to United Nations ’ agencies, nongovernmental organizations (NGOs), disaster - affected c ommunities , and the media occurred thanks to collaboration between WMO and other development partners. WMO also partnered with Environment and Climate Change Canada in 2012 to develop the project “Climate Services to Reduce Vulnerability in Haiti” whose ai m was to install technical equipment such as forecaster workstations, Internet, communications, and related technology in a newly built UHM headquarters. Website: < Climate Services to Reduce Vulnerability in Haiti | World Meteorological Organization (wmo.int) > 3 IDB website: < Background Paper: LAC Small Island Development States | Publications (iadb.org) > 4 World Bank website: < WorldBank - rapid - global - assessment.pdf (whycos.org) > The World Bank HT Strengthening Hydro - Met Services (P148259) Page 7 of 46 - A strong hydromet unit in Haiti ( UHM ) and government officials well - trained in the collection and management of hydromet data will result in on - time preparedness activities , thus enhancing the protection of the population and improv ing agricultural production and livelihoods. Project Develo pment Objectives (PDOs) 5. The Project Development Objective (PDO), as stated in the financial agreement approved on June 26, 2015, which is consistent with the PDO statement in the PAD, is to strengthen Haiti’s institutional capacity to provide hydrometeoro logical and climate information services customized to the needs of the civil protection and agriculture sectors, which contribute to increasing disaster and climate resilience Key Expected Outcomes and Outcome Indicators 6. The key outcome indicators were: (i) Data collected from hydromet networks are accessible on a centralized online data management platform, with standard operating procedures for validation and storage; (ii) Number of subnetworks feeding into the centralized plat form; and (iii) End users’ satisfaction rate toward improved hydromet information services , percentage and gender - disaggregated. Components 7. The project included three components : - Component 1, Institutional strengthening of the hydrometeorological services and development of data management tools (S pecial D rawing R ights {SDR} 2.1 million / US$ 3 million) : 1.1. Institutional strengthening, capacity building , and partnership development . Provision of support, among others , for: 1 . Carrying out technical assistance (TA) to develop a long - term strategy based on sustainable financing and staffing model for the national entity that will be responsible for the development, operation , and maintenance of all hydrometeorological services. 2 . Carrying out training and knowledge exchange programs to develop data management capacity of the producers and users of hydrometeorological data. 3. Developing applications and applied research pro grams aimed at supporting students or young professionals or both groups selected in accordance with the criteria set forth in the p roject o perations m anual in carrying out field work to improve hydromet and climate data collection and management. 1.2. Strengthening i nter i nstitutional data sharing and coordination . Provision of support, among others, for: 1. Designing a central data platform for hydrometeorological data integration based on an open data approach through: (i) the implementation of a countrywide, georeferenced baseline assessment of hydrometeorological equipment; (ii) the proposal for an optimal network, which would meet priority national needs while taking into account the maintenance and operational budget; (iii) the development of the architecture and technical specifications of a data platform capable to gather data from all existing hydrometeorological sensors and feed in real time several user - specific interfaces; and (iv) the develop ment of standard procedures for equipment and data management. 2. Developing specialized interfaces based on a geographical information system (GIS) for select end users to allow them access to hydrometeorological information. 3. Carrying out training for hydrometeorological data producers and users on the use and maintenance of equipment and management of the online data management platform, including the specialized interfaces. The World Bank HT Strengthening Hydro - Met Services (P148259) Page 8 of 46 4. Repairing, replacing , or upgrading existing hydrometeorological equipment throughout the r ecipient's territory based on the assessment carried out in Part 1.2.1. - Component 2, Identifying hydrometeorological and climate services' requirements for select end users and developing information services to support decision making (SDR 1.0 million / US$ 1.4 million) . Provision of support for : 1. Scoping hydrometeorological information requirements of end users, pursuant to methodology and selection criteria set fo rth in the p roject o perational m anual. 2. Updating operating procedures for the optimal use of hydrometeorological services for agriculture and food security through initiating a pilot information service for farmers and other f ood s ecurity s takeholders, designed to factor requirements from end users of the food security and agricultural communities. 3. Updating operating procedures for the optimal use of hydrometeorological services for civil protection through the launch of an information service for civil protection and through the provision of expert assistance for early warning and emergency respo nse in pilot zones. 4. Updating the estimated return period of selected weather hazards , including floods, windstorms, and storm surges , to facilitate the design of infrastructure and planning of development processes. - Component 3, Project implementatio n, evaluation and monitoring and p ilot p rogram for c limate r esilience (PPCR ) knowledge management (SDR 0.4 million / US$ 0.6 million): Provision of support to the Ministry of Agriculture, Natural Resources and Rural Development (MARNDR) for project implemen tation, management, procurement, and monitoring and evaluation. The project relied on a multi - project centralized procurement unit (UPMP), also in charge of contract management that was established by the World Bank in 2014 under MARNDR. The creation of the UPMP aimed to strengthen MARNDR’s capacity to manage projects. The project coordinator was selected by MARNDR among the ministry’s staff B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 8. The PDO indicators and outcome targets were not changed during the lifetime of the project. Revised PDO Indicators 9. The PDO indicators were not changed during the lifetime of the project. Revised Components 10. Under Component 3, the project had initially planned to rely on MARNDR's own staff and the UPMP for procurement and project management to avoid the need for a full project implementation unit (PIU). However, given MARNDR’s capacity constraints and the resignation of the Project Coordinator, the project ended up mobilizing a full PIU after the project midterm review (MTR). Other Changes 11. Further to the creation of the PIU after the MTR, the project was restructured in June 2020 to extend the grant closing date by four months from June 30, 2020 to October 30, 2020 (because of delays resulting from the COVID - 19 pandemic) so that the project could take delivery of the 62 hydromet stations. The residual grant funds of US$1.3 million were canceled and the stations are installed under the World Bank’s “Relaunching Agriculture: Strengthening Agriculture Public Services II Project” (RESEPAG II - P1 26744). Further TA and safeguards compliance monitoring to accompany the The World Bank HT Strengthening Hydro - Met Services (P148259) Page 9 of 46 installation process and support the hydromet sector will continue under the GFDRR TA, which is available under the World Bank’s “Strengthening Disaster Risk Management and Climate Re silience Project” (PGRAC - P165870). Rationale for Changes and T heir Implication on the Original Theory of Change 12. The extension of the closing date intended to allow the project to complete shipment, delivery and installation of hydromet equipment. The limited project extension (four months) affected the accomplishment of the shor t - term outcomes shown in the Theory of Change. II. OUTCOME A. RELEVANCE OF PDOs Rating: High Assessment of Relevance of PDOs and Rating 13. The project objective remains highly relevant at closing . The PDO remains consistent with the c ountry p artnership f ramework (CPF) FY2016 – 19 (98132 - HT) — which is the ongoing CPF 5 — as it contributes to two of the three areas of CPF ’s focus: (i) the resilience area, supported by enhanced understanding of hydrometeorological and climate risks to assess social and economic impacts , and to develop adequate policy responses t o support sustained development; and (ii) the inclusive growth area , by improving agricultural productivity and resilience. The p erforma nce and l earning r eview (PLR) updated the CPF FY2018 (144812 - HT) and sought to further strengthen the CPF’s robust approach to resilience in 2018 by: (i) increasing funding; (ii) improving targeting to high - risk areas; (iii) sharpening the focus on local c apacity to prepare for and respond to disasters; and (iv) mainstreaming DRM in other sectors. The PDO also remains consistent with the P ilot P rogram for C limate R esilience’s (PPCR ’s ) targeted program of the Strategic Climate Fund (SCF) , as the project indi cators contribute to the PPCR core indicators. Moreover, the PDO is in line with the risk - based, people - centered and multi - hazard e arly w arning s ystems that are being implemented by the GFDRR, the WMO, and the United Nations Office for Disaster Risk Reduct ion under the multi - donor C limate R isk E arly W arning S ystems (CREWS) initiative in the 15 Caribbean c ommunity (CARICOM) member countries. 6 14. The PDO remains a priority for the GoH. The frequency and severity of hydromet hazards in Haiti, which was affected by 18 meteorological mild and severe events over the lifetime of the project (Annex 7), are anticipated to increase because of climate change, with the population and notably the agricultural sector remaining vulnerabl e to their impacts. The PDO contributes to Haiti’s hydromet services under the 2020 Second National Disaster Risk Management Plan 2019 – 2030. 7 , 8 The plan is divided into four strategic axes that are aligned with international and regional commitments, s uch as the Sustainable Development Goals (SDGs) and climate change, 9 and the Caribbean Disaster Management Strategy 10 respectively. The streamlining of the institutional set up and operationalization of the hydromet system will improve the prevention and preparedness capacity element of the PDO that feeds into the GoH’s 2020 DRM 5 A fter the Performance and Learning Review of the CPF was completed in 2018, CPF milestones were extended until 2021 pending the preparation of a new CPF . 6 GFDRR website: < CREWS Caribbean | GFDRR > 7 N DRNP 2020: <www.haitilibre.com/docs/PNGRD_BAT_2020_SMALL.pdf> 8 Decree 103 on the National DRM Plan in Le Moniteur, 15 juin, 2020. 9 UN 2015 SDG 1 (Target 1.5.3: adopt and implement national disaster risk reduction strategies in line with the Sendai Framew ork for Disaster Risk Reduction 2015 - 2030) <https://sdgs.un.org/goals> 10 CDMS 2014: <CDwww.cdema.org/CDMStrategy2014 - 2024.pdf> The World Bank HT Strengthening Hydro - Met Services (P148259) Page 10 of 46 Legal and Institutional Framework 11 (the national DRM System) and the national hydromet unit’s (UHM’s) National Strategic Plan 2020 – 2024 that was prepared under this project. B. ACHIEVEMENT OF PDOs (EFFICACY) Rating: Modest Assessment of Achievement of E ach Objective/Outcome 15. A chievement of the PDO is assessed against the targets set in the 2015 PAD and is unpacked along the lines of the two complementary elements of the PDO that contribute to increasing disaster and climate resilience through: (i) strengthening Haiti’s institutional capacity to prov ide hydromet and climate information services; and (ii) strengthening Haiti’s institutional capacity to customize hydromet and climate information to the needs of the civil protection and agriculture sectors (Annex 1). PDO#1: Strengthening Haiti’s institu tional capacity to provide hydromet and climate information services 16. PDO indicator 1: Data collected from hydromet networks are accessible on a centralized online hydromet data management platform, with its own operating procedures for validation and storage – Target P artially A chieved. The project established the National Hydromet Platform (NHP) for data management that is operated under the newly created UHM under MARNDR’ s tutelage that merged the National Meteorological Center (CNM) and the National Service for Water Resources (SNRE) within the same ministry. UHM is housed in a new building constructed under WMO’s Climate Services to Reduce Vulnerability in Haiti since 2017. 12 However , the onl ine open access of the hydromet platform h as been delayed and was not achieved, although data is provided up on request (Annex 9) . The intermediate result indicator (IRI) 1 on training users and instructors , considered a s an essential element of institution al capacity building, was also not achieved (11 percent of its target). O ther activities that resulted in tangible achievements are discussed below but did not have indicators assigned and are not shown in Table 1. • The project allowed all stakeholders to r ecognize UHM as Haiti’s hydromet focal point and service provider, as UHM was instrumental in providing hydromet information before, during and after Hurricane Laura in August 2020. T he project helped formulate the UHM N ational S trategic P lan 2020 – 2024 of Haiti which was developed by WMO and reviewed by the World Bank by the end of the project . The strategy has five priorities to strengthen the operationalization of UHM through: (i) consolidating the UHM institutional anchor in the existing or planned go vernmentwide institutional set up; (ii) ensuring sustainability of the resources and skills necessary for the proper functioning and development UHM in conjunction with the GoH’s strategies and priorities; (iii) increasing UHM capacity to provide quality s ervices to all users in weather, hydrological and climat e - sensitive areas; (iv) positioning meteorology, hydrology, climatology and related disciplines as a priority area for research and development to tap development funding better; and (v) optimizing Ha iti's effective integration into regional or global hydromet programs. The project has already contributed toward the first and third priorities. T he institutional set up and financial sustainability are an important element of the project, however they ne ed to be duly considered by the other ongoing hydromet projects for funding such as the ongoing multi - donor CREWS project (Annex 9) . 13 11 Decree 103 on the creation and organization of the National DRM System in Le Moniteur, 15 juin, 2020. 12 WMO website: < Climate Services to Reduce Vulnerability in Haiti | World Meteorological Organization (wmo.int) > 13 The CREWS project has three components : (i) regional strategy and identification of priority investments; (ii) institutional The World Bank HT Strengthening Hydro - Met Services (P148259) Page 11 of 46 • The project installed the NHP software with open - source capabilities for hydromet data management under UHM. The NHP, wh ich was developed by WMO, is referred as the meteorological, climatological , and hydrological (MCH) database management system, and was installed instead of the high - tech and costly platform suggested by the consortium hired to develop the hydromet design under Phase I . I n close collaboration with WMO’s ongoing project in Haiti, the hydromet project provided the necessary software and hardware technical support , as well as training and capacity building . The latter is expected to be continued under the CREWS (Annex 9) financed project and by WMO with the support of Météo - France, the Caribbean Institute for Hydrology and Meteorology, the Cuban National Institute for Meteorology (INSMET) , and the Regional Specialized Meteorological Centre in Miami. The MCH standard operating procedures have been developed for operating and maintaining all hydromet data and data collection processes. The training program to develop data management capacity of the producers and users of hydromet data was initiated through rem ote support and activities. However, the target f o r training UHM staff members to develop data management capacity of hydromet data producers and users with WMO support was not met, because of the COVID - 19 pandemic as well as UHM’s institutional weaknesses , and lack of appropriate human resources on the producers’ side. Academics and university students were ultimately hired by the project to support UMH in data collection, validation, and treatment. Table 1: Extent of Achievement of I ntermediate R esults I ndicator (IRI) Targets of PDO Indicator 1 Identify hydromet and climate services’ requirements for select end users and development of information services to support decision making Baseline / Unit Target Number Achieved % Achieved IRI 1. Number of users and instructors trained on the use of the centralized data platform #0 80 9 11% Note: Percentage achieved is color coded: yellow – not achieved set target; and light green – achieved set target. 17. PDO indicator 2: Number of subnetworks feeding into the centralized platform – Target Achieved . The project consolidated and centralized the exis ting inter - institutional hydromet networks’ data that fe e d into the MCH’s NHP. The platform’s permanent server hosts the UHM network , comprising 34 stations from five sub - networks : the National Center for Geo - Spatial Information (CNIGS) of the Ministry of Planning and External Cooperation (MPCE) ; Italy’s Cooperazione Internazionale (COOPI) ; and NGO partners’ networks , Fondation Nouvelle , Grand - Anse and CESAL . H ydromet historical data since 1926 from 230 stations were also added to the platform. These 34 stations will remain under the aegis of the five networks for future operation and maintenance (Annex 9) . The NHP will gradually receive additional information and data from government agencies , including MARNDR’s departmental agricultural directi ons (DDAs) , while several agencies and NGOs will keep updating the MCH. Only one of two IRIs targets was achieved, as discussed below and shown in Table 2 . • The output target for the indicator IRI 2 was achieved by hosting 34 stations in the UHM network fro m five independent sub - networks. • The target of having at least 50 hydromet stations operating was not reached. The institutional diagnostic of the hydromet sector carried out by the project recommended the optimization of the hydromet equipment and stren gthening of data management . The programed training for the UHM staff members to develop equipment management capacity was completed . As recommended by the consortium to optimize the hydromet system, strengthening and capacity building of hydromet services and early warning systems; and (iii) implementation of national - level pilot projects to comprehensively strength en early warning systems. The World Bank HT Strengthening Hydro - Met Services (P148259) Page 12 of 46 the number of stations was increased to 62 stations that were successfully procured and delivered. However, travel restrictions on account of the COVID - 19 pandemic significantly delayed the delivery and hampered the installation of the equipment. Only two stations were installed and were not connected to MCH by closing date . The installation of the stations will be completed under RESEPAG II, with technical support from the PGRAC. Table 2: Extent of Achievement of I ntermediate R esults I ndicator (IRI) Targets of PDO Indicator 2 Identify hydromet and climate services’ requirements for select end users and development of information services to support decision making Baseline / Unit Target Number Achieved % Achieved IRI 2. Hydrological and meteorological stations reporting data to the platform in line with agreed standard operation procedures. #0 34 34 100% IRI 3. Operational stations #0 50 2 4% Note: The target for operational stations was not changed although 62 were procured. Percentage achieved is color coded: yellow – not achieved set target; and light green – achieved set target. PDO#2: Strengthening Haiti’s institutional capacity to c ustomiz e hy dromet and climate information to the needs of the civil protection and agriculture sectors 18. PDO indicator 3: End users' satisfaction rate — of whom 50 percent was female — toward improved hydromet information services – Target not Achieved . The end user satisfaction target of 100 percent was not met because the platform was not online. While the platform is operational and managed by UHM, data from the platform are not directly accessible to end users through an online portal but are only provided up on request which explains the 52 percent satisfactio n rate . The WMO TA is supporting UHM to set up the online portal for direct data sharing. The three IRI targets were not achieved as discussed below and shown in Table 3 : • T he project had planned to analyze the data collected by the hydromet networks and co nsolidated in the national data platform to develop software interfaces customized to meet the needs of the Directorate of Civil Protection and the agriculture sector. The project did not have the time to customize the full application from MCH to be used by the municipal civil protection committees (CCPCs) and the D epartmental A gricultural D irections ( DDAs ). However, it was able to advance in the following areas: scoping the hydromet information requirements of end users and the development of initial cust omized hydromet products to be used by the agriculture sector. Moreover, the quality of the existing UHM hydromet products was improved. The project also provided equipment, such as servers to save and better process the data and carried out maintenance of hydromet stations to ensure the quality and regularity of information. A designated training session for 85 end users, including 13 women, from CCPCs and DDAs in the interpretation and the use of UHM’s hydromet products were carried out and was not captur ed by an indicator. Table 3: Extent of Achievement of I ntermediate R esults I ndicator (IRI) Targets of PDO Indicator 3 Strengthen the institution for hydromet services and development of data management tools Baseline / Unit Target Number Achieved % Achieved IRI 4. Number of c ivil p rotection c ommittees (CCPCs) using the customized application from the centralized data platform #0 40 0 0% IRI 5. Number of d epartmental a gricultural d irections (DDA s ) using the customized application from the centralized data platform #0 10 0 0% IRI 6. Direct project beneficiaries #0 1,000,000 0 0% Note : Percentage achieved is color coded: yellow – not achieved set target; and light green – achieved set target. The World Bank HT Strengthening Hydro - Met Services (P148259) Page 13 of 46 Justification of Overall Efficacy Rating Rating: Modest 19. The project achieved the end target of only one of the three PDO indicators; it also did not meet the end targets of most IRIs. C. EFFICIENCY Rating: Modest Assessment of Efficiency and Rating 20. Economic efficiency . The project ’s ex - ante economic analysis was based on a World Bank report on the us e of international benchmarks to derive potential benefits associated with well - functioning, modern early warning systems that could reduce disaster - related asset losses and improve agricultural productivity. 14 The project had an ex - ante positive return on inv estment , with a present value benefit to cost ratio ranging between 1.6 and 33 over a 15 - year timeframe to account for high uncertainties. At closing, the hydromet system was not fully operational, and the present value of benefit over cost ratio of less t han zero . Despite the additional cost to project management because of COVID - 19 and contract renegotiation owing to force majeure, the ex - ante benefits could well accrue when the hydromet system is fully operational. 21. Design efficiency. The project suffered from design flaws , inadequate institutional arrangements for project implementation, and a poor Results F ramework (RF) that affected the Monitoring and Evaluation arrangements. The design, built on the World Bank’s global hydromet experience, was considered appropriate for a poor, fragile, conflict and violence (FCV) affected country . Components 1 and 2 were straightforwa rd, consisting mainly of consulting services , institutional and interinstitutional capacity building, and the procurement and installation of equipment. However, project management was anchored in the new UHM under MARNDR , with all procurement and contract management delegated to the UPMP ; both institutions had low capacity. The project opted to contract two consortiums instead of one as initially foreseen in the procurement plan, which proved to be ineffective. The World Bank team had to spend considerable time and effort to reconcile the views of both consortiums and provide close TA to the UHM and the new PIU created after the MTR. The initial absence of a PIU, the low capacity of UHM, the UPMP and the new PIU as well as the stalled reform process of MARNDR (see Section IIIB) hampered activities, resulted in serious delays and in the cancelation of US$1.3 million grant (26 percent , see Annex 3). The design did not properly account for the low ca pacity of the institutions. 22. Implementation efficiency . The project faced several endogenous and exogenous factors ( see Sections III and IV) that led to early delays in contract ing and in disbursement processing. MARNDR, which was subjected to an overall reform to streamline processes and improve effectiveness in 2014, displayed weaknesses in manag ing and coordinating the project . After a first mis - procurement (Section IV) , MARNDR selected a new consortium to execute the design, mon itor , and develop the project elements of the first phase. The consortium did not provide the deliverables on time , which affected the project implementation timetable . Based on the MTR recommendations, t he World Bank set up a PIU that enabled the speed ing up of implementation and disbursements , but the weakness of the PIU did not allow i t to complete the project activities . The COVID - 19 pandemic also had some bearing on the final achievements, as most deliverables due between mid - 2019 and mid - 2020 were del ayed, canceled, or transferred to other ongoing projects. O nly 7 5 percent of the grant was disbursed , and the re maining amount was canceled. Yet, the design proved to be 14 A Cost - Effective Solution to Reduce Disaster Losses in Developing Countries: Hydro - Meteorological Services, Early Warning, and Evacuation (World Bank Policy Research, Working Paper #6058, 2012) The World Bank HT Strengthening Hydro - Met Services (P148259) Page 14 of 46 flexible as the MTR recommended the setting up of a PIU while the soft activities and equipment procurement and installation had to be calibrated to respond to time constraints . 23. Administrative efficiency . Given the country and project context, the project showed excessive administrative spending and required extensive World Bank implementation support for: (i) project preparation, where the World Bank cost reached about US$300,000 (8 percent of the disbursed grant, see Annex 2); (ii) project supervision, wh ere the World Bank cost reached about US$450,000 (12 percent of the disbursed grant, see Annex 2), and project coordination to procurement and contract management needed careful and continuous support; and (iii) project management from grant funds reached about US$800,000 (16 percent of the disbursed grant, see Annex 3), although the envelope was not increased despite the introduction of a PIU after the MTR. Despite this considerable expenditure and support, the project did not have a successful outcome. 24. T he discussion above confirms that even after making allowances for the FCV context of Haiti, the grant could have been implemented more efficiently . The efficiency of the project is therefore rated “modest”. D. JUSTIFICATION OF OVERALL OUTCOME RATING Rating: Moderately Unsatisfactory 25. The ratings for the relevance of the PDO, efficacy, and efficiency were respectively rated “high”, “modest”, and “modest”. The overall outcome of the project is rated “Moderately Unsatisfactory”. E. OTHER OUTCOMES AND IMP ACTS 26. Gender. The project set a number of targets for benefiting women; however, none of the targets were reached. These included targets on women’s satisfaction on improved hydromet information services, training of female users and instructors on the cen tralized data platform, and the target number of female project beneficiaries. 27. Institutional Strengthening. The project started building UHM capacity and carrying out institutional and interinstitutional capacity building and data sharing, whose benefits will be reaped once the activities are completed under RESEPAG II. It will also provide a better understanding of hydromet and climate risks and impacts on the economy and on the social fabric, as well as help develop adequate policy responses to support r esilient and sustained development. 28. Mobilizing Private Sector Financing. This was not specifically relevant for this project. 29. Poverty Reduction and Shared Prosperity. When fully operational, the hydromet system is expected to positively affect almost one million beneficiaries who will be covered by the early warning system and meteorological forecasts; these would increase their wellbeing through the prevention of premature deaths, morbidity and injuries. Moreover, farmers will be able to rely on hydromet information and forecasts and take the necessary measures before and after climate events to improve their livestock and agricultural productivity and by extension livelihoods, especially in remote areas where the poor and vulnerable live. 30. Other Unintended Outcomes and Impacts. The World Bank technical and financial assistance under the project supported the creation of the UHM in 2016. The MARNDR merged the hydrological and meteorological services in a single more efficient unit for the strengthening of hydromet services in Hait i. This institutional achievement was also supported by the Climate Services to Reduce Vulnerability in Haiti project carried out by WMO and Environment and Climate Change Canada. While the creation of the UHM was not included as a target of the Hydromet p roject, its creation The World Bank HT Strengthening Hydro - Met Services (P148259) Page 15 of 46 represents a substantial achievement for the country to increase resilience against weather related disasters and climate change. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 31. P roject preparation was challenging , as it had to: (i) set up realistic objectives; (ii) ensure a simple design; (ii i ) select appropriate stakeholders using hydromet services; and (iv) establish adequate risk mitigation measures. During project preparation, the po litical and socioeconomic context was conducive for reform, as development partners and GoH turned the catastrophic 2010 earthquake into an opportunity where, notably, preparedness and hydromet services were high on the GoH agenda . The institutional reform stance included a complete reform of MARNDR and associated public services. In thi