(2021) Pwojè Ranfòse Sèvis Idwo-meteyo ann Ayiti

Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005439 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF-A0226 ON A GRANT IN THE AMOUNT OF SDR 3.5 MILLION (US$5 MILLION EQUIVALENT) TO THE REPUBLIC OF HAITI FOR THE HT Strengthening Hydro-Met Services Project April 13, 2021 Urban, Resilience And Land Global Practice Latin America And Caribbean Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized CURRENCY EQUIVALENTS (Exchange Rate Effective March 15, 2021) Currency Unit = Haitian Gourdes (HTG) HTG78.993 = US$1 US$1.431 = SDR 1 FISCAL YEAR October 1 - September 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Tahseen Sayed Khan Regional Director: Anna Wellenstein Practice Manager: David N. Sislen Task Team Leader(s): Claudia Ruth Soto Orozco ICR Main Contributor: Naraya Carrasco ABBREVIATIONS AND ACRONYMS ACCPAC Account Package CARICOM Caribbean Community CCPC Municipal Civil Protection Committee (Comité Communal de Protection Civile) CIAT Comité Interministériel d’Aménagement du Territoire (Inter-ministerial Committee for Territorial Planning) CIF Climate Investment Funds CNIGS Centre National de l'Information Géo-Spatiale (National Center for Geo-Spatial Information) CNM Centre National de Météorologie (National Center for Meteorology) CNSA Coordination Nationale de la Sécurité Alimentaire (National Coordination for Food Security) COOPI Cooperazione Internazionale (Italy’s International Cooperation) CPF Country Partnership Framework DCC Division Changement Climatique (Climate Change Directorate) DDA Direction Départementale de l’Agriculture (Departmental Directorate for Agriculture) DG Director General DIA Direction des Infrastructures Agricoles (Agriculture Infrastructure Directorate) DPC Direction de la Protection Civile (Civil Protection Directorate) DRE Direction des Ressources en Eau (Water Resource Directorate) EIRR Economic Internal Rate of Return EMP Environmental Management Plan ESMF Environmental and Social Management Framework EU European Union EWS Early Warning System FCV Fragile, Conflict and Violence GDP Gross Domestic Product GFDRR Global Facility for Disaster Reduction and Recovery GIS Geographical Information System GoH Government of Haiti HT Haiti IDB Inter-American Development Bank INSMET Cuban National Institute for Meteorology (Instituto de Meteorología de la República de Cuba) ISN Interim Strategy Note ISR Implementation Status and Results MARNDR Ministère de l’Agriculture, Ressources Naturelles et Développement Rural (Ministry of Agriculture, Natural Resources and Rural Development) MCH Meteorological, Climatological and Hydrological Database Management System MDE Ministère de l'Environnement (Ministry of the Environment) MICT Ministère de l’Intérieur et des Collectivités Territoriales (Ministry of Interior and Territorial Communities) MIS Management Information System MPCE Ministère du Plan et de la Coopération Externe (Ministry of Planning and External Cooperation) MTPTC Ministère des Travaux Publics, Transports et Communications (Ministry of Public Works, Transport and Communications) NAPA National Action Plan for Adaptation NHP National Hydro-met Platform NPV Net Present Value OFNAC Office National de l’Aviation Civile (National Civil Aviation Agency) OM Operations Manual ONEV Observatoire National de l’Environnement et de la Vulnérabilité (National Observatory on Environment and Vulnerability) OP Operational Policy PAD Project Appraisal Document PDO Project Development Objective PGRAC Strengthening Disaster Risk Management and Climate Resilience Project (Projet de Gestion des Risques et de Résilience aux Aléas Climatique) PIU Project Implementation Unit PNAP Programme National d’Alerte Précoce (National Early Warning Project) PPCR Pilot Program for Climate Resilience RESEPAG II Relaunching Agriculture: Strengthening Agriculture Public Services Project (Projet de Renforcement des Services Publics Agricoles II) RF Results Framework SCF Strategic Climate Fund SDGs Sustainable Development Goals SDR Special Drawing Rights SEMANAH Service Maritime et de Navigation d'Haïti (Haiti National Maritime and Navigation Service) SNRE Service National des Ressources en Eau (National Service for Water Resources) SOP Standard operating procedures SORT Systematic Operations Risk-rating Tool SPCR Strategic Program for Climate Resilience STEP Systematic Tracking of Exchanges in Procurement TA Technical Assistance ToC Theory of Change UEP Unité d’Études et de Programmation (Study and Planning Unit, MARNDR) UHM Unité Hydrométéorologique d’Haïti (Hydrometeorological Unit of Haiti) UIS Unité Informatique et Statistique (Statistics and IT Unit, MARNDR) UPMP Unité de Passation de Marchés Publics (Centralized Procurement Unit, MARNDR) USAID United States Agency for International Development WMO World Meteorological Organization TABLE OF CONTENTS DATA SHEET ................................................................................................................................1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES .........................................................5 A. CONTEXT AT APPRAISAL ............................................................................................................. 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ......................................... 8 II. OUTCOME ...........................................................................................................................9 A. RELEVANCE OF PDOs .................................................................................................................. 9 B. ACHIEVEMENT OF PDOs (EFFICACY) .......................................................................................... 10 C. EFFICIENCY ................................................................................................................................ 13 D. JUSTIFICATION OF OVERALL OUTCOME RATING....................................................................... 14 E. OTHER OUTCOMES AND IMPACTS ............................................................................................ 14 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ..................................15 A. KEY FACTORS DURING PREPARATION ....................................................................................... 15 B. KEY FACTORS DURING IMPLEMENTATION ................................................................................ 16 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ...18 A. QUALITY OF MONITORING AND EVALUATION (M&E) .............................................................. 18 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ....................................................... 19 C. BANK PERFORMANCE ............................................................................................................... 20 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................... 21 V. LESSONS AND RECOMMENDATIONS .................................................................................22 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ..............................................................24 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION...........................32 ANNEX 3. PROJECT COST BY COMPONENT ............................................................................... 35 ANNEX 4. EFFICIENCY ANALYSIS ............................................................................................... 36 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 37 ANNEX 6. SUPPORTING DOCUMENTS ......................................................................................42 ANNEX 7. PROJECT IMPLEMENTATION TIMELINE: EXOGENOUS FACTORS ...............................43 ANNEX 8. PROJECT RESULTS FRAMEWORK AND PPCR CORE INDICATORS...............................44 ANNEX 9. PROJECT COMPONENT ACHIEVEMENTS AND LINK TO CREWS .................................45 The World Bank HT Strengthening Hydro-Met Services (P148259) Page 1 of 46 DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P148259 HT Strengthening Hydro-Met Services Country Financing Instrument Haiti Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of Haiti Ministry of Agriculture, Natural Resources and Rural Development (MARNDR) Project Development Objective (PDO) Original PDO The Project Development Objective (PDO) is to strengthen the Republic of Haiti's institutional capacity to providehydro-meteorological and climate information services customized to the needs of the civil protection and agriculture sectors,which contributes to increasing disaster and climate resilience. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 2 of 46 FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing TF-A0226 5,000,000 5,000,000 3,742,007 Total 5,000,000 5,000,000 3,742,007 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 5,000,000 5,000,000 3,742,007 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 26-Jun-2015 19-Oct-2015 12-Nov-2018 30-Jun-2020 30-Oct-2020 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 26-Jun-2020 3.70 Change in Loan Closing Date(s) Change in Implementation Schedule 30-Jun-2020 3.70 KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Unsatisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs No. Date ISR Archived DO Rating IP Rating Actual Disbursements (US$M) 01 24-Nov-2015 Satisfactory Satisfactory 0 The World Bank HT Strengthening Hydro-Met Services (P148259) Page 3 of 46 02 08-Jun-2016 Satisfactory Moderately Satisfactory .50 03 27-Dec-2016 Moderately Satisfactory Moderately Satisfactory .50 04 18-May-2017 Moderately Satisfactory Moderately Unsatisfactory .51 05 05-Dec-2017 Moderately Satisfactory Moderately Unsatisfactory .52 06 18-May-2018 Moderately Satisfactory Moderately Satisfactory .52 07 31-Dec-2018 Moderately Satisfactory Moderately Satisfactory .85 08 26-Jun-2019 Moderately Unsatisfactory Moderately Unsatisfactory .85 09 31-Dec-2019 Moderately Unsatisfactory Moderately Unsatisfactory 1.29 10 30-Oct-2020 Moderately Unsatisfactory Moderately Unsatisfactory 3.70 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 14 Other Agriculture, Fishing and Forestry 14 Water, Sanitation and Waste Management 86 Public Administration - Water, Sanitation and Waste Management 72 Other Water Supply, Sanitation and Waste Management 14 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Finance 16 Finance for Development 16 Disaster Risk Finance 16 The World Bank HT Strengthening Hydro-Met Services (P148259) Page 4 of 46 Public Sector Management 36 Public Administration 36 E-Government, incl. e-services 36 Urban and Rural Development 48 Disaster Risk Management 48 Disaster Response and Recovery 16 Disaster Risk Reduction 16 Disaster Preparedness 16 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Carlos Felipe Jaramillo Country Director: Mary A. Barton-Dock Tahseen Sayed Khan Director: Ede Jorge Ijjasz-Vasquez Anna Wellenstein Practice Manager: Anna Wellenstein David N. Sislen Task Team Leader(s): Gaetano Vivo Claudia Ruth Soto Orozco ICR Contributing Author: Naraya Carrasco The World Bank HT Strengthening Hydro-Met Services (P148259) Page 5 of 46 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At appraisal in May 2015, Haiti was still confronting considerable challenges due to exogenous and endogenous factors. Haiti continued to be severely affected by the aftermath of the catastrophic 2010 earthquake that led to a toll of 220,000 casualties, including one in three civil servants, 1.5 million people displaced, and the equivalent of 120 percent of gross domestic product (GDP) destroyed, while facing a growing political and socioeconomic crisis. Climate change was projected to heighten the risk of hydrometeorological (hydromet) hazards, with an increased frequency and intensity of extreme events such as Hurricane Sandy in 2012—about 75 deaths, 200,000 affected and US$0.5 billion in damages. The poverty rate was 58.5 percent, with almost 6.3 million Haitians unable to meet their basic needs and one million Haitians living slightly above the poverty level, risking falling back into poverty because of external shocks, including climatic-induced shocks. 2. Haiti’s economic recovery and growth potential could not be sustained without improving the country’s meteorological data to support early warning systems to enhance its resilience as a whole and informing the agricultural sector. A World Bank global study 1 ranked Haiti fifth in exposure to the risk of two or more hazards, with more than 93 percent of its territory and 96 percent of its population at risk of two or more hazards and 56 percent of its GDP linked to an area exposed to risk stemming from two or more hazards. High coastal population density, informal settlements, weak infrastructure and flood-prone agricultural land render Haiti’s population and the agricultural sector particularly vulnerable. Over the past decade, hydromet hazards have claimed lives, affected livelihoods, and have taken a heavy toll on all sectors of the Haitian economy, especially in agriculture and food security. Rationale for World Bank Engagement 3. Four years after the 2010 Earthquake, the Government of Haiti (GoH) had embarked on a structural reform of the country where catastrophic events and climate change were high on national priority. In this context, a World Bank recipient-executed trust fund grant was provided to Haiti under the Pilot Program for Climate Resilience (PPCR) under the Strategic Climate Fund. At appraisal, the objectives of the project aligned with the World Bank Group’s Haiti Interim Strategy Note (ISN) FY13–FY14 (Report #71885-HT) and supported two of the four objectives of the ISN: (i) reducing vulnerability and increasing resilience, and (ii) promoting inclusive growth. It also contributed to the ISN crosscutting theme of strengthening governance. Moreover, the World Bank had a comparative advantage in hydromet operations and experience in several countries, in conjunction with the Global Facility for Disaster Reduction and Recovery (GFDRR) Hydromet Program, to better predict, prepare, and respond to disastrous events, manage natural resources—water in particular—and better understand climate change effects, as well as support transport activities in aviation and navigation. The project aligned with the GoH 2013 National Strategic Program for Climate Resilience (SPCR), which included “Strengthening Knowledge Management of Hydro-met, Water Resources, and Climate Data to Inform Decision Making and Policy Dialogue”. It also contributed to the implementation of Haiti’s 2006 National Action Plan for Adaptation (NAPA). The project design responded to challenges and opportunities highlighted in various studies, such as 1 World Bank, Natural disaster hotspots: A global risk analysis, 2005. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 6 of 46 those of the World Meteorological Organization (WMO 2010 and 2012), 2 the Interamerican Development Bank (2014) 3 and the World Bank (2014). 4 The 2012 Haiti Strategic Program for Climate Resilience (SPCR) especially pointed to a new institutional framework for the hydromet services with the following priorities: (i) rationalizing and streamlining the national data collection and management system; (ii) establishing a mechanism to bring together data producers and main end users—aviation, civil protection, water utility—and providing feedback, and increasing the socioeconomic value of data; and (iii) ensuring a sustainable financing model based on a cost-recovery principle. Theory of Change (Results Chain) 4. The project appraisal document (PAD) did not include a theory of change (ToC). The inferred ToC (Figure 1), including the critical assumptions, are presented below: Figure 1: Theory of Change - Hydromet data accessible through a centralized platform will support decision-making processes and therefore increase preparedness and overall resilience of the Haitian population, as well as agricultural production. 2 The re-establishment of meteorological services in Haiti in 2010 with a website (www.meteo-haiti.gouv.ht) to disseminate information and warnings to United Nations’ agencies, nongovernmental organizations (NGOs), disaster-affected communities, and the media occurred thanks to collaboration between WMO and other development partners. WMO also partnered with Environment and Climate Change Canada in 2012 to develop the project “Climate Services to Reduce Vulnerability in Haiti” whose aim was to install technical equipment such as forecaster workstations, Internet, communications, and related technology in a newly built UHM headquarters. Website: <Climate Services to Reduce Vulnerability in Haiti | World Meteorological Organization (wmo.int)> 3 IDB website: <Background Paper: LAC Small Island Development States | Publications (iadb.org)> 4 World Bank website: <WorldBank-rapid-global-assessment.pdf (whycos.org)> The World Bank HT Strengthening Hydro-Met Services (P148259) Page 7 of 46 - A strong hydromet unit in Haiti (UHM) and government officials well-trained in the collection and management of hydromet data will result in on-time preparedness activities, thus enhancing the protection of the population and improving agricultural production and livelihoods. Project Development Objectives (PDOs) 5. The Project Development Objective (PDO), as stated in the financial agreement approved on June 26, 2015, which is consistent with the PDO statement in the PAD, is to strengthen Haiti’s institutional capacity to provide hydrometeorological and climate information services customized to the needs of the civil protection and agriculture sectors, which contribute to increasing disaster and climate resilience Key Expected Outcomes and Outcome Indicators 6. The key outcome indicators were: (i) Data collected from hydromet networks are accessible on a centralized online data management platform, with standard operating procedures for validation and storage; (ii) Number of subnetworks feeding into the centralized platform; and (iii) End users’ satisfaction rate toward improved hydromet information services, percentage and gender- disaggregated. Components 7. The project included three components: - Component 1, Institutional strengthening of the hydrometeorological services and development of data management tools (Special Drawing Rights {SDR} 2.1 million / US$3 million): 1.1. Institutional strengthening, capacity building, and partnership development. Provision of support, among others, for: 1. Carrying out technical assistance (TA) to develop a long-term strategy based on sustainable financing and staffing model for the national entity that will be responsible for the development, operation, and maintenance of all hydrometeorological services. 2. Carrying out training and knowledge exchange programs to develop data management capacity of the producers and users of hydrometeorological data. 3. Developing applications and applied research programs aimed at supporting students or young professionals or both groups selected in accordance with the criteria set forth in the project operations manual in carrying out field work to improve hydromet and climate data collection and management. 1.2. Strengthening interinstitutional data sharing and coordination. Provision of support, among others, for: 1. Designing a central data platform for hydrometeorological data integration based on an open data approach through: (i) the implementation of a countrywide, georeferenced baseline assessment of hydrometeorological equipment; (ii) the proposal for an optimal network, which would meet priority national needs while taking into account the maintenance and operational budget; (iii) the development of the architecture and technical specifications of a data platform capable to gather data from all existing hydrometeorological sensors and feed in real time several user-specific interfaces; and (iv) the development of standard procedures for equipment and data management. 2. Developing specialized interfaces based on a geographical information system (GIS) for select end users to allow them access to hydrometeorological information. 3. Carrying out training for hydrometeorological data producers and users on the use and maintenance of equipment and management of the online data management platform, including the specialized interfaces. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 8 of 46 4. Repairing, replacing, or upgrading existing hydrometeorological equipment throughout the recipient's territory based on the assessment carried out in Part 1.2.1. - Component 2, Identifying hydrometeorological and climate services' requirements for select end users and developing information services to support decision making (SDR 1.0 million / US$1.4 million). Provision of support for: 1. Scoping hydrometeorological information requirements of end users, pursuant to methodology and selection criteria set forth in the project operational manual. 2. Updating operating procedures for the optimal use of hydrometeorological services for agriculture and food security through initiating a pilot information service for farmers and other food security stakeholders, designed to factor requirements from end users of the food security and agricultural communities. 3. Updating operating procedures for the optimal use of hydrometeorological services for civil protection through the launch of an information service for civil protection and through the provision of expert assistance for early warning and emergency response in pilot zones. 4. Updating the estimated return period of selected weather hazards, including floods, windstorms, and storm surges, to facilitate the design of infrastructure and planning of development processes. - Component 3, Project implementation, evaluation and monitoring and pilot program for climate resilience (PPCR) knowledge management (SDR 0.4 million / US$0.6 million): Provision of support to the Ministry of Agriculture, Natural Resources and Rural Development (MARNDR) for project implementation, management, procurement, and monitoring and evaluation. The project relied on a multi-project centralized procurement unit (UPMP), also in charge of contract management that was established by the World Bank in 2014 under MARNDR. The creation of the UPMP aimed to strengthen MARNDR’s capacity to manage projects. The project coordinator was selected by MARNDR among the ministry’s staff B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 8. The PDO indicators and outcome targets were not changed during the lifetime of the project. Revised PDO Indicators 9. The PDO indicators were not changed during the lifetime of the project. Revised Components 10. Under Component 3, the project had initially planned to rely on MARNDR's own staff and the UPMP for procurement and project management to avoid the need for a full project implementation unit (PIU). However, given MARNDR’s capacity constraints and the resignation of the Project Coordinator, the project ended up mobilizing a full PIU after the project midterm review (MTR). Other Changes 11. Further to the creation of the PIU after the MTR, the project was restructured in June 2020 to extend the grant closing date by four months from June 30, 2020 to October 30, 2020 (because of delays resulting from the COVID-19 pandemic) so that the project could take delivery of the 62 hydromet stations. The residual grant funds of US$1.3 million were canceled and the stations are installed under the World Bank’s “Relaunching Agriculture: Strengthening Agriculture Public Services II Project” (RESEPAG II - P126744). Further TA and safeguards compliance monitoring to accompany the The World Bank HT Strengthening Hydro-Met Services (P148259) Page 9 of 46 installation process and support the hydromet sector will continue under the GFDRR TA, which is available under the World Bank’s “Strengthening Disaster Risk Management and Climate Resilience Project” (PGRAC - P165870). Rationale for Changes and Their Implication on the Original Theory of Change 12. The extension of the closing date intended to allow the project to complete shipment, delivery and installation of hydromet equipment. The limited project extension (four months) affected the accomplishment of the short-term outcomes shown in the Theory of Change. II. OUTCOME A. RELEVANCE OF PDOs Rating: High Assessment of Relevance of PDOs and Rating 13. The project objective remains highly relevant at closing. The PDO remains consistent with the country partnership framework (CPF) FY2016–19 (98132-HT)—which is the ongoing CPF 5 —as it contributes to two of the three areas of CPF’s focus: (i) the resilience area, supported by enhanced understanding of hydrometeorological and climate risks to assess social and economic impacts, and to develop adequate policy responses to support sustained development; and (ii) the inclusive growth area, by improving agricultural productivity and resilience. The performance and learning review (PLR) updated the CPF FY2018 (144812-HT) and sought to further strengthen the CPF’s robust approach to resilience in 2018 by: (i) increasing funding; (ii) improving targeting to high-risk areas; (iii) sharpening the focus on local capacity to prepare for and respond to disasters; and (iv) mainstreaming DRM in other sectors. The PDO also remains consistent with the Pilot Program for Climate Resilience’s (PPCR’s) targeted program of the Strategic Climate Fund (SCF), as the project indicators contribute to the PPCR core indicators. Moreover, the PDO is in line with the risk- based, people-centered and multi-hazard early warning systems that are being implemented by the GFDRR, the WMO, and the United Nations Office for Disaster Risk Reduction under the multi-donor Climate Risk Early Warning Systems (CREWS) initiative in the 15 Caribbean community (CARICOM) member countries. 6 14. The PDO remains a priority for the GoH. The frequency and severity of hydromet hazards in Haiti, which was affected by 18 meteorological mild and severe events over the lifetime of the project (Annex 7), are anticipated to increase because of climate change, with the population and notably the agricultural sector remaining vulnerable to their impacts. The PDO contributes to Haiti’s hydromet services under the 2020 Second National Disaster Risk Management Plan 2019–2030. 7, 8 The plan is divided into four strategic axes that are aligned with international and regional commitments, such as the Sustainable Development Goals (SDGs) and climate change, 9 and the Caribbean Disaster Management Strategy 10 respectively. The streamlining of the institutional set up and operationalization of the hydromet system will improve the prevention and preparedness capacity element of the PDO that feeds into the GoH’s 2020 DRM 5 After the Performance and Learning Review of the CPF was completed in 2018, CPF milestones were extended until 2021 pending the preparation of a new CPF. 6 GFDRR website: <CREWS Caribbean | GFDRR> 7 NDRNP 2020: <www.haitilibre.com/docs/PNGRD_BAT_2020_SMALL.pdf> 8 Decree 103 on the National DRM Plan in Le Moniteur, 15 juin, 2020. 9 UN 2015 SDG 1 (Target 1.5.3: adopt and implement national disaster risk reduction strategies in line with the Sendai Framework for Disaster Risk Reduction 2015-2030) <https://sdgs.un.org/goals> 10 CDMS 2014: <CDwww.cdema.org/CDMStrategy2014-2024.pdf> The World Bank HT Strengthening Hydro-Met Services (P148259) Page 10 of 46 Legal and Institutional Framework 11 (the national DRM System) and the national hydromet unit’s (UHM’s) National Strategic Plan 2020–2024 that was prepared under this project. B. ACHIEVEMENT OF PDOs (EFFICACY) Rating: Modest Assessment of Achievement of Each Objective/Outcome 15. Achievement of the PDO is assessed against the targets set in the 2015 PAD and is unpacked along the lines of the two complementary elements of the PDO that contribute to increasing disaster and climate resilience through: (i) strengthening Haiti’s institutional capacity to provide hydromet and climate information services; and (ii) strengthening Haiti’s institutional capacity to customize hydromet and climate information to the needs of the civil protection and agriculture sectors (Annex 1). PDO#1: Strengthening Haiti’s institutional capacity to provide hydromet and climate information services 16. PDO indicator 1: Data collected from hydromet networks are accessible on a centralized online hydromet data management platform, with its own operating procedures for validation and storage – Target Partially Achieved. The project established the National Hydromet Platform (NHP) for data management that is operated under the newly created UHM under MARNDR’s tutelage that merged the National Meteorological Center (CNM) and the National Service for Water Resources (SNRE) within the same ministry. UHM is housed in a new building constructed under WMO’s Climate Services to Reduce Vulnerability in Haiti since 2017. 12 However, the online open access of the hydromet platform has been delayed and was not achieved, although data is provided upon request (Annex 9). The intermediate result indicator (IRI) 1 on training users and instructors, considered as an essential element of institutional capacity building, was also not achieved (11 percent of its target). Other activities that resulted in tangible achievements are discussed below but did not have indicators assigned and are not shown in Table 1. • The project allowed all stakeholders to recognize UHM as Haiti’s hydromet focal point and service provider, as UHM was instrumental in providing hydromet information before, during and after Hurricane Laura in August 2020. The project helped formulate the UHM National Strategic Plan 2020–2024 of Haiti which was developed by WMO and reviewed by the World Bank by the end of the project. The strategy has five priorities to strengthen the operationalization of UHM through: (i) consolidating the UHM institutional anchor in the existing or planned governmentwide institutional set up; (ii) ensuring sustainability of the resources and skills necessary for the proper functioning and development UHM in conjunction with the GoH’s strategies and priorities; (iii) increasing UHM capacity to provide quality services to all users in weather, hydrological and climate-sensitive areas; (iv) positioning meteorology, hydrology, climatology and related disciplines as a priority area for research and development to tap development funding better; and (v) optimizing Haiti's effective integration into regional or global hydromet programs. The project has already contributed toward the first and third priorities. The institutional set up and financial sustainability are an important element of the project, however they need to be duly considered by the other ongoing hydromet projects for funding such as the ongoing multi-donor CREWS project (Annex 9). 13 11 Decree 103 on the creation and organization of the National DRM System in Le Moniteur, 15 juin, 2020. 12 WMO website: <Climate Services to Reduce Vulnerability in Haiti | World Meteorological Organization (wmo.int)> 13 The CREWS project has three components: (i) regional strategy and identification of priority investments; (ii) institutional The World Bank HT Strengthening Hydro-Met Services (P148259) Page 11 of 46 • The project installed the NHP software with open-source capabilities for hydromet data management under UHM. The NHP, which was developed by WMO, is referred as the meteorological, climatological, and hydrological (MCH) database management system, and was installed instead of the high-tech and costly platform suggested by the consortium hired to develop the hydromet design under Phase I. In close collaboration with WMO’s ongoing project in Haiti, the hydromet project provided the necessary software and hardware technical support, as well as training and capacity building. The latter is expected to be continued under the CREWS (Annex 9) financed project and by WMO with the support of Météo-France, the Caribbean Institute for Hydrology and Meteorology, the Cuban National Institute for Meteorology (INSMET), and the Regional Specialized Meteorological Centre in Miami. The MCH standard operating procedures have been developed for operating and maintaining all hydromet data and data collection processes. The training program to develop data management capacity of the producers and users of hydromet data was initiated through remote support and activities. However, the target for training UHM staff members to develop data management capacity of hydromet data producers and users with WMO support was not met, because of the COVID-19 pandemic as well as UHM’s institutional weaknesses, and lack of appropriate human resources on the producers’ side. Academics and university students were ultimately hired by the project to support UMH in data collection, validation, and treatment. Table 1: Extent of Achievement of Intermediate Results Indicator (IRI) Targets of PDO Indicator 1 Identify hydromet and climate services’ requirements for select end users and development of information services to support decision making Baseline / Unit Target Number Achieved % Achieved IRI 1. Number of users and instructors trained on the use of the centralized data platform #0 80 9 11% Note: Percentage achieved is color coded: yellow – not achieved set target; and light green – achieved set target. 17. PDO indicator 2: Number of subnetworks feeding into the centralized platform – Target Achieved. The project consolidated and centralized the existing inter-institutional hydromet networks’ data that feed into the MCH’s NHP. The platform’s permanent server hosts the UHM network, comprising 34 stations from five sub-networks: the National Center for Geo-Spatial Information (CNIGS) of the Ministry of Planning and External Cooperation (MPCE); Italy’s Cooperazione Internazionale (COOPI); and NGO partners’ networks, Fondation Nouvelle, Grand-Anse and CESAL. Hydromet historical data since 1926 from 230 stations were also added to the platform. These 34 stations will remain under the aegis of the five networks for future operation and maintenance (Annex 9). The NHP will gradually receive additional information and data from government agencies, including MARNDR’s departmental agricultural directions (DDAs), while several agencies and NGOs will keep updating the MCH. Only one of two IRIs targets was achieved, as discussed below and shown in Table 2. • The output target for the indicator IRI 2 was achieved by hosting 34 stations in the UHM network from five independent sub-networks. • The target of having at least 50 hydromet stations operating was not reached. The institutional diagnostic of the hydromet sector carried out by the project recommended the optimization of the hydromet equipment and strengthening of data management. The programed training for the UHM staff members to develop equipment management capacity was completed. As recommended by the consortium to optimize the hydromet system, strengthening and capacity building of hydromet services and early warning systems; and (iii) implementation of national-level pilot projects to comprehensively strengthen early warning systems. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 12 of 46 the number of stations was increased to 62 stations that were successfully procured and delivered. However, travel restrictions on account of the COVID-19 pandemic significantly delayed the delivery and hampered the installation of the equipment. Only two stations were installed and were not connected to MCH by closing date. The installation of the stations will be completed under RESEPAG II, with technical support from the PGRAC. Table 2: Extent of Achievement of Intermediate Results Indicator (IRI) Targets of PDO Indicator 2 Identify hydromet and climate services’ requirements for select end users and development of information services to support decision making Baseline / Unit Target Number Achieved % Achieved IRI 2. Hydrological and meteorological stations reporting data to the platform in line with agreed standard operation procedures. #0 34 34 100% IRI 3. Operational stations #0 50 2 4% Note: The target for operational stations was not changed although 62 were procured. Percentage achieved is color coded: yellow – not achieved set target; and light green – achieved set target. PDO#2: Strengthening Haiti’s institutional capacity to customize hydromet and climate information to the needs of the civil protection and agriculture sectors 18. PDO indicator 3: End users' satisfaction rate — of whom 50 percent was female — toward improved hydromet information services – Target not Achieved. The end user satisfaction target of 100 percent was not met because the platform was not online. While the platform is operational and managed by UHM, data from the platform are not directly accessible to end users through an online portal but are only provided upon request which explains the 52 percent satisfaction rate. The WMO TA is supporting UHM to set up the online portal for direct data sharing. The three IRI targets were not achieved as discussed below and shown in Table 3: • The project had planned to analyze the data collected by the hydromet networks and consolidated in the national data platform to develop software interfaces customized to meet the needs of the Directorate of Civil Protection and the agriculture sector. The project did not have the time to customize the full application from MCH to be used by the municipal civil protection committees (CCPCs) and the Departmental Agricultural Directions (DDAs). However, it was able to advance in the following areas: scoping the hydromet information requirements of end users and the development of initial customized hydromet products to be used by the agriculture sector. Moreover, the quality of the existing UHM hydromet products was improved. The project also provided equipment, such as servers to save and better process the data and carried out maintenance of hydromet stations to ensure the quality and regularity of information. A designated training session for 85 end users, including 13 women, from CCPCs and DDAs in the interpretation and the use of UHM’s hydromet products were carried out and was not captured by an indicator. Table 3: Extent of Achievement of Intermediate Results Indicator (IRI) Targets of PDO Indicator 3 Strengthen the institution for hydromet services and development of data management tools Baseline / Unit Target Number Achieved % Achieved IRI 4. Number of civil protection committees (CCPCs) using the customized application from the centralized data platform #0 40 0 0% IRI 5. Number of departmental agricultural directions (DDAs) using the customized application from the centralized data platform #0 10 0 0% IRI 6. Direct project beneficiaries #0 1,000,000 0 0% Note: Percentage achieved is color coded: yellow – not achieved set target; and light green – achieved set target. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 13 of 46 Justification of Overall Efficacy Rating Rating: Modest 19. The project achieved the end target of only one of the three PDO indicators; it also did not meet the end targets of most IRIs. C. EFFICIENCY Rating: Modest Assessment of Efficiency and Rating 20. Economic efficiency. The project’s ex-ante economic analysis was based on a World Bank report on the use of international benchmarks to derive potential benefits associated with well-functioning, modern early warning systems that could reduce disaster-related asset losses and improve agricultural productivity. 14 The project had an ex-ante positive return on investment, with a present value benefit to cost ratio ranging between 1.6 and 33 over a 15-year timeframe to account for high uncertainties. At closing, the hydromet system was not fully operational, and the present value of benefit over cost ratio of less than zero. Despite the additional cost to project management because of COVID- 19 and contract renegotiation owing to force majeure, the ex-ante benefits could well accrue when the hydromet system is fully operational. 21. Design efficiency. The project suffered from design flaws, inadequate institutional arrangements for project implementation, and a poor Results Framework (RF) that affected the Monitoring and Evaluation arrangements. The design, built on the World Bank’s global hydromet experience, was considered appropriate for a poor, fragile, conflict and violence (FCV) affected country. Components 1 and 2 were straightforward, consisting mainly of consulting services, institutional and interinstitutional capacity building, and the procurement and installation of equipment. However, project management was anchored in the new UHM under MARNDR, with all procurement and contract management delegated to the UPMP; both institutions had low capacity. The project opted to contract two consortiums instead of one as initially foreseen in the procurement plan, which proved to be ineffective. The World Bank team had to spend considerable time and effort to reconcile the views of both consortiums and provide close TA to the UHM and the new PIU created after the MTR. The initial absence of a PIU, the low capacity of UHM, the UPMP and the new PIU as well as the stalled reform process of MARNDR (see Section IIIB) hampered activities, resulted in serious delays and in the cancelation of US$1.3 million grant (26 percent, see Annex 3). The design did not properly account for the low capacity of the institutions. 22. Implementation efficiency. The project faced several endogenous and exogenous factors (see Sections III and IV) that led to early delays in contracting and in disbursement processing. MARNDR, which was subjected to an overall reform to streamline processes and improve effectiveness in 2014, displayed weaknesses in managing and coordinating the project. After a first mis-procurement (Section IV), MARNDR selected a new consortium to execute the design, monitor, and develop the project elements of the first phase. The consortium did not provide the deliverables on time, which affected the project implementation timetable. Based on the MTR recommendations, the World Bank set up a PIU that enabled the speeding up of implementation and disbursements, but the weakness of the PIU did not allow it to complete the project activities. The COVID-19 pandemic also had some bearing on the final achievements, as most deliverables due between mid-2019 and mid-2020 were delayed, canceled, or transferred to other ongoing projects. Only 75 percent of the grant was disbursed, and the remaining amount was canceled. Yet, the design proved to be 14 A Cost-Effective Solution to Reduce Disaster Losses in Developing Countries: Hydro-Meteorological Services, Early Warning, and Evacuation (World Bank Policy Research, Working Paper #6058, 2012) The World Bank HT Strengthening Hydro-Met Services (P148259) Page 14 of 46 flexible as the MTR recommended the setting up of a PIU while the soft activities and equipment procurement and installation had to be calibrated to respond to time constraints. 23. Administrative efficiency. Given the country and project context, the project showed excessive administrative spending and required extensive World Bank implementation support for: (i) project preparation, where the World Bank cost reached about US$300,000 (8 percent of the disbursed grant, see Annex 2); (ii) project supervision, where the World Bank cost reached about US$450,000 (12 percent of the disbursed grant, see Annex 2), and project coordination to procurement and contract management needed careful and continuous support; and (iii) project management from grant funds reached about US$800,000 (16 percent of the disbursed grant, see Annex 3), although the envelope was not increased despite the introduction of a PIU after the MTR. Despite this considerable expenditure and support, the project did not have a successful outcome. 24. The discussion above confirms that even after making allowances for the FCV context of Haiti, the grant could have been implemented more efficiently. The efficiency of the project is therefore rated “modest”. D. JUSTIFICATION OF OVERALL OUTCOME RATING Rating: Moderately Unsatisfactory 25. The ratings for the relevance of the PDO, efficacy, and efficiency were respectively rated “high”, “modest”, and “modest”. The overall outcome of the project is rated “Moderately Unsatisfactory”. E. OTHER OUTCOMES AND IMPACTS 26. Gender. The project set a number of targets for benefiting women; however, none of the targets were reached. These included targets on women’s satisfaction on improved hydromet information services, training of female users and instructors on the centralized data platform, and the target number of female project beneficiaries. 27. Institutional Strengthening. The project started building UHM capacity and carrying out institutional and interinstitutional capacity building and data sharing, whose benefits will be reaped once the activities are completed under RESEPAG II. It will also provide a better understanding of hydromet and climate risks and impacts on the economy and on the social fabric, as well as help develop adequate policy responses to support resilient and sustained development. 28. Mobilizing Private Sector Financing. This was not specifically relevant for this project. 29. Poverty Reduction and Shared Prosperity. When fully operational, the hydromet system is expected to positively affect almost one million beneficiaries who will be covered by the early warning system and meteorological forecasts; these would increase their wellbeing through the prevention of premature deaths, morbidity and injuries. Moreover, farmers will be able to rely on hydromet information and forecasts and take the necessary measures before and after climate events to improve their livestock and agricultural productivity and by extension livelihoods, especially in remote areas where the poor and vulnerable live. 30. Other Unintended Outcomes and Impacts. The World Bank technical and financial assistance under the project supported the creation of the UHM in 2016. The MARNDR merged the hydrological and meteorological services in a single more efficient unit for the strengthening of hydromet services in Haiti. This institutional achievement was also supported by the Climate Services to Reduce Vulnerability in Haiti project carried out by WMO and Environment and Climate Change Canada. While the creation of the UHM was not included as a target of the Hydromet project, its creation The World Bank HT Strengthening Hydro-Met Services (P148259) Page 15 of 46 represents a substantial achievement for the country to increase resilience against weather related disasters and climate change. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 31. Project preparation was challenging, as it had to: (i) set up realistic objectives; (ii) ensure a simple design; (iii) select appropriate stakeholders using hydromet services; and (iv) establish adequate risk mitigation measures. During project preparation, the political and socioeconomic context was conducive for reform, as development partners and GoH turned the catastrophic 2010 earthquake into an opportunity where, notably, preparedness and hydromet services were high on the GoH agenda. The institutional reform stance included a complete reform of MARNDR and associated public services. In this context, ambitious institutional reform plans were drafted and a series of projects - supported by various development partners, such as the Interamerican Development Bank, the International Fund for Agricultural Development, the Food and Agriculture Organization, and the World Bank - were designed to support this global effort to reform essential institutions, including the strengthening of public services related to hydromet, DRM, and agriculture. It is in this context that the new UHM was set up. However, project preparation did not appreciate from the outset the extent of capacity building and time needed to reform government institutions such as MARNDR and the newly created UHM. Moreover, the grant envelope was too small to maintain GoH’s buy-in during implementation, especially since the positive context during preparation was reversed during implementation. Therefore, the PPCR grant could have been considered as an additional financing under the ongoing RESEPAG II or another ongoing project as an alternative. 32. Realism of Objectives. The PDOs were focused but were not realistic, as they were too broad and were ambitious in the context of the grant amount, even though they were in line with Haiti’s SPCR as well as with the PPCR core indicators. 33. Design Challenges. The project benefited from lessons learned worldwide from hydromet projects, and more specifically form Caribbean projects. However, despite collaboration and resource sharing between the World Bank and WMO during project preparation and implementation, the technological capabilities of the consolidated hydromet platform should have been calibrated at the outset within parameters that fit Haiti’s institutional capacity. Conversely, the institutional and inter-institutional capacity building activities proved to be challenging in an FCV context, including exposure to exogenous factors (Section III B) five years after the 2010 earthquake. In addition, the UHM was newly set up and did not get MARNDR’s required political and budgetary commitment throughout the project. As a result, the initial hydromet system that was suggested by the consortium in charge of the project design during the first phase was overambitious. Moreover, the RF lacked coherence, where some IRIs and PDO indicators should have been selected more carefully, while there were no indicators to measure some important achievements, e.g., UHM National Strategic Plan 2020–2024 of Haiti. 34. Selection of Implementing Agencies. Given the GoH’s ongoing reform stance in 2014, MARNDR, which housed the UHM, was the natural recipient of the grant, and its functions were retained in lieu of a full-fledged PIU. However, MARNDR’s capabilities were weak, and they significantly delayed project implementation. For interinstitutional capacity building, the project targeted the newly created inter-ministerial coordination of the hydromet platform: MARNDR, inter- ministerial Committee for Territorial Planning (CIAT), Ministry of Environment (MDE), CNIGS, and end users, that required more strengthening and cohesion. 35. Risk Identification and Mitigation Measures. At appraisal, the project was rightly categorized as “substantial risk”, and the overall risk rating was increased to “high” in 2017 as the government’s internal stakeholders lacked The World Bank HT Strengthening Hydro-Met Services (P148259) Page 16 of 46 commitment to the PDO and were holding up on the UHM consolidation and the process for platform integration. Risk mitigation measures were considered, including re-engaging the dialogue with MARNDR’s new minister and new director general and setting up a new PIU, which partially helped achieve the PDO. 36. Reputational risks. The project incorporated several measures to ensure transparency in implementation but did not require a redress mechanism for potential grievances from stakeholders. Moreover, not installing the remaining stations after project closing could affect the hydromet services in a country prone to extreme meteorological events and climate change effects. 37. Implementation capacity and sustainability. As project activities were considered simple in relation to procurement and FM, the World Bank decided to rely on MARNDR and its UPMP to manage the project, where fiduciary responsibility was under RESEPAG II while the Ministry’s FM unit was being created and made operational. This arrangement proved ineffective, and the World Bank decided to set up a full PIU after the MTR. The PIU required constant World Bank support and capacity building. 38. Procurement and FM. Assessments during project preparation had pointed to substantial procurement and FM risks. At appraisal, the World Bank included two indicators in the Results Framework on procurement and FM reporting to monitor MARNDR functions (Table 5). The initial arrangements were replaced by a new PIU that required constant World Bank support to implement the project. 39. Community participation. The project design incorporated the identification of beneficiaries and the inclusion of women, although the targets were not reached. The targets for training of public servants and NGOs were not reached. Academia and young students were mobilized to carry out data entry at the UHM as well as field work to improve hydromet and climate data collection and aggregation. B. KEY FACTORS DURING IMPLEMENTATION Factors subject to either or both government and implementing entities control 40. Commitment to hydromet process. After a slow start, MARNDR in conjunction with the MPCE, the Italian Cooperation, and two targeted NGOs demonstrated a strong commitment to improve UHM’s role by improving and consolidating the MCH after the MTR. Despite low capacity, MARNDR remained committed to pursuing the PDO by establishing and consolidating the platform to accommodate five subnetworks to increase Haiti’s disaster and climate resilience, although it still needs to introduce the hydromet budgetary line to sustain the platform services in the future. 41. Fund flow, staffing, and procurement. Implementation had a slow start, mainly because of the lack of adequate management, staffing, and delays in procurement of the two main contracts in the initial procurement plan, on account of MARNDR’s managerial weaknesses to implement the project. The initial project coordinator resigned, and the delays in bringing a new project coordinator on board after establishing a PIU also affected project implementation. The project was put on a fast-track disbursement mode after setting up the PIU; however, only 75 percent of the grant was disbursed. Initially, the project was meant to be implemented through two phases, with one large contract in each phase. After an initial mis-procurement (as the selected firm misrepresented its capabilities to perform the first phase activities), the tender was relaunched and the first phase relied on an initial single contract to design, monitor, and develop the project: the consortium design proved technologically ambitious, while the quality of deliverables was below expectation and deliverables were not produced on time. After the MTR, the PIU considered three (instead of one) contracts for the second phase to increase the chances of achieving the targets: (i) hydromet equipment in 62 stations, which were delivered but only two were installed; (ii) flood modeling for three river basins in lieu of the estimated return period of selected weather hazards activity, which was not included in the initial scope of work of the project but was awarded to the same international consulting firm leading the first phase’s consortium, and was canceled as the firm was not able to The World Bank HT Strengthening Hydro-Met Services (P148259) Page 17 of 46 deliver the flood modeling before closing; and (iii) centralized platform that was initially mis-procured, as none of the proposals was acceptable and an alternative plan was adopted to select the WMO MCH that proved effective given the country’s situation. Of the 62 stations, only one company of the 14 (which had downloaded the request for proposal) submitted a proposal for the delivery and installation of the hydromet equipment. The contract was awarded on an exceptional basis, even though the financial proposal of US$2.8 million exceeded the World Bank ceiling of US$1.4 million. After some delays to sort out the newly awarded contract and payment procedures, the equipment delivery and installation as well as the training were supposed to be completed by July 2020, which turned out to be unrealistic because of COVID-19 pandemic related delays. In the end, the training was carried out and the equipment was delivered, but only two stations were installed. 42. Restructuring. An initial restructuring after the MTR, which could have helped revised the scope, results framework, and deliverables of the project, was not considered. The June 2020 restructuring was carried out after the COVID-19 pandemic lockdowns and disruptions had delayed the delivery of the hydromet equipment for 62 stations. However, an extension of only four months was provided based on a corporate decision to extend projects in the Haiti portfolio only by a minimal extent and instead transfer remaining activities to other on-going projects. 43. Collaborative and participatory approach. Implementation of the project required collaborative work within and across sectors to set up the MCH, while integrating the five hydromet networks that were initially used independently of each other. The project sought an effective collaboration with WMO to set up the MCH and to ensure the successful implementation of the project. This required: capacity building at all levels; data acquisition; development of a strategic plan; and reaching out to all stakeholders using or needing hydromet services, notably DDAs and CCPCs under the DPC (that was elevated to a directorate general under the 2020 National DRM system). Such collaboration after the MTR led to a more effective implementation of most activities than would have been the case otherwise. Factors subject to the World Bank Control 44. Adequacy of supervision. Ten implementation support missions provided MARNDR with extensive support from the World Bank, especially after the setting up a new PIU. These missions attempted to put the project deliverables back on track. The World Bank’s supervision missions helped identify and overcome some of the key implementation challenges and bottlenecks. Mitigation measures included the setting up of a PIU after the MTR, the capacity building of the hydromet specialists, and the transfer of the installation of the remaining stations at the end of the project. The last supervision mission was carried out online due to the COVID-19 pandemic. 45. Regular, strategic, outcome-focused support. The World Bank worked with GoH to institutionalize the MCH within the newly created UHM and to provide the necessary visibility to UHM to be recognized at the national hydromet focal point, as well as helped produce the UHM Strategic Plan 2020–2024 to improve and sustain hydromet services in the future. Despite the turnover of Task Team Leaders, of whom one was based in Haiti from early 2016 to end 2019, the World Bank team and consultants provided expert guidance during regular missions and interim technical missions on procurement, financial management, monitoring, and evaluation, as well as the strengthening of hydromet capacity by hiring two international hydromet consultants. The implementation status and results reports (ISRs) candidly detailed the project’s setbacks and challenges and recorded the agreed measures to overcome them. Despite all these efforts, the implementation progress rating remained moderately unsatisfactory during the last three ISRs, as activities were delayed and canceled. Factors outside the control of either or both the government and the implementing agency 46. Between project effectiveness in October 2015 and project closing in October 2020, Haiti was hit by 18 meteorological events - floods, hurricanes including the disruptive Hurricane Matthew, and droughts - political and social unrest, and the COVID-19 pandemic from February 2020 onward. These factors disrupted project implementation The World Bank HT Strengthening Hydro-Met Services (P148259) Page 18 of 46 and the procurement and installation of the hydromet equipment (see Annex 7). Political gridlock and social unrest, interspersed with spikes in violence, lasted throughout the project implementation period. Violence and unrest culminated to new heights during the final two years, especially between October 2019 and January 2020. The depreciation of the Haitian gourde by 117 percent against the US dollar affected the implementation of some national contracts. Six new cabinets during project implementation meant multiple re-appropriations for the project by new ministers at MARNDR, while the lack of MARNDR capacity and UHM appropriation early on also affected project implementation. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 47. The PDO indicators and all IRIs except one were linked to PPCR core indicators (Annex 8). The monitoring arrangements described in the PAD were not always in accordance with generally accepted practices. The project would use various M&E tools to assess the implementation of the project and its impact on stakeholders and beneficiaries, including the establishment and operation of: (i) a baseline survey; (ii) semiannual technical and financial audits; (iii) specific indicators in the results framework to monitor FM and procurement reporting; (iv) an MTR; (v) a final project evaluation; and (vi) regular M&E reports. Except for the end user’s satisfaction baseline that was carried out at project end, most baselines were appropriately set during appraisal and remained unchanged during project implementation, although they should have been adjusted together with the project scope after the MTR through a restructuring. Some indicators and targets turned out to be too ambitious, as both the country’s FCV conditions and the COVID-19 added to the uncertainties under which the project was implemented. One PDO indicator was inadequate to measure the progress of the project towards achieving its objectives: PDO indicator 2 was achieved, however one of the two IRIs associated with this PDO was not, which is counterintuitive. Instead, “number of new stations and existing stations feeding into the centralized platform” should have been the PDO indicator 2. The M&E system did not also capture the formulation of the UHM Strategic Plan 2020–2024, the training of CCPC and DDA staff, measure the institutional progress of the UHM and its financial sustainability. The targets for the training of hydromet specialists and gender training were set too high in the Haiti context and were not achieved. M&E Implementation 48. The M&E system was not implemented until 2019, as activities picked up only then. Even thereafter, the progress reports did not provide updates on all the agreed indicators. M&E data on most activities under Component 1 were collected, measured, recorded, analyzed, verified and stored (including data disaggregated by gender) based on the RF. However, M&E data on the remaining Component 2 activities were not collected, as these activities were not implemented. Regarding the PDO indicator 3, the user’s satisfaction baseline (35 percent satisfaction) sample was much larger than the user satisfaction (52 percent satisfaction) target sample and both the baseline and the target results of PDO indicator 3 are questionable although it had achieved an improvement by a 1.5 factor by the end of the project as reported in the RF PAD. However, a new end target of user’s satisfaction of 100 percent was introduced in the RF in the ISR 4 of May 2017 without any explanation. This target was considered to rate the achievement of the PDO indicator 3. M&E Utilization 49. Reports and information submitted by MARNDR provided key inputs to prepare the MTR and the ISRs, although some delays were reported. Data were collected and analyzed on outcomes and outputs for PDO#1 and PDO#2 The World Bank HT Strengthening Hydro-Met Services (P148259) Page 19 of 46 and their IRIs when the project relied on both MARNDR functions and post-MTR PIU, resulting in a “moderately satisfactory” rating for M&E by project end. Justification of Overall Rating of Quality of M&E 50. Overall M&E quality is rated as modest. The M&E design was inadequate, and its implementation was not timely to assess the achievement of the PDO and measure project implementation progress. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 51. Compliance with overall social and environmental safeguards is rated satisfactory. The project was assessed to be an environmental risk category B investment under the World Bank’s Operational Policy on Environmental Assessment (OP/BP 4.01) and was rated as an extremely low environmental risk. PGRAC will be responsible for ensuring safeguard compliance for the installation of the remaining stations under RESEPAG II, as full coordination between the two projects is required to achieve the deferred targets of the project. 52. The project implemented the environmental safeguard policies effectively. It maintained and replaced two hydromet equipment and provided services and capacity building for these two stations. Awareness-raising and communication activities were carried out at both sites to ensure the engagement and support of the two local communities. During project implementation, compliance with safeguards remained “satisfactory” as: (i) the environmental and social management framework (ESMF) was prepared and updated after the setup of PIU to include social issues such as the right of way, and the safeguard instruments were implemented smoothly; (ii) the safety of workers involved in the repair or removal of equipment was maintained; and (iii) possible exposure to minor electromagnetic fields was avoided. The project identified and screened the locations for most, if not all, of the 50 stations for environmental and social impacts, and the due diligence screening performed by the firm retained to procure and install the stations was satisfactory to the World Bank. This screening will be used to install the remaining stations. 53. Social safeguard policies were also effectively implemented by the project through the two installed stations. The project is composed of a series of TA activities and instrument installation on government-owned sites involving minimal safeguard implications. It did not trigger either OP 4.10 or OP 4.12 and did not affect people. Sites affecting populations that might trigger OP 4.12 were not considered. The project did not implement a grievance redress mechanism (GRM), as a GRM was not required. Site visits allowed the client to monitor measures to prevent or mitigate social impacts. The ISRs rated compliance with social safeguards as “satisfactory” throughout the project. 54. Financial Management. Compliance with the project’s FM was satisfactory throughout project implementation. After the MTR, the project accounting was transferred to the Accounting Package (ACCPAC) system that was finally up and running by the end of the project, although it was supposed be active by the end of 2018. Interim financial reports (IFRs) and financial audits were provided on time and were accepted by the World Bank. The World Bank team provided guidance as required to project FM staff, who were proactive and responsive. 55. Procurement. Compliance with the project’s procurement regulations was satisfactory until 2017 but was downgraded to moderately satisfactory from 2018 to 2019 and was further downgraded to moderately unsatisfactory at closing. The capacity to produce, launch, and evaluate the challenging tenders was the issue that delayed project implementation during the project’s first and second phases. Procurement remained rated as satisfactory until 2017 due to the timely issuance of a few fairly straightforward bid documents, evaluation of bids, and proposals. However, there were significant delays in major procurements for the second phase’s implementation of the UHM development plan. Moreover, the consortium executing the first phase of the project (design, monitor, and develop the project elements) not only delayed the deliverables as set in the timetable but also delayed the issuance of payment receipts. The creation The World Bank HT Strengthening Hydro-Met Services (P148259) Page 20 of 46 of a PIU after the MTR improved the procurement process, including the introduction of the Systematic Tracking of Exchanges in Procurement (STEP, which had been requested by the World Bank since 2016). Contract management was also a challenge and resulted in delays and poor implementation of contracts, while the cancellation of some contracts before closing was warranted due to the accumulated delays. Under Component 3, as the project had initially planned to rely on MARNDR’s own staff for project management, two project management indicators, which usually do not belong in the RF, were added (Table 4) 56. Fiduciary Indicators. The Results Framework included procurement and FM indicators and both indicators met their targets by closing. Table 4: Extent of Achievement of IRI Project Management Reporting Targets Intermediate Results Indicators: Support to project implementation, monitoring and evaluation, and PPCR knowledge management Baseline/ Unit Target Number Achieved % Achieved Timely submission of project procurement reports No Yes Yes 100% Timely submission of financial management reports No Yes Yes 100% Note: Percentage achieved is color coded: yellow – not achieved set target; and light green – achieved set target. C. BANK PERFORMANCE Quality at Entry 57. The World Bank built on the WMO’s stance in preparing the project using a phased approach to ensure effectiveness. The development of project activities was based on the WMO’s Climate Services to Reduce Vulnerability in Haiti that notably led to the creation of UHM under MARNDR, a World Bank assessment (Institutional Expertise of Haiti's Hydromet Service, 2014 carried out by WMO consultants), and the 2012 Haiti SPCR and PPCR practices. The housing of UHM under MARNDR should have been more carefully assessed by the World Bank, and housed, like CIAT, under an overarching institution, such as the premiership, although CIAT is still facing weaknesses. However, activities were selected and implemented after close consultations with GoH and with the stakeholders who would benefit from hydromet services: (i) MARNDR housing UHM, DDAs and National Coordination of Food Security; (ii) CIAT under the tutelage of the premiership; (iii) MPCE housing CNIGS; (iv) MDE housing the climate change division; (v) Ministry of Interior and Territorial Communities (MICT) housing the DPC and the Department of Territorial Communities; and (vi) Ministry of Public Works, Transport and Communications housing the Haiti Maritime and Navigation Service, the Urban Planning Service, and the National Civil Aviation Office that eventually transferred some of its staff to work within UHM premises after 2017. Project activities provided tailormade TA to pursue hydromet system strengthening and to build capacity within: (i) UHM and across institutions, such as the CNIGS and the DPC, for communications and early warning systems; and (ii) the CNIGS for Earth observation data and agencies within ministries, such as the DDAs. Although drawing from the World Bank’s global experience across several hydromet projects, the project team did not fully benefit from the lessons from investments in Haiti, e.g., the 2009–2015 Emergency Bridge Reconstruction and Vulnerability Reduction Project (PROReV) that oversaw the production by CIAT of the National Multihazard Atlas, as well as the 2012–2020 “Disaster Risk Management and Reconstruction Project” (PRGRD) that installed early warning communication systems and capitalized on the ongoing dialogue with the GoH regarding disaster risk management and climate change. 58. The project basics were not sound and the project components not well targeted; the delivery mechanisms were assumed to be effective, in particular the fiduciary aspects, despite the FCV context. The World Bank should have paid more attention during project preparation to meet its strategic, technical, economic, financial, safeguard, fiduciary, and institutional requirements. The project was overambitious, and the targets should have been selected more realistically. A better institutional assessment was needed on the adequacy of anchoring hydromet services The World Bank HT Strengthening Hydro-Met Services (P148259) Page 21 of 46 within MARNDR. Some of the fiduciary and institutional aspects proved to have been inadequately assessed, as observed during project implementation. The risk assessment was comprehensive, and appropriate risk management measures were included in the project design and in the implementation support plan. However, the RF did not capture the breadth of the project, as the six IRIs did not accurately measure the activities that would contribute to the achievement of the three PDO indicators. Moreover, the latter did not consider UHM’s sustainability as a corner stone of the country’s entire hydromet and early warning systems, and as one of the ISN’s priorities in reducing vulnerability and increasing resilience. Triggered by the COVID-19 pandemic, the restructuring came too late, but was nevertheless essential at the end of the project to extend the project closing by four months to ensure that the hydromet equipment was delivered by the closing date. Quality of Supervision 59. The World Bank provided timely and adequate support and guidance to the project through biannual supervision missions and frequent informal contacts and technical discussions. Nevertheless, implementation of the project stalled until MTR’s mitigation measures to set up a PIU and reconsider the deliverables and the timetable. Constant dialogue with multi-stakeholders and development partners were integral to the supervision missions. Yet, it took more than two years for the World Bank to react effectively to the reality that the project was not moving. Hence, although the achievement of the PDO was the team’s high priority, and the Bank’s efforts and constant technical support to the PIU helped bring the project back on track, the pace of implementation was too slow to achieve the targets set. The World Bank team reported on project status candidly in periodic aide-mémoires and ISRs and highlighted the shortcomings, challenges, and actions required to get the project back on track after the MTR, including a project extension to realize planned outputs and outcomes. Such actions included direct and consultant support through training and assistance, especially for procurement after the setting up of the PIU. The World Bank team maintained an exceptionally good working relationship with WMO during the MCH set up, which helped put the project back on track. Justification of Overall Rating of Bank Performance Rating: Moderately Unsatisfactory 60. Based on the above discussion, the World Bank’s performance is rated “Moderately Unsatisfactory”. D. RISK TO DEVELOPMENT OUTCOME 61. The 2012 SPCR allowed the setting up of interministerial coordination of the hydromet platform, create the platform under one entity (UHM), and consolidate the platform (MCH) to increase disaster and climate resilience across sectors and themes, for example, energy, transport, environment, agriculture, gender, and poverty reduction. The UHM Strategic Plan 2020–2024 prepared under the project sets clear priorities to improve and maintain the platform. The project faces moderate risk to sustainability in the short term, as the installation of the stations will be carried out under RESEPAG II with TA from the PGRAC and will benefit from parallel ongoing projects implemented by development partners (such as the ongoing multi-donor CREWS project and WMO capacity strengthening). However, the moderate risk to sustainability could increase in the medium term, should the UHM’s institutional and financial sustainability not be realized as articulated in the UHM Strategic Plan 2020-2024. When the hydromet platform is fully operational and UHM is provided a dedicated budget and charges a fee for services, it will have the capacity to perform simulations and projections that will inform the formulation of strategies and policies, improve decision making as well as the design of programs and projects to manage future climate and disaster risks better. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 22 of 46 62. Political instability, the difficult economic outlook, the COVID-19 pandemic, and civil unrest continue to remain an important characteristic of Haiti’s landscape. While the UHM has been created and a strategy has been formulated, the open access and increased number of end users should be accompanied by political commitment, dissemination to NGOs, and buy-in from beneficiaries to maintain the momentum of mainstreaming, consolidating, and disseminating hydromet services. V. LESSONS AND RECOMMENDATIONS 63. The lessons learned are articulated along six themes in an FCV country: new project versus additional financing for an ongoing project, design, management, procedures, contractual arrangements, and institutional and financial sustainability of hydromet. 64. Given the small size of the grant, there is a need to better gauge the pros and cons of preparing a standalone project against adding the grant to an ongoing project, as small projects do not attract political commitment and support from governments. For the various reasons discussed earlier, the project was not effective as the hydromet stations were not installed, the UHM institutional strengthening under MARNDR was incomplete, and the sustainability of the UHM was not considered. Low institutional capacity was also one of the reasons for the project to have several setbacks, despite the initial positive context during project preparation. Development agencies, including the World Bank, need to carefully gauge the trade-offs between having a standalone project versus attaching relatively small grants to ongoing projects with already established and functioning implementation arrangements. This would also optimize administration costs, as the running costs of a PIU can be high, especially in FCV countries. Adding the grant to the ongoing RESEPAG II under MARNDR should have been judiciously considered by the Bank. 65. There is a need to consider very simple project designs and strike the right balance between the technology considered and the technical capacity of the various institutional stakeholders, where service provision should be decoupled from system expansion. Capitalizing on and strengthening the existing system should be the priority of decision makers before embarking on a system expansion. Albeit unfinished, the strengthening of services brought the UHM to the fore, especially during Hurricane Laura in 2020, where UHM’s critical information was used before, during, and after the event. The project design favored not only a system expansion, but also a hi-tech platform where technology should be relative to its absorptive capacity. The selection of a hi-tech platform resulted in limited supplier participation in procurement processes, as only a limited number of international firms had the technical and financial capacity to execute the activities while small local firms (that had the knowledge to customize hydromet systems to the needs and capacity of the country) were sidelined. It is imperative to consider existing services, the reality of the market and the institutional capacity of the recipient when designing activities, so that they are properly calibrated to an FCV country to allow for the gradual strengthening of the system first and then moving to system expansion by relying on a wider range of firms (including local firms) to respond to procurement processes. The very sophisticated and costly hydromet system recommended by the international consortium proved to be unrealistic and ineffective and was therefore replaced by a more user-friendly and robust platform. 66. In FCV contexts, implementation arrangements relying on the recipient’s own structures should not be considered, while the political and socioeconomic context should be closely monitored to trigger restructuring in time for course corrections. In an FCV country with low capacity, relying on the recipient’s own newly created structures proved to be ineffective and costly. The World Bank had to set up a PIU after the MTR in December 2018 as project implementation and disbursement faced substantial delays. Moreover, close monitoring and strong World Bank technical support to the PIU were needed to compensate for the low technical and project management capacity, including contract management capacity. Accurately assessing recipient capabilities and financing of the essential human resources for key project management functions including procurement (for contract size, scope, and The World Bank HT Strengthening Hydro-Met Services (P148259) Page 23 of 46 number) as well as FM is essential to avoid having to rely on other World Bank project PIUs or structures outside the PIU. 67. Splitting the project into two large contracts - one for the diagnosis, design, and supervision, and the other for implementation - awarded to two different firms could present various implementation challenges in the context of the recipient’s limited institutional and technical capacities. The split between the soft and the hard contracts caused multiple slowdowns by increasing the number of stakeholders and diluting responsibilities. As a result, the coordination and the establishment of effective and transparent communication between the actors - firms, PIU, and UHM - proved to be challenging. 68. As already evidenced in other recent projects in Haiti (PROReV and PRGRD), relying on contractual arrangements with clearly defined scope, timelines, and judicious technical choices proved to be a good practice. The close coordination - with actions supported by WMO from the project design stage and throughout project implementation, and during the installation of the WMO-based MCH - was effective and allowed synergies, complementarities, and optimization of resources leading to positive results. The approach adopted by the World Bank and WMO after the MTR was adopted for the development of the multi-donor CREWS in the CARICOM member countries. 69. The institutional and financial sustainability of a hydromet entity such as UHM in a country prone to extreme meteorological events and increased vulnerability caused by climate change requires the utmost attention of development partners and long-term commitment, as such development outcomes take time to be achieved. The institutional anchoring of the hydromet system should not be taken for granted and requires careful consideration as the setting up of the system at the UHM under MARNDR lacked the necessary GoH buy-in while the UHM requires long-term strengthening and support. Conversely, the financial sustainability of a hydromet system should be carefully considered by supplementing budgetary appropriation with grants from development partners, as well as the introduction of fee-based hydromet services. The hydromet system is the end goal of a value chain of information that would provide the country’s stakeholders with critical information for strategies, policies, preparedness, and response. Setting up an efficient hydromet system is time consuming, but it is an essential building block of a disaster risk management strategy. For instance, the feeding of information into peripheral early warning systems will be provided from the hydromet system. Conversely, a poorly performing system will hamper the effective performance of these early warning systems that are imperative to reducing vulnerability and increasing resilience in an FCV country. . The World Bank HT Strengthening Hydro-Met Services (P148259) Page 24 of 46 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Hydro-meteorological and climate information services are provided to increase disaster resilience Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Data collected from hydro- met networks are accessible on a centralized online hydro- meteorological data management platform, with i ts own operating procedures for validation and storage Yes/No N Y N 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): Data collected from hydromet networks are centralized in an hydromet data management platform, accessible to end users upon request and not online. Therefore, the PDO was not achieved. WMO technical assistance worked with UHM on setting up the online portal for direct data sharing and WMO is continuing the process under their own ongoing project. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 25 of 46 Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Number of sub-networks feeding into the centralized platform Number 0.00 5.00 5.00 06-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): The sub-networks feeding into the UHM centralized platform were set up by the end of June 2020. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion End users' satisfaction rate towards improved hydromet information services Percentage 35.00 53.00 52.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 End users' satisfaction rate towards improved hydromet information services (gender disaggregated) Percentage 21.00 42.00 21.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): In the PAD the end target for this indicator is “baseline * 1.5”, leading to end target of 53%. However, in ISRs starting on May 2017, the end target was set to 100%, but was not formally revised. This value that was considered for the evaluation. The achieved target was only 52 percent of the end- users satisfaction with the MCH as the online portal was still not fully operational by project end. The gender target was doubled to reach 42% against 50% targeted. To derive the values of this indicator, a second satisfaction survey was conducted with a sample of 83 entities: Community organization such as Citoyenneté et Inclusion (Citizenship and Inclusion); Service Maritime et de Navigation d'Haïti (Haiti Maritime and Navigation Service) ; Bureau Agricole The World Bank HT Strengthening Hydro-Met Services (P148259) Page 26 of 46 Communale (Communal Agricultural Office); CCPCs; Coordination Départementale de Gestion Des Risques de Désastres (Departmental Coordination of Disaster Risk Management); and DDAs. However, this represents half of the sample that was considered in the first survey and includes, apart from NGOs and some self-governing institutions, the same categories of users. The 52 percent of the end-users satisfaction is not as representative as the first survey that was carried out to draw the baseline of 35 percent. A.2 Intermediate Results Indicators Component: Identification of hydro-meteorological and climate services’ requirements for select end users and development of information services to support decision making Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Number of users and instructors trained on the use of the centralized data platform Number 0.00 80.00 9.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 Number of users and instructors trained on the use of the centralized data platform (% of women) Number 0.00 20.00 3.75 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): Nine of 80 targeted staff members were fully trained to develop data management capacity of the producers and users of hydromet data. Four female staff members were trained, which is also below target. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 27 of 46 Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Hydrological and meteorological stations reporting data to the platform in line with agreed SOPs Number 0.00 34.00 34.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): The national Hydromet platform that was installed and relies on the MCH open-source software developed by WMO and is hosted in a permanent server. However, the platform hosts data from 34 stations from the five networks is not being constantly updated to remain in sync with the software upgrades carried out by the stations from the 5 networks. Since February 2020, the World Bank and WMO have installed the MCH. Four manuals were prepared for the standard operating procedures and used for online training to: (i) provide the procedure by which MCH users can enter the information from observations to the system through the MCH client; (ii) provide the procedure by which MCH users can enter the information from observations to the system through the MCH website; (iii) provide a procedure for loading data from Davis automatic weather stations of the ProVantage and ProVantage2 models; and (iv) help users develop Excel sheets that can extract data directly from the MCH database. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Operational stations (physical structure) Number 0.00 50.00 2.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): A contract for the supply and delivery of key Hydro-met equipment was signed in February 2020 between MARNDR and the firm BFOR and the delivery and installation of 62 stations were planned to be completed before the end of June 30, 2020, the grant’s closing date. However, due to travel restrictions The World Bank HT Strengthening Hydro-Met Services (P148259) Page 28 of 46 linked to COVID-19 (the suppliers are based in Germany and Spain, which were very affected by the pandemic between March and June 2020) the delivery was postponed to the end of July and the installation was planned be completed before the end of October 2020. However, only two stations have been installed by the closing date and were not connected to the MCH. Component: Support to project implementation, monitoring and evaluation, and PPCR knowledge management Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Timely submission of project procurement reports Yes/No Y Y Y 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): Procurement reporting was regularly submitted before and after the setting up of the PIU after the MTR. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Timely submission of financial management reports Yes/No Y Y Y 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): Financial management reporting was regularly submitted before and after the setting up of the PIU after the MTR. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 29 of 46 Component: Institutional strengthening of hydro-meteorological services and development of data management tools Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Number of Civil Protection Committees using the customized application from the centralized data platform Number 0.00 40.00 0.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): CCPCs were trained on initial products but the customized application was not finalized. Therefore, this indicator did not achieve its set targets. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Number of Departmental Agricultural Directions (DDA) using the customized application from the centralized data platform Number 0.00 10.00 0.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 Comments (achievements against targets): DDAs were trained on initial products but the customized application was not finalized. Therefore, this indicator did not achieve its set targets. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 30 of 46 Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Direct project beneficiaries Number 0.00 1,000,000.00 0.00 30-Jun-2015 30-Jun-2015 29-Oct-2020 Female beneficiaries Percentage 0.00 0.00 0.00 Comments (achievements against targets): As the MCH is not fully operational as targeted by the project objective, therefore the beneficiaries, of whom women, will not benefit from the project. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 31 of 46 B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Strengthening Haiti’s institutional capacity to provide hydromet and climate information services Component 1: Outcome Indicators 1.1 Data collected from hydromet networks are accessible on a centralized online hydro-meteorological data management platform, with its own operating procedures for validation and storage - Partially achieved. 1.2 Number of subnetworks feeding into the centralized platform - Achieved. Intermediate Results Indicators 1.2.1 Number of users and instructors trained on the use of the centralized data platform of whom women – Not achieved. 1.2.2 Hydrological and meteorological stations reporting data to the platform in line with agreed standard operation procedures – Achieved. 1.2.3 Operational stations – Not achieved. Objective/Outcome 2: Strengthening Haiti’s institutional capacity to customize hydromet and climate information to the needs of the civil protection and agriculture sectors Component 2: Outcome Indicators 2.1 End users' satisfaction rate toward improved hydro-met information services, of whom women – Not achieved. Intermediate Results Indicators 2.1.1 Number of Civil Protection Committees (CCPCs) using the customized application from the centralized data platform – Not achieved. 2.1.2 Number of Departmental Agricultural Directions (DDAs) using the customized application from the centralized data platform – Not achieved. 2.1.3 Direct project beneficiaries of whom women – Not achieved. Component 3: Project Management: Procurement and Fiduciary Reporting Key Outputs 3.1 Timely submission of project procurement reports – Achieved. 3.2 Timely submission of financial management reports – Achieved. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 32 of 46 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Gaetano Vivo Task Team Leader(s) Prosper Nindorera Procurement Specialist(s) Fabienne Mroczka Financial Management Specialist Claudia Ruth Soto Orozco Team Member Rose Caline Desruisseaux-Cadet Team Member Josue Akre Team Member Jean Baptiste Migraine Team Member Ramiro Ignacio Jauregui-Zabalaga Counsel Peter F. B. A. Lafere Social Specialist Michel Matera Team Member Nancy Chaarani Meza Team Member Nyaneba E. Nkrumah Social Specialist Carolina J. Cuba Hammond Team Member Victor Manuel Ordonez Conde Team Member Aboubacar Magassouba Team Member Supervision/ICR Claudia Ruth Soto Orozco Task Team Leader(s) Mamata Tiendrebeogo, Aboubacar Magassouba Procurement Specialist(s) Lucas Carrer Financial Management Specialist The World Bank HT Strengthening Hydro-Met Services (P148259) Page 33 of 46 Alfred Jean-Marie Borgonovo Financial Management Specialist Eduardo Porras Calderon Team Member Noris Viviana Sandoval Sierra Procurement Team Maria Laettitia Antoine Procurement Team Marie Eleonor Jean Baptiste Procurement Team Jean Noel Degrace Team Member Marcela Nandllely Gonzalez Team Member Naraya Carrasco Team Member Djeanane Monfort Procurement Team Kevin McCall Environmental Specialist Khaled Mohamed Ben Brahim Team Member Ramiro Ignacio Jauregui-Zabalaga Counsel Ingrid Sandra Milord Procurement Team Hadji Huseynov Team Member Peter F. B. A. Lafere Social Specialist Felipe Jacome Social Specialist Pepita Hortense C. Olympio Procurement Team Asli Gurkan Social Specialist Andrianirina Michel Eric Ranjeva Team Member Ali Alwahti Procurement Team Carl Christian Gyde Jacobsen Team Member Isabella Micali Drossos Team Member Brad Kelly Lassiter Procurement Team Manju Ghumman Team Member Carolina J. Cuba Hammond Team Member Serge A. Pieyns Team Member The World Bank HT Strengthening Hydro-Met Services (P148259) Page 34 of 46 B. STAFF TIME AND COST Stage of Project Cycle Staff Time and Cost No. of staff weeks US$ (including travel and consultant costs) Preparation FY14 8.775 94,977.95 FY15 18.791 118,721.71 FY16 16.992 77,499.30 Total 44.56 291,198.96 Supervision/ICR FY16 1.900 13,978.00 FY17 19.218 158,421.23 FY18 12.975 111,824.04 FY19 1.032 75,989.59 FY20 9.574 88,226.63 Total 44.70 448,439.49 The World Bank HT Strengthening Hydro-Met Services (P148259) Page 35 of 46 ANNEX 3. PROJECT COST BY COMPONENT Components Amount at Approval Amounts reallocated between disbursement categories during FY20 Actual at Project Closing (US$M) Percentage of Approval (US$M) (US$M) (US$M) (%) Institutional strengthening of hydro- meteorological services and development of data management tools 3.00 1.75 0.54 18% Identification of hydro-meteorological and climate services’ requirements for select end users and development of information services to support decision making 1.40 2.36 1.90 136% Support to project implementation, monitoring and evaluation, and PPCR knowledge management 0.60 0.90 0.80 133% Cash in Government’s account at the end of the project 0.53 Total 5.00 5.00 3.77 75% Note: Grant amount of US$ 5 million was not allocated by component as reported in the Grant Agreement where eligible costs included: Goods, non-consulting services, consultants' services, Training and Operating Costs. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 36 of 46 ANNEX 4. EFFICIENCY ANALYSIS Ex-Ante Economic Analysis 1. Since 2010, Haiti’s weather forecasts and severe weather warnings are based on models and analysis run abroad (mainly by US NOAA and Méteo France). Moreover, there were no seasonal forecasts or timely planting and harvesting advisories adapted to the local conditions/data that could enhance the productivity of farming, particularly as climate change undermines the predictive value of historical climate knowledge and associated traditional practices. This left an opening for producing more adapted forecasts and warnings, thus diminishing the loss of lives and assets while increasing agricultural production in this highly climate-vulnerable country. 2. An ex-ante economic analysis of the project investments confirmed a positive return on investment, with a very wide benefit/cost ratio between 1.6 and 33 over a 15-year timeframe. These results were based on international benchmarks 15 reported in percentage of GDP. In the case of Haiti, this was supposed to translate into gains of approximately 0.03% and 0.3% of GDP. Ex-Post Economic Analysis 3. As the Hydro-met project missed its objective of setting up and operationalizing the MCH, the benefits of the project will accrue in the future under RESEPAG II. By the end of the project, the MCH integrated 34 stations from five subnetworks and improved the efficiency of hydromet services. However, while end-user access will improve, CCPC and DDA staff, although trained, are not yet using the data to perform the needed forecasts for meteorological events and for meteorological-based agricultural management. Table 4.1 summarizes the results of the ex-post economic analysis. Table 4.1: Ex-Post Benefit-Cost Analysis of the Hydro-met Project Indicators Discount Rates over 20 years Criteria 4% 6% 8% NPV (US$ Million) Negative Negative Negative >0 ERR (±%) Negative Negative Negative ≥ discount rate considered PVBCR <0 <0 <0 >1 15 A Cost-Effective Solution to Reduce Disaster Losses in Developing Countries: Hydro-Meteorological Services, Early Warning, and Evacuation (World Bank Policy Research, Working Paper #6058, 2012) The World Bank HT Strengthening Hydro-Met Services (P148259) Page 37 of 46 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS MINISTRY OF AGRICULTURE, NATURAL RESOURCES AND RURAL DEVELOPMENT (MARNDR) HYDROMETEOROLOGICAL UNIT (UHM) COMMENTS ON THE FINAL EVALUATION OF THE HT Strengthening Hydro-Met Services Project (HYDROMET) TF-0A0226 GRANT AGREEMENT FUNDING: Strategic Climate Fund-Pilot Program for Climate Resilience Prepared by: Coordination of the UHM April 2021 The World Bank HT Strengthening Hydro-Met Services (P148259) Page 38 of 46 Background and Institutional Anchor The Hydromet project, although with limited resources, has been of great strategic importance to the Haitian state as it aimed of improving access to good quality data and information for rational multi-sector decision- making. It is in this sense that the Haitian State, through the MARNDR, was committed to this project from the onset using its various operational bodies and which continue to be strengthened, i.e., the administrative and technical structures. Administrative Structure From an administrative point of view, the HYDROMET project benefited from the experience of the RESEPAG II project financial management and the MARNDR Public Procurement Unit (UPMP). The latter are entities with full decision-making autonomy in strict compliance with Haitian laws’ procedures and World Bank directives. Apart from the long delay in starting the project, these two entities were able to fulfill their mission with professionalism despite a difficult socio-political context since 2018. However, it is important to draw attention to UPMP staff that operated with a reduced number of UPMP executives. All staff, consultant or service contracts go through the UPMP for all projects’ foreign funds provided to MARNDR. The available staff are constantly overwhelmed by a flood of files. So inevitably, there are inevitable delays in processing of these files. The Hydromet project suffered like any other project. It would have been wise for the projects to support the recruitment of additional UPMP staff and the strengthening of capacity of the new recruits through ongoing training. Technical Structure The Hydrometeorological Unit (UHM) was the technical anchor of the project and was to be the main beneficiary of the strengthening planned by HYDROMET. However, at the time of launching the Hydromet project in 2015, the UHM was not operational as CNM and the SNRE with each a department head were not yet merged under UHM. It was not until the May 2017 inauguration of UHM new premises and subsequent move of the staff, furniture and equipment in December 2018 and the nomination of a new coordinator (UHM) on January 30, 2018 that UHM became operational. The absence of an active UHM Coordinator during the first 3 years of project implementation and a lack of ownership did not help the project to start under the best technical conditions. The difference was noticeable from 2018 onward with greater involvement of the UHM through its new coordinator. The Project Coordination Unit The project was administered by a coordination unit comprising a coordinator, a mid-level executive as well as a monitoring and evaluation (M&E) specialist. The first major challenge of the project was the resignation of its first coordinator in the 19 th month of implementation. It took nine months to formally replace him. The The World Bank HT Strengthening Hydro-Met Services (P148259) Page 39 of 46 recruitment of the M&E specialist was delayed and occurred in mid-2018. Difficulties External Difficulties Haiti's socio-political environment from 2018 until project closing in October 2021 did not really play in the project's favor. The year 2019 that should have been the year of the implementation of the second phase of the project and was marked by a total shutdown over several months. This situation has disrupted the tendering schedule and led to the forced cancellation of most of the tenders when the situation worsened in early 2020 globally due to the COVID-19 pandemic. The latter has imposed a near-global blockade that prevented the execution of the major contract consisting of the acquisition and installation of hydrometeorological stations and forced the cancellation of the modelling tender as well as the development of the platform tailored to the users’ needs. To compensate for this, the World Bank has granted an extension of four months to compensate for the delays, but the extension was largely insufficient to complete the latest activities of the project, in particular those of the second phase. Recommendations of the November 2018 Mid-Term Review Report During the mid-term review, the coordination of the project drew the attention of the World Bank to the fact that the initial objectives do not have a high probability of being achieved without an extension of at least eighteen months, thus until 31 December 2021. The report even proposed scenarios to address these issues, i.e., for the activities that would not be completed by 30 June 2020. Moreover, at that time the coordination could not even foresee the 2019 shutdown and the 2020 pandemic. The finding of the project initial delays should have encouraged the World Bank to accept an obvious and longer extension of the project. We believe that this unique decision would have made a very big difference in the outcome as well as a better evolution of the indicators. We continue to regret this decision, which took little account of the difficult history of the project, especially at the start-up. Gains and Perspectives Despite the many difficulties that have compromised some of the important results expected of the project, it is important to note that the project leaves several achievements that will need to be consolidated. The project has contributed to confer a certain national notoriety to the UHM in the sector. The project enabled the furnishing of the workspace and the acquisition of specialized office computer equipment for the MCH platform acquired under the project. The UHM now has a platform to collect, process and interpret data produced in the field in real time. Through this project, UHM executives have benefited from several online and in-person training courses on various branches related to the field of hydrometeorology. The UHM, in turn, organizes training The World Bank HT Strengthening Hydro-Met Services (P148259) Page 40 of 46 sessions for key users (DDA, BAC, CDPC, CCPC) on the efficient use of its various products. These trainings, coupled with open houses for children and adults, have finally ensured that the UHM has a high profile in the country. One of the significant achievements of the Hydromet project is the development of key documents as part of the first major design contract. These are the first documents that certify UHM as a full-fledged entity with clear responsibilities and competences. These documents complete the circular creating UHM. These include the diagnosis of the Haitian hydrometeorological sector and inter-institutional coordination, followed and especially the development plan of the UHM with a business plan and an evolutionary budget preparation mechanism over the next ten years. This document served as the basis for the development of the National Strategic Plan (2020- 2024). Add to this the design of the UHM's optimum network, which takes into account its capabilities as well as its technical and budgetary limitations. Another important achievement is the acquisition of Hydromet stations, which will be distributed across various priority regions of the country. All the equipment was acquired and stored at the UHM premises. As of October 30, 2020, two stations have been installed in Léogâne (Momance River) and Damien. Constant follow-up is provided with the selected company to continue the installation of stations during the year 2021. The UHM hopes to have nearly 40 new stations installed and operational by the end of August 2021. All these achievements have enabled UHM to position itself as the true leader in the field of hydrometeorology in Haiti. Thus, the project has provided the UHM with tools to move towards its autonomy. General Considerations and Recommendations The evaluation strategy of the project affected the evolution of the indicators after the execution of the first major design contract and from the execution of the second major contract. Any delay or inconvenience in the original contract would logically have had a negative impact on the start of Phase 2. Thus, this first tender, which was initially declared unsuccessful, greatly delayed the key activities of the second phase to which all indicators are linked. The evaluation strategy of the project did not take into account at the outset the significant achievements of the first major contract through mainly the development plan of the UHM. The latter could have been one of the indicators of the project. Furthermore, we believe that on the basis of the mid-term review carried out by the project team in November 2018, the stakeholders could logically have restructured the project or extended the project substantially: • All project partners believe that the project has closed almost at the time of its actual start, so they would like to see a HYDROMET 2 project to consolidate the achievements of the HYDROMET 1 project. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 41 of 46 The UHM still needs sustained support to be able to fulfill its mission. • It is recommended that the implementation of the different phases of the projects be made more flexible. A long delay in Phase 1 can jeopardize its related activities in Phase 2 and so on. • It is also desirable for projects to support long-term higher education to better support institutional strengthening, as well-trained staff offer better guarantees of consolidation of investment gains. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 42 of 46 ANNEX 6. SUPPORTING DOCUMENTS HT Strengthening Hydro-Met Services PAD 2015 Grant Agreement 2015 Restructuring Paper 2020 ISRs (10) Mid-Term Review 2018 Aide-Mémoires of Implementation Support and Technical Missions Government ICR (in French) The World Bank HT Strengthening Hydro-Met Services (P148259) Page 43 of 46 ANNEX 7. PROJECT IMPLEMENTATION TIMELINE: EXOGENOUS FACTORS The World Bank HT Strengthening Hydro-Met Services (P148259) Page 44 of 46 ANNEX 8. PROJECT RESULTS FRAMEWORK AND PPCR CORE INDICATORS Table A8.1: Link Between Hydromet PDO Indicators and Pilot Program for Climate Resilience Core Indicators Hydromet Indicators PPCR Core Indicators PPCR Core Indicators #2 (“Evidence of strengthened government capacity and coordination mechanism to mainstream climate resilience”) PPCR Core Indicators #3 (“Quality and extent to which climate responsive instruments/investment models are developed and tested”) Core Indicators #4 (“Extent to which vulnerable households, communities, businesses and public sector services use improved PPCR supported tools, instruments, strategies and activities to respond to climate variability or climate change”) PDO indicators Data collected from hydromet networks are accessible on a centralized online hydrometeorological data management platform, with its own operating procedures for validation and storage Number of sub-networks feeding into the centralized platform End users' satisfaction rate towards improved hydromet information services of whom percentage women IRIs Number of users and instructors trained on the use of the centralized data platform of whom % women Hydrological and meteorological stations reporting data to the platform in line with agreed SOPs Number of Civil Protection Committees using the customized application from the centralized data platform Operational stations Number of Departmental Agricultural Directions (DDA) using the customized application from the centralized data platform Direct project beneficiaries of whom women No Link No Link No Link The World Bank HT Strengthening Hydro-Met Services (P148259) Page 45 of 46 ANNEX 9. PROJECT COMPONENT ACHIEVEMENTS AND LINK TO CREWS Component 1: Institutional strengthening of hydro-meteorological services and development of data management tools. National Hydromet platform: The MARNDR opted to install a simpler software managed by WMO called Meteorological, Climatological and Hydrological Database Management System (MCH) as the national hydromet platform (NHP). MCH is a database management system based on open-source database and software, supported by WMO programs and strategy especially for least developed countries which can guarantee a certain level of sustainability and low-cost updates. MCH allows to connect automatic stations and gather real time data offering solutions to digitize, store, process large amount of data and generate reports and products on large amount of meteorological, climatological and hydrological data. The national platform is hosted in a permanent server at the UHM. Standard operating procedures (SOP) have been developed for operating and maintaining all hydrometeorological data and data collection processes. The national hydromet platform hosts data from 34 stations from five subnetworks, including the UMH’s network, the National Center for Geo-Spatial Information (CNIGS) of the Ministry of Planning, Cooperazione Internazionale (COOPI), and NGO partners’ networks (e.g., Fondation Nouvelle Grand-Anse and CESAL). The national platform also hosts historical hydrometeorological data (from 1926) from 230 stations of the MARNDR Water Resources Services’ (SNRE) hydrological network. The national platform will gradually include other networks managed by different government entities, including the MARNDR, through the Departmental Agricultural Directions (DDAs). As well, the new equipment acquired by the Hydromet Project will be connected to the national platform once the new automatic hydromet stations will be installed. Training on MCH platform has been specifically designed for Haiti based on human competences and observing networks capacities. Hence, 11 people (IT staff as administrator, climatologists, hydrologists and UHM coordinator) were trained on the management of the MCH system, capture, consultation, management and process of meteorological and climatological data and production of specific information for the end-users. Strengthening of the hydromet network: Based on the results of the Institutional Diagnostic of the Hydromet Sector, the needs for optimization of hydromet equipment and strengthening of data management tools were identified. Hydrological monitoring equipment and stations were procured to the UHM. Equipment will be installed by the end of 2021 with the support of two ongoing Bank-financed projects (RESEPAG II and PGRAC). Component 2: Identification of hydro-meteorological and climate services’ requirements for select end users and development of information services to support decision making. The Institutional Diagnostic of the Hydromet Sector conducted in the first part of the project allowed to assess and analyze strengths, weaknesses, opportunities, and constraints of the UHM to meet the requirements of hydromet services of the end users. The UHM identified strategies to strengthen the institution and fulfill its mandate and meet the requirements of hydromet information of end-users. This diagnostic also contributed to the development of the UHM National Strategic Plan 2020-2024 achieved under Component 1. The World Bank HT Strengthening Hydro-Met Services (P148259) Page 46 of 46 Additionally, under this component the project planned to analyze the data collected by the hydromet networks consolidated in the national data platform, in order to develop software interfaces customized to meet the needs of the Directorate of Civil Protection and the agriculture sector. The project did not fully complete its objectives but was able to advance in the following areas: scoping the hydromet information requirements of end-users and development of initial customized hydromet products to be used by the agriculture sector. Also, a designated training session was carried out for 85 end users (including 13 women) from the Civil Protection (CCPCs) and Agricultural Departmental Offices (DDAs) in the interpretation and use of the UHM’s hydrometeorological products (bulletins). With the support of the Hydromet project the quality of the existing UHM hydromet products has been improved. The project has provided equipment such as servers to save and better process the data and carried out the maintenance of hydromet stations to ensure the quality and regularity of the information. How CREWS project will capitalize on the hydromet achievements One of the main activities of the CREWS project is to develop a comprehensive multi-hazards and multi-scales early warning services (EWS) focused on hydromet risks, meeting the needs of Civil Protection and food security (and other stakeholders) with regards to the four pillars defined for an efficient EWS: (i) risk assessment and monitoring; (ii) forecasting and warning services; (iii) dissemination and communication; and (iv) planning and response capacity. The hydromet and climate observation networks and the MCH database implemented by the Hydromet project, accompanied by ad-hoc training, will give the country (through UHM and partners) the basic capacities to contribute to at least three of the four pillars of CREWs. Indeed, it will help to determine and fine tune risk information, it will also strengthen capacities for real time monitoring and forecasting and will concur to develop EWS planning and SOPs while, for example, defining thresholds for warning and associated procedures. Thus, the outputs of the Hydromet project will facilitate the operationalization of CREWS activities and will contribute to achieve the expected results.