(2021) Promouvoir l'inclusion des personnes handicapées dans les programmes de protection sociale et d'emploi

Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005641 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF 18710 ON A SMALL GRANT IN THE AMOUNT OF USD 2,294,320 MILLION FROM THE POLICY AND HUMAN RESOURCES DEVELOPMENT (PHRD) FUND TO THE REPUBLIC OF HAITI FOR PROMOTING INCLUSION OF PERSONS WITH DISABILITIESS IN SOCIAL PROTECTION AND EMPLOYMENT PROGRAMS (P145468) December 17, 2021 Social Protection & Jobs Global Practice Latin America And Caribbean Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Regional Vice President: Carlos Felipe Jaramillo Country Director: Lilia Burunciuc Regional Director: Luis Benveniste Practice Manager: Pablo Gottret Task Team Leader(s): Caroline Tassot, Briana N. Wilson ICR Main Contributor: Emma Schwirck Willenborg CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2021) Currency Unit = Haitian Gourde (HTG) US$ 1 = HTG 93,50 ABBREVIATIONS AND ACRONYMS BSEIPH Office of the Secretary of State for the Integration of persons with disabilities (Bureau du Secrétaire d’État à l’Intégration des Personnes Handicapées) CPF Country Partnership Framework FAES Economic and Social Assistance Fund (Fonds d’Assistance Économique et Sociale) GDP Gross Domestic Product GEMS Geo-Enabled Monitoring and Supervision GoH Government of Haiti ICR Implementation Completion and Results Report IRI Intermediate Results Indicator ISR Implementation Status Report M&E Monitoring and evaluation MAST Ministry of Social Affairs and Labor (Ministère des Affaires Sociales et du Travail) MIS Management Information System MTR Midterm Review NGO Non-government Organization OAS Organization of American States OPD Organizations of persons with disabilities PNPPS National Policy for Social Protection and Promotion (Politique Nationale de la Protection et Promotion Sociale) PPIPDSPEP Project to Promote the Inclusion of People with Disabilities in Social Protection and Employment Programs PDI Project Development Indicator PDO Project Development Objective PHRD Policy and Human Resources Development Trust Fund PIU Project Implementation Unit PLR Performance and Learning Review RETF Recipient-Executed Trust Fund SIMAST Social Registry of the Ministry of Social Affairs and Labor (Système d’Information du MAST) TSPS Social Protection Sectoral Table (Tables Sectorielles de Protection Sociale) TTL Task Team Leader UNICEF United Nations’ Children’s Fund USAID United States Agency for International Development WFP United Nations’ World Food Program TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 II. OUTCOME .................................................................................................................... 12 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 20 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 22 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 25 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 33 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 34 The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 1 of 38 DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P145468 Promoting Inclusion of Persons with Disabilities in Social Protection and Employment Programs Country Financing Instrument Haiti Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Ministry of Economy and Finance Office of the Secretary of State for the Integration of the Persons with Disabilities Project Development Objective (PDO) Original PDO The development objective is to identify persons with disabilities (PwD) and increase their access to social services and to employment opportunities. Revised PDO The development objective is to identify persons with disabilities (PwD) and increase their access to employment services The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 2 of 38 FINANCING F INA NC E_TB L Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-18710 2,294,320 2,294,320 2,152,776 Total 2,294,320 2,294,320 2,152,776 Total Project Cost 2,294,320 2,294,320 2,152,776 KEY DATES Approval Effectiveness Original Closing Actual Closing 28-Jun-2018 05-Oct-2018 30-Jun-2020 30-Jun-2021 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 28-Jun-2020 0.26 Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Implementation Schedule Other Change(s) 09-Sep-2020 0.32 KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Moderately Satisfactory Substantial The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 3 of 38 RATINGS OF PROJECT PERFORMANCE IN ISRs No. Date ISR Archived DO Rating IP Rating Actual Disbursements (US$M) 01 18-Mar-2020 Moderately Unsatisfactory Moderately Unsatisfactory 0.26 02 17-Dec-2020 Moderately Satisfactory Moderately Satisfactory 0.57 03 17-May-2021 Moderately Satisfactory Moderately Satisfactory 1.31 04 07-Jun-2021 Satisfactory Satisfactory 1.80 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Carlos Felipe Jaramillo Country Director: Anabela Abreu Lilia Burunciuc Director: Michal J. Rutkowski Luis Benveniste Practice Manager: Pablo Gottret Pablo Gottret Task Team Leader(s): Snjezana Plevko, Aissatou Maisha Dicko Caroline Anne Isabelle Tassot, Briana N. Wilson ICR Contributing Author: Emma Schwirck Willenborg The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 4 of 38 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context at Appraisal A. Country Context 1. At appraisal, Haiti’s population faced high levels of poverty, low levels of human development and heightened risk to political crises, natural disasters, and other shocks. The devastating January 12, 2010 earthquake that struck just outside of Port-au-Prince killed 220,000 people and displaced 1.5 million. As of 2012, nearly 60 percent of Haiti’s population was living below the national poverty line and 24 percent (2.5 million) was extremely poor. 1 In 2015, Haiti was ranked 168 out of 187 countries according to the United Nations Human Development Index based on social indicators such as literacy, life expectancy, infant and maternal mortality, and child malnutrition. Recurrent shocks exacerbated Haiti’s underlying socioeconomic drivers of poverty and vulnerability, threatened livelihoods, hampered service delivery, and worsened human development outcomes. In October 2016, category four Hurricane Matthew hit Haiti’s south-west coast, affecting over 2 million people and reconstruction needs were assessed at 25 percent of GDP. 2 2. Persons with disabilities were among the most at-risk groups in Haiti to both poverty and shocks and faced low human development outcomes. At the same time, minimal reliable nationally representative estimates of disability prevalence existed. According to Handicap International and the National Network of Associations for the Integration of Persons with Disabilities, 800,000 individuals had some form of disability in 2003. The latest census conducted in 2003 estimated the prevalence of disability at 1.6 percent of the population, based on questions about severe impairments. 3 One quarter of the 280,000 persons injured during the 2010 earthquake were severely injured and seven percent had at least one permanent disability due to physical or mental factors. 4 Furthermore, public infrastructure was severely damaged which rendered the environment less accessible to individuals with disabilities and disrupted informal, community-based networks of support. As of 2014, approximately 172,000 people, including at least 900 persons with disabilities, were still living in camps. 5 3. A lack of national-level coordination and limited evidence base hampered the Government of Haiti’s (GoH) ability to identify and reach persons with disabilities with services and support. Although the GoH did prioritize the issue of disability, institutional capacity was low and donor interventions were often fragmented. Some resources for persons with disabilities were available, however a lack of data meant that needs were not well understood and often overlooked by public agencies. Anecdotal evidence was consistent with other low-income countries, as persons with disabilities and their households were more likely to be poor when compared to other households, and persons with disabilities themselves were more likely to live in extreme poverty. Unemployment of persons with disabilities also tends to be higher due to barriers to entering the workplace, such as competition for limited job opportunities, lack of accessibility, and limited access to education or training programs. In Haiti, there was a need to strengthen institutional 1 World Bank World Development Indicators. 2 Government of Haiti, World Bank, and Inter-American Development Bank, 2016. Rapid Damage and Loss Assessment of Hurricane Matthew. 3 World Bank, 2011. Report on Disability. Severe disability is defined as “the equivalent of disability inferred for conditions such as quadriplegia, severe depression, or blindness.” 4 London School of Hygiene and Tropical Medicine and Handicap International, 2012. Representation and evaluation of disability in Haiti. 5 Temporary camps were established for individuals displaced due to the 2010 earthquake. Source: Handicap International, 2014. Haiti: Situation update – Jan. 2014. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 5 of 38 capacity, create a system to identify eligible beneficiaries, and systemize interventions to improve efficiency and ensure existing resources reach those most in need. B. Sectoral and Institutional Context 4. In 2007, the Office of the Secretary of State for the Integration of Persons with Disabilities (BSEIPH) was established by Presidential decree under the Ministry of Social Affairs and Labor (MAST). Therefore, the BSEIPH had its own mandate but contributed to overarching objectives for social protection coverage in Haiti as defined by the MAST. BSEIPH’s objectives were to implement a national policy to support the integration of persons with disabilities, improve their living conditions, raise awareness among the public about disability, strengthen the capacity of organizations working in the field of disability, and put in place a legal framework for the protection of persons with disabilities. At appraisal, the Secretary of State of the BSEIPH was a long-time advocate regarding issues of disability and had support from the President. While the agency lacked resources, institutional capacity, and had a limited presence outside of Port- au-Prince, the BSEIPH had taken initial steps to create a job placement service to support employment opportunities for persons with disabilities. A full-time staff member had been hired to develop employment services, organize job fairs, and establish a website to better link persons with disabilities and prospective employers. However, capacity gaps persisted, including limited outreach, poor coordination with other social ministries, and low capacity to monitor, evaluate, and use data on persons with disabilities to plan and target potential benefits and programs. 5. Following the 2010 earthquake, Haiti’s Parliament enacted a law that required improved social and economic inclusion of persons with disabilities. Schools and public buildings were required to be accessible and two percent of jobs in companies with more than 20 employees had to be reserved for persons with disabilities. The textile industry had also agreed to employ an unspecified number of persons with disabilities. In March 2012, Haiti approved additional legislation to integrate persons with disabilities into public programs and services. In October 2012, the President and the Prime Minister signed a government declaration to mainstream the disability agenda across ministries, demonstrating support for the disability agenda by the highest level of the Government. However, a lack of resources, technical expertise and coordination, including to identify and register persons with disabilities and link them with services, were preventing these high-level objectives from being met. Employers were also largely unaware of issues of accessibility and the employment potential of workers with disabilities. 6. At appraisal, several non-state actors were involved in the field of disability in Haiti. However, uncoordinated and unregulated service provision contributed to significant variation in the quality and reach of services provided. At the time, the Organization of American States (OAS) supported legal frameworks for the inclusion of persons with disabilities; the United States Agency for International Development (USAID) was strengthening legal frameworks for disability and supporting rehabilitation and reintegration of persons with disabilities; and several NGOs (e.g., Christian Blind Mission, Humanity and Inclusion) were providing direct services to persons with disabilities. No agency was developing a registry or systematically providing job support to persons with disabilities, and the government did not have a strong role in setting policy and coordinating assistance from donors, development organizations, and private sector providers working in the disability space. However, the national social registry SIMAST, managed by the MAST, was established in 2013 with household-level demographic and socioeconomic information to facilitate targeting of social assistance programs in Haiti. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 6 of 38 Project Development Objectives (PDOs) 7. The Project Development Objective was to identify persons with disabilities and increase their access to social services and to employment opportunities. 8. Three priority actions were identified at the time of appraisal to better assess the needs of persons with disabilities and reach them with adequate services and support. These included: (1) establish a registry of persons with disabilities to assess the needs of persons with disabilities and support more effective planning and targeting of social programs and services; (2) strengthen government institutions to foster an effective design and implementation of disability policies and programs; (3) improve employment access and retention to economically empower persons with disabilities, as well as their household members and the communities where they live. 9. The operation aimed to support the GoH’s efforts to identify and register persons with disabilities and support their access to social services and employment opportunities. A mapping of the original Results Chain underlying the original operation is shown in Figure 1. Component 1 (registration of persons with disabilities) supported the BSEIPH to develop a registration system for persons with disabilities to identify their needs and improve the targeting of social services. The registration system for persons with disabilities was designed to link to the national social registry SIMAST, which is managed by the MAST and used for the targeting of social protection programs. Establishing a context-adapted and accessible system would allow for the continual registration and updating of information on persons with disabilities. Institutional support to the BSEIPH at central and departmental levels would ensure staff have the capacity to manage the registry. Component 1 also included the identification and registration of persons with disabilities in the system to support more accurate targeting of social services and support based on identified needs, and enhance access to health, education, social protection services. Component 2 (piloting and promoting a model to increase labor market access for persons with disabilities) consisted of support to the existing job placement service at BSEIPH through trainings, counseling, fairs, and the development of a management information system (MIS) for the job placement database. These activities aimed to support the institutional and staff capacity at the BSEIPH to support job-seeking persons with disabilities, while also supporting persons with disabilities to gain the skills and connections with employers to better link them to the labor market. This component also included awareness-raising efforts with employers to promote awareness and enhance employment of person with disabilities. These efforts would in turn increase access of persons with disabilities to employment opportunities. 10. Project objectives and approach were influenced by the source of financing as well as other planned World Bank projects. Financing was from the Policy and Human Resources Development (PHRD) Trust Fund, which identified two priority areas: (1) addressing the capacity building needs of government organizations; and (2) supporting the introduction of scalable methods, led by government or other development actors, to improve the well-being of persons with disabilities. Furthermore, the project was designed as complementary to the IDA-financed project, Improving Maternal and Child Health and Nutrition through Integrated Social Services (PASMISSI, P123706). The original PASMISSI design included a component to increase access of poor and vulnerable households to health and social protection services by scaling up the Kore Fanmi network of community agents and to support the GoH to develop a national social registry (which would later become integrated into the SIMAST). Therefore, the PHRD project aimed to complement these activities by promoting the inclusion of persons with disabilities in the registry and support their access to health and nutrition services. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 7 of 38 Key Expected Outcomes and Outcome Indicators 11. PDO achievement was to be captured by the following PDO Indicators (PDIs): • PDI 1 – Persons with disabilities documented in the national registry (number) • PDI 2 – BSEIPH offices that receive capacity building in the registry The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 8 of 38 12. Achievement of intermediate outcomes was captured by the following Intermediate Results Indicators (IRIs). For the purposes of this report, IRIs are used as proxies for PDIs in certain outcome areas where a PDI is not defined. • IRI 1 – Human resources trained to identify persons with disabilities (number) • IRI 2 – Persons with disabilities who participate in the job placement service (number) • IRI 3 – Persons with disabilities who participate in the job placement service who are female (percentage) • IRI 4 – Employers included in the job placement service database (number) • IRI 5 – BSEIPH staff and consultants receiving communication training (number) • IRI 6 – Awareness raising events implemented (number) Components 13. Component 1: Registration of persons with disabilities (US$709,000, equivalent to 30.9 percent of total grant amount). The objective of this component was to provide reliable data on persons with disabilities in Haiti. The component supported the development and application of a context-adapted, disability-specific survey alongside communication efforts to raise awareness of the registration campaign. The project aimed to register 100,000 persons with disabilities in the national social registry over a three-year period. Geographic coverage of the survey was partial and included metropolitan Port-au-Prince as well as 50 communes with legacy social registry data to allow for an initial identification of households. Activities were to be carried out in coordination with key social ministries and other public agencies. The two main sub-components included the development of a registration system for persons with disabilities and identification and registration of persons with disabilities. 14. Component 2: Piloting and promoting a model to increase labor market access for persons with disabilities (US$1,138,700, equivalent to 49.6 percent of total grant amount). This component supported the job placement service for persons with disabilities, a database operated by the BSEIPH that links persons with disabilities to job opportunities in the formal sector as well as skills training, counseling, and job fairs. Both persons with disabilities and employers are registered in the job placement service to facilitate the job search and recruitment efforts. Under the law for the integration of persons with disabilities in Haiti, at least two percent of companies’ positions were required to be filled by persons with disabilities. This component aimed to help persons with disabilities gain employment opportunities and companies to comply with existing regulations. Additionally, all persons with disabilities using the service were advised on available job offers identified through consultation and meetings with local employers. This sub-component aimed to train employers to raise their awareness of disability and help them identify actions to take to increase employment of persons with disabilities. 15. Component 3: Project management, monitoring & evaluation, and knowledge dissemination (US$446,620, equivalent to 19.5 percent of total grant amount). This component aimed to strengthen the BSEIPH’s capacity for effective outreach and communication, as well as registry management and quality control. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 9 of 38 Significant Changes During Implementation A. Restructuring 16. The project underwent one Level 2 restructuring in June 2020. The restructuring aimed to address exogenous disruptions such as repeated political unrest and the COVID-19 pandemic and sector-specific constraints including limited availability of social services and employment opportunities for persons with disabilities. The main changes included: (1) Extension of the project closing date from June 30, 2020 to June 30, 2021; (2) Revision of the PDO and project title to align with revised timeframe and constraints imposed by the COVID-19 pandemic; (3) Revision of the Results Framework; (4) Revision of the component funds allocation; and (5) Revision of the disbursement category allocation. B. Revised PDO and Project Title 17. The operation’s PDO, closing date and project title were modified during the restructuring in June 2020. The original PDO to ‘identify persons with disabilities and increase their access to social services and employment opportunities’ was revised to ‘identify persons with disabilities and increase their access to employment services.’ The project closing date was extended by one year from June 30, 2020 to June 30, 2021 and the project title ‘Improving Access to Social Services & Employment Opportunities for Persons with Disabilities’ was revised to ‘Promoting Inclusion of Persons with Disabilities in Social Protection and Employment Programs.’ These changes are summarized below and discussed in the sections that follow. • ‘Access to social services’ was removed from the PDO for two reasons. First, social services were not available as anticipated during appraisal. The GoH-led Kore Fanmi initiative which aimed to improve service delivery through a network of community agents, including with financing from the World Bank PASMISSI project (see above), was not successfully scaled up and eventually discontinued. 6 The original project design also did not link the objective of increased access to services with defined project components, activities, or indicators, which represented a flaw in the original project theory of change. • ‘Access to employment opportunities’ was changed to ‘employment services’ due to project specific constraints notably the overly ambitious project design and short implementation timeframe which was not sufficient to achieve final employment outcomes. A longer project timeframe with more robust demand-side support, such as apprenticeship programs or more intensive employer engagement, would have a greater potential to contribute to meaningful, long-term employment for persons with disabilities in Haiti. C. Revised Indicators 18. PDO indicators (PDIs) and Intermediate Results Indicators (IRIs) were modified to reflect the changes introduced under the project restructuring (see Table 1). The main modifications at the PDI level included: i) the elimination of original PDI 2 due to the lack of staff and other resources in decentralized BSEIPH offices to perform registry management; ii) the inclusion of the original IRIs 2 and 3 as revised PDIs 2 and 3; and iii) the addition of PDI 4. These adjustments were made to ensure activities under Component 2 were reflected in the results framework. Finally, targets were revised downward for certain indicators to correct for an overly ambitious project design alongside delays in implementation, including in the context of the COVID-19 pandemic which limited in-person data collection and training activities. These changes are reflected in Table 1 below and described in more detail in the Efficacy section. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 10 of 38 Table 1. Changes in PDO Indicators (PDIs) and Intermediate Results Indicators (IRIs) Original PDO Indicators (PDI) Target Changes Revised PDIs Target PDI 1. persons with disabilities documented in the national registry (number) 100,000 Revised target PDI 1. persons with disabilities documented in the national registry (number) 50,000 PDI 2. BSEIPH offices that receive capacity- building in the registry (number) 5 Dropped -- -- Added as PDI (prior IRI); revised target PDI 2. persons with disabilities who participate in the job placement service (number) 500 Added as PDI (prior IRI) PDI 3. persons with disabilities who participate in the job placement service who are female (percentage) 50% Added/New PDI 4. persons with disabilities who participate in training and obtain certification (number) 200 Original Intermediate Results Indicators (IRI) Target Changes Revised IRIs Target IRI 1. Human resources trained to identify persons with disabilities (number) 1,000 Revised indicator and target IRI 1. Human resources trained on employment opportunities for persons with disabilities (number) 30 IRI 2. persons with disabilities who participate in the job placement service (number) 1,000 Changed to PDI; Revised target IRI 3. persons with disabilities who participate in the job placement service who are female (percentage) 50% Changed to PDI IRI 4. Employers included in the job placement service database (number) 100 Revised target Became IRI 2. No change to wording. 30 IRI 5. BSEIPH staff and consultants receiving communication training (number) 10 No change Became IRI 3. No change to wording. 10 IRI 6. Awareness raising events implemented (number) 5 Revised target Became IRI 4. No change to wording. 3 Other Changes 19. Changes to the component and disbursement category allocations were made during the restructuring. Component 1 allocation increased from US$709,000 to US$1,244,320, whereas Component 2 decreased from US$1,138,700 to US$600,000. Component 3 increased slightly from US$446,620 to US$450,000. Component 1 cost allocation was increased due to higher than estimated costs of the registration survey, which required fielding the SIMAST module in addition to the disability survey (discussed in the Efficiency section below). Revision of the disbursement 6 Kore Fanmi (KF) had been introduced as a pilot project in the Central Plateau department and involved mapping of social services (using the KF management information system) and support for a local network of community agents. However, the KF approach faced challenges in achieving scale due to resource and capacity constraints and was ultimately phased out. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 11 of 38 category allocation reflected the need for higher disbursements for Consulting Services for the survey and lower disbursements for Training and Workshops and Goods, due to reduced equipment needs. Synergies with the social registry SIMAST system allowed the BSEIPH to access equipment without support from this grant, while maintaining the training activities. Revised Theory of Change 20. The changes to the PDO and results framework are reflected in the revised theory of change (Figure 2). While project activities did not fundamentally change with the restructuring, intermediate and PDO outcomes and indicators were revised to better reflect what the original design intended to achieve, as well as to reflect challenging local conditions due to COVID-19 and political unrest. No PDO indicators were specified under Component 2 in the original project design, so the revised theory of change includes new PDO indicators that were added to better reflect outcome- level progress under Component 2. Under Component 2, the focus was shifted to enhance access to employment services The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 12 of 38 rather than employment opportunities to better reflect existing project activities and feasible outcomes given the short project timeframe. Finally, the new PDO removed reference to increased access to social services, so the associated intermediate outcome and PDO outcome are removed from the revised theory of change. II. OUTCOME Assessment of Achievement of Each Objective/Outcome A. Relevance of PDOs Overall Rating: High 21. The relevance of the operation’s revised PDO is deemed High. The PDO is aligned with the WB’s Country Partnership Framework (CPF) for the Republic of Haiti FY16-19 (Report No. 98132-HT) discussed by the Board of Directors on September 29, 2015 and the Haiti Performance and Learning Review (PLR) of the CPF (Report No. 124812-HT). 7 CPF Area of Focus 2 (Human Capital) aimed to support poverty reduction through investments in health and education and by closing gaps in service delivery. The Project contributed to this objective by improving the targeting of social protection services to reach persons with disabilities and by enhancing their access to social and employment services. The project contributed to CPF Areas of Focus 1 (Inclusive Growth) through employment support for persons with disabilities to promote a more inclusive and accessible labor market. In addition, the 2015 Systematic Country Diagnostic (SCD) for Haiti identified improved targeting in social protection as a priority in the face of shocks and vulnerability. Project activities contributed to several cross-cutting SCD priorities, including strengthening state capacity, generating more and better jobs, reducing vulnerability, and developing evidence to inform planning and targeting of policies and programs. Finally, the Project supported the CPF cross-cutting theme on governance by enhancing efficiency of service delivery through improved targeting of persons with disabilities using the social registry and by supporting the BSEIPH to produce relevant data, manage the social protection sector, and set evidence-based policy priorities. 22. The PDO is consistent with domestic priorities and continues to be highly relevant to Haiti's growing social protection agenda. With the creation of BSEIPH in 2007 and passage of the 2012 law requiring better inclusion of persons with disabilities, the Project represented a crucial step towards building GoH capacity to better identify and support persons with disabilities in a context of high vulnerability. Additionally, the National Policy of Social Protection and Promotion (PNPPS) that was adopted in 2020 prioritizes support for persons with disabilities including improved coordination between MAST and the BSEIPH and recognized the SIMAST social registry as the foundation for the targeting of social protection programs. B. Achievement of PDOs (Efficacy) 23. The following considerations should be noted regarding the methodological approach adopted for this Implementation Completion and Results Report (ICR): • The changes introduced under the restructuring in June 2020 required utilization of the split methodology. The relative weights reflect the disbursements made during the pre- and post-restructuring periods out of total disbursements (11.4 percent and 88.6 percent respectively). 7 The CPF period was originally set from 2015 to 2019. However, after completion of the PLR in 2018, CPF milestones were extended until 2021 while a new CPF is prepared. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 13 of 38 • Efficacy for the pre-restructuring period is assessed based on the original PDO, which has been unpacked into three PDOs: identification of persons with disabilities (PDO 1); access to social services (PDO 2); and access to employment opportunities (PDO 3). Performance is assessed against the original targets in the Project Results Framework. • Efficacy for the post-restructuring period is assessed based on the revised PDO, which has been unpacked into two PDOs: identification of persons with disabilities (PDO 1) and access to employment services (PDO 3). Performance is assessed against the revised targets in the Project Results Framework. Table 2. Summary of Efficacy Ratings – Pre and Post Restructuring Periods Outcomes Efficacy Rating Pre-restructuring (Original) Post-restructuring (Revised) PDO 1: to identify persons with disabilities Substantial High PDO 2: to increase access of persons with disabilities to social services Negligible N/A (Dropped) Original PDO 3: to increase access of persons with disabilities to employment opportunities Revised PDO 3: to increase access of persons with disabilities to employment services Modest High Pre-restructuring Rating: Modest 24. Efficacy for PDO 1 during the pre-restructuring period is assessed as Substantial. PDO 1 included two key activities: (1) development of a registration system for persons with disabilities, and (2) identification and registration of persons with disabilities. By project closure, PDI 1 was over partially achieved with 53,453 persons with disabilities 8 registered in the SIMAST (see Table 3). A survey instrument was developed to capture information on persons with disabilities and was adapted to the local context, Creole language, and specificities of conducting surveys with persons with disabilities (including sign language interpreters and proxy interviews). 9 The questionnaire was based on the extended set of survey questions recommended by the Washington Group 10 in addition to questions on education, labor market status, and assistance received based on disability status. The survey tool was designed to link to the SIMAST survey in order to include persons with disabilities in the social registry. The core social registry module was also administered to collect information on household characteristics. Data collection was delayed due to political unrest, COVID-19 restrictions on in-person activities, and complexities in preparing the disability module as described above. A database manager hired by the Project received training in the SIMAST registry. 8 Surveys were conducted for all levels of disability (including severe, moderate, and mild). 9 The process for piloting, adapting, and deploying the survey tool included: (1) a workshop to pilot the questionnaire with people with different types of disabilities; (2) an inclusive data collection methodology; (3) specific training on issues for data collection staff; and (4) close collaboration with local authorities and Organizations for Persons with Disabilities (OPDs) for data collection. The BSEIPH worked closely with the National Food Security Coordination (CNSA) to conduct the process. 10 The Washington Group on Disability Statistics (WG) was established under the United Nations Statistical Commission to address the need for cross-nationally comparable population-based measures of disability. Its mandate is to promote international coordination for health statistics by focusing on disability data collection for censuses and national surveys. The extended survey includes detailed modules on types of disability, as well as needs in terms of assistive devices. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 14 of 38 Table 3. Status of original PDO 1 Indicators at closure Indicator Actual at Closure Original Target PDI 1: persons with disabilities documented in the national registry (number) 53,453 100,000 PDI 2: BSEIPH offices that receive capacity-building in the registry (number) Dropped (only the central office received training) 5 25. Efficacy for PDO 2 during the pre-restructuring period is assessed as Negligible. PDO 2 was ‘to increase access of persons with disabilities to social services.’ By including persons with disabilities in the social registry (SIMAST), it was assumed that they would be reached with existing social services and support. While progress was made towards including persons with disabilities in the registry, improving access to social services would have required a stronger supply side intervention to ensure that services were made available to persons with disabilities. The original project design did not include any specific activities or indicators to assess progress towards achievement of PDO 2, and at the same time overestimated the availability of social services for persons with disabilities. Therefore, achievement of this objective was negligible, and PDO 2 was dropped during the project restructuring. 26. Efficacy for PDO 3 during the pre-restructuring period is assessed as Modest. PDO 3 was ‘to increase access of persons with disabilities to employment opportunities’ and consisted of two key activities: (1) support the job placement service at BSEIPH and (2) raise awareness among employers to promote employment of persons with disabilities. There were no PDO indicators (PDIs) included in the original project design 11 to measure progress at the outcome level for this component of the PDO, so intermediate results indicators (IRIs) are used instead. By project closure, IRI targets for PDO 3 were either achieved or partially achieved (see Table 4). Therefore, the operation was expected to partly achieve its original objectives under PDO 3. The gaps between actual at closure and the original target are explained by an overly ambitious project design given the short implementation timeframe and difficult operating context in Haiti, which are discussed further in Section 3 below. Table 4. Status of original PDO 3 Intermediate Results Indicators (IRI) at closure Indicator Actual at Closure Original Target IRI 1 (original). Human resources trained to identify persons with disabilities (number) IRI 1 (revised). Human resources trained on employment opportunities for persons with disabilities (number) 40 1,000 IRI 2. persons with disabilities who participate in the job placement service (number) 679 1,000 IRI 3. persons with disabilities who participate in the job placement service who are female (percentage) 44.8% 50% IRI 4. Employers included in the job placement service database (number) 40 100 IRI 5. BSEIPH staff and consultants receiving communication training (number) 10 10 IRI 6. Awareness raising events implemented (number) 7 5 Post-restructuring Rating: Substantial 27. Efficacy for PDO 1 during the post-restructuring period is assessed as High. PDO 1 remained unchanged from the pre-restructuring period, however the end target was revised from 100,000 to 50,000 persons with disabilities documented in the national registry. The revised target reflected conditions in the country at the time of restructuring 11 Only two PDIs were included in the original project design, and both were associated with Component 1 (PDO 1 – to identify persons with disabilities). This is reflected in the original theory of change (Figure 1). The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 15 of 38 (discussed in Section 3 below) and accounted for the longer process of developing the disability survey module, and was therefore more realistic given the short project timeframe. However, considerable progress was made in the final year of implementation. Outreach was conducted through a communications campaign led by a service provider, and by using existing lists of persons with disabilities available from the BSEIPH, other organizations including Organizations for People with Disabilities (OPDs), and local authorities. By project closure, 53,453 persons with disabilities were documented in the national registry. 12 To facilitate registration, BSEIPH introduced several innovations including a system to issue unique beneficiary cards to registered persons with disabilities, a media presence to encourage enrolment (through radio, Twitter, Facebook, billboards and posters), and piloting of self-registration for persons with disabilities using kiosks, either in-person, via dependents or organizations that support persons with disabilities. The self-registration was used for pre- identification purposes and followed up with home visits conducted by enumerators to validate the information provided and conduct the full survey. The original PDI 2 (BSEIPH offices that receive capacity-building in the registry) was dropped due to the short project implementation timeframe and lack of staff and resources in the decentralized BSEIPH offices, however the central BSEIPH office received capacity building. Table 5. Status of revised PDO 1 Indicators at closure Indicator Actual at Closure Revised Target PDI 1: persons with disabilities documented in the national registry (number) 53,453 50,000 28. Efficacy for PDO 3 during the post-restructuring period is assessed as High. PDO 3 changed from the pre- restructuring period to reflect increased access of persons with disabilities to employment services rather than employment opportunities. The revised PDO 3, indicators and targets, were in line with the conditions in the country at the time of restructuring and the short implementation timeframe. One PDI was added and two IRIs were elevated to PDIs to ensure that progress towards PDO 3 was sufficiently captured at the outcome level, based on the two major activities under Component 2 (support to job placement service and provision of training). The target for PDI 2 (original IRI 2) was reduced from 1,000 to 500 persons with disabilities participating in the job placement service due to the difficult country conditions, and several end targets were revised to the original IRIs as well. PDI 4 was added at restructuring to measure the number of persons with disabilities who participated in training and obtained certification. Requiring certification added an element of rigor to the training while signaling the higher value of the training to potential employers. Many individuals who participated in the job placement service and/or trainings did go on to obtain productive employment (including self-employment), indicating that final employment outcomes were achieved beyond the scope of the revised PDO. 29. Significant progress was made in the final year of implementation for PDO 3. Most revised targets were surpassed thanks to a proactive outreach strategy to employers and human resources departments and awareness-raising events to promote the integration of persons with disabilities in the labor market. Outcomes beyond the PDIs were achieved, including increased access to employment as envisaged in the original PDO. The BSEIPH job placement database expanded rapidly in the final year to include 679 individuals (exceeding the revised target of 500), including those who completed the trainings or participated in the job fairs, coaching workshops, or other BSEIPH services and activities. Approximately 45 percent of individuals who participated in the job placement service were female, which fell slightly short of the 50 percent target. Through the job placement service, 27 persons with disabilities were matched with employment opportunities and 12 received scholarships to continue their studies. Training opportunities were provided 12 It is important to note that the survey did not have full geographic coverage in Haiti but covered 50 communes already included in the existing SIMAST social registry as well as select communes in the Port-au-Prince metropolitan area. Therefore, while the number of persons with disabilities registered is significant and exceeded final revised targets, additional efforts will be needed to ensure full coverage of the social registry and disability module to register the full population of individuals with some form of disability in Haiti. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 16 of 38 for 157 individuals to improve their job-related skills in agricultural production and product transformation, motorcycle repair, cooking and baking, cellphone repair, infographics, massage therapy, as well as in employment search techniques. The training programs were carried out by service providers and designed for individuals with all types of disabilities except intellectual disabilities. Of the 157 individuals who completed training and obtained certification, 74 went on to create their own income-generating activities. Training was conducted in skill areas identified as ‘high-demand’ based on an employment study, which is evidenced by employment within one month of nearly one half of those trained. By the end of the project, an additional 60 individuals had completed most of the IT training with certification and distribution of job starter kits pending. 13 Another funding source was proactively identified to allow training completion. Table 6. Status of revised PDO 3 Indicators at closure Indicator Actual at Closure Revised Target PDI 2: persons with disabilities who participate in the job placement service (number) 679 500 PDI 3. persons with disabilities who participate in the job placement service who are female (percentage) 44.8% 50% PDI 4. persons with disabilities who participate in training and obtain certification (number) 157 200 C. Efficiency (implementation efficiency and economic efficiency) Rating: Substantial 30. Overall, efficiency is rated as Substantial. While the operation’s efficiency was overestimated during the design stage as discussed in Section 3 below, actual efficiency during implementation aligned with what would be expected in the social sectors in Haiti. With regards to implementation efficiency, the time between approval and first disbursement (12.6 months) was not uncommon for projects in Haiti and can be largely explained by external factors including institutional turnover and political instability (see Section 3). The restructuring corrected for initial shortcomings in project design (including overly ambitious objectives and targets given the context and implementation timeframe), and following restructuring, project implementation picked up significantly as project resources were efficiently translated into outcomes for all project activities. 31. With regards to economic efficiency, economic analyses prepared during appraisal focused on two key areas: (1) registering persons with disabilities to improve targeting of social protection programs; and (2) increasing labor market participation of persons with disabilities to bring about economic benefits. First, the project aimed to improve the capacity of the GoH to design, target, and manage social protection programs to better meet the needs of persons with disabilities. At appraisal, costs of registration were estimated at US$7 per person registered. The estimated cost did not capture the actual costs of the disability module in addition to the existing SIMAST survey or the added costs of surveying a hard-to- reach population which requires intense outreach and adapted survey methodology. The actual cost of registration was $22 per person registered, which is more in line with similar types of surveys in Haiti and better reflects local realities. Survey costs tend to be much higher in fragile and conflict affected areas with weak infrastructure and high levels of 13 The additional 60 individuals are not included in the PDI 4 actuals because training would be completed after project closure. With the anticipated completion of training, PDI 4 end target would be exceeded (217 persons with disabilities participated in training and obtained certification). It is important to note that COVID-19 significantly affected implementation of training activities and contributed to delays in the final IT training. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 17 of 38 insecurity, which makes access extremely difficult. 14 Despite the cost, the investments made in establishing systems and building capacity to continue to identify and register persons with disabilities in the SIMAST will provide more accurate data to improve targeting of social programs and allow for better planning of service provision and resource allocation. The registry will allow the GoH and local and international partners, to know the number and location of persons with disabilities, analyze their needs and plan for future policies and programs to facilitate a more efficient use of limited resources. For example, the SIMAST data was used to support analytical work estimating the impact of the August 14 th earthquake 15 as well as to inform the operational response in terms of emergency cash transfers to vulnerable populations (prioritizing persons with disabilities) in the affected area. 32. The exclusion of persons with disabilities from the labor market has been found to reduce human potential and limit national economic output. The International Labor Organization (ILO) estimates show that if persons with disabilities were employed at the same level as persons without disabilities, GDP would increase by 3 to 7 percent. 16 The potential economic benefits arising from inclusion of persons with disabilities in the labor market can be significant, particularly in a country such as Haiti which is estimated to have a high prevalence of disability. The project aimed to contribute to increasing labor market access for persons with disabilities through the job placement service, job training, and improving employer awareness. At appraisal, costs of vocational training were estimated at US$1,675 per person for a total of 600 individuals. Actual costs were lower than anticipated at US$1,260 per person on average, including starter kits which were provided to each trainee. While there was not enough information to quantify benefits upon project closure, the high rates of employment including self-employment following training completion suggest that the longer-term benefits may be substantial, particularly given the high equity impacts of integrating persons with disabilities into the labor market in Haiti. Job placement services have been shown to be one of the more effective ways to enhance labor market participation of persons with disabilities worldwide. 17 . Finally, enhanced linkages between registration of persons with disabilities in the SIMAST and their inclusion in the job placement service could bring additional benefits. Overall Outcome Rating 33. The project restructuring to revise the PDO and PDO targets justifies the use of the split rating method to arrive at the final outcome rating. This method considers both original and revised objectives and outcome targets in deriving the project’s overall outcome rating. Efficacy ratings for the Original and Restructured project timeframes are weighted in proportion to the share of actual grant disbursements made in the periods before and after the restructuring. Table 7 demonstrates the calculation of the final outcome rating of Satisfactory based on the split rating method. 14 While not directly comparable to household data collection for social registries, Hoogeveen and Pape (2020) report that costs per interview for a phone-based Rapid Emergency Response Survey (RERS) ranged from US$86 in Nigeria and US$23 in Somalia. See https://link.springer.com/content/pdf/10.1007%2F978-3-030-25120-8.pdf 15 See Global Rapid Damage Estimation (GRADE) Report finalized on August 27,2021. 16 ILO and OECD. 2018. Labour market inclusion of people with disabilities. https://www.ilo.org/global/about-the-ilo/how-the-ilo- works/multilateral-system/g20/reports/WCMS_646041/lang--en/index.htm 17 Ibid. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 18 of 38 Table 7. Calculation of Overall Outcome Rating Original Restructured Relevance 18 High Efficacy Modest Substantial Efficiency 19 Substantial #1: Outcome ratings Moderately Unsatisfactory Satisfactory #2: Numerical value of outcome ratings 3 5 #3: Disbursement 0.26 million 2.03 million #4: Share of disbursement 11.4% 88.6% Weighted value of the outcome rating (# 2 x # 4) 0.34 4.43 Final outcome rating 4.77 (rounded up to 5.0) = Satisfactory Other Outcomes and Impacts A. Social Inclusion 34. The project made significant strides towards promoting inclusion of persons with disabilities in a context characterized by extreme vulnerability. Persons with disabilities were largely overlooked by government institutions and employers in Haiti and faced specific challenges such as high unemployment, an inaccessible environment, and unequal access to social services. The development of the disability module for the SIMAST will allow for detailed information on disability to be included in the national social registry, which is an essential step towards understanding the needs of persons with disabilities and building linkages with social protection programs and other social services. For example, persons with disabilities are included in the target population for the Adaptive Social Protection for Increased Resilience (ASPIRE) project financed by the World Bank (P174111). Accurate and up-to-date information on persons with disabilities will facilitate faster outreach and targeting in the event of natural disasters, such as the recent earthquake of August 14, 2021. In addition, increased awareness among employers will help persons with disabilities become better integrated into the job market. Early results show that there is high potential for employment of persons with disabilities based on the combined package of job services, training, and awareness among employers, but also that many persons with disabilities go on to start their own small businesses. At project closure, 40 employers were included in the job placement service and 27 persons with disabilities had already been matched with job opportunities. Finally, the project has increased awareness among the general public of the role of the BSEIPH and available services, including the job placement service, as well as awareness of persons with disabilities. The project focus on communications helped to further enhance these messages to a range of audiences (see Figures). 18 One rating for relevance in line with the ICR guidelines. 19 One rating for efficiency in line with the ICR guidelines. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 19 of 38 Figures 3-5. Examples from project communications campaign to raise awareness and advocate for persons with disabilities in Haiti. From left to right, Figure 3: Information on the law for the integration of persons with disabilities highlighting the employer requirement to hire persons with disabilities; Figure 4: “I am disabled, and I am talented.”; Figure 5: promotional materials to support registration of persons with disabilities in the social registry. Additional materials can be found at the BSEIPH Facebook page at https://www.facebook.com/bseiph/. B. Institutional Strengthening 35. Upon project approval, the BSEIPH Secretary of State had strong political will but the agency lacked resources and capacity to fulfill its mandate. A focus on institutional strengthening was embedded in project design and operationalized throughout project implementation. In a data-poor environment such as Haiti, a lack of information on the population with disabilities inhibited the ability of government and other actors to coordinate and implement effective interventions. Project activities aimed to improve the ability of the GoH to collect relevant information and mainstream disability into service delivery systems. Actions were taken to progressively increase the management and fiduciary capacity of the BSEIPH to expand the scope and reach of disability-focused initiatives in Haiti. With a strengthened mandate, the BSEIPH was able to increase engagement with the MAST and coordinate with non-governmental organizations that engage with persons with disabilities in Haiti. The BSEIPH led certain activities such as the coaching sessions directly, and hired external firms to organize the job fairs and support the communications campaigns to promote outreach, raise awareness, and register persons with disabilities in the SIMAST and the job placement service. The existing legal framework provides the GoH with a further opportunity to increase engagement with employers and promote compliance with the March 2012 law to achieve the two percent employment quota of persons with disabilities. Finally, the progress made in registering persons with disabilities was made at a critical time as significant expansion of the SIMAST is expected in the coming years alongside the operationalization of the PNPPS. 20 The SIMAST has proven to be an essential tool to increase availability of reliable data, build stakeholder buy-in for a central social registry managed by the MAST, and help promote coordination between the main social protection actors in Haiti. The success of these efforts is expected to support resource mobilization in the future for continued BSEIPH activities, who is actively seeking other financing to continue its work. 20 This includes investments by the World Bank and partners, including the recently approved Adaptive Social Protection for Increased Resilience (ASPIRE) project (P174111). The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 20 of 38 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME Preparation 36. Project preparation was significantly delayed due to a number of factors both subject to and outside of World Bank and GoH control. Time between initiation and approval spanned five years (2013-2018) when the country context changed significantly. Following the devastating 2010 earthquake, Haiti faced an outbreak of cholera from 2010-2019 that killed nearly 10,000 people. Both donor and government resources were heavily allocated towards the earthquake and public health response. At the same time, the social protection sector in Haiti faced a reduction in donor funding and institutional support. The social assistance strategy EDE-PEP 21 was developed in 2014, however most EDE-PEP programs, including Kore Fanmi, were discontinued due to a drastic reduction in funding, as well as political uncertainty. There was little continued investment in social protection programs at the national level and the project was not prioritized by either GoH or World Bank stakeholders. When relative political stability was restored in 2017 following the contested 2016 elections, the project was reintroduced for approval by the Country Management Unit (CMU) of the World Bank. However, given the long period of time that passed since initiation, there was little appetite on the part of the government or PHRD management to alter project design, which would have entailed further delays. As a result, the project was approved as it was designed in 2013, which led to mismatches between design and implementation, particularly with regards to the original component of the PDO to increase access to social services. Project objectives and targets were overly ambitious in light of the complex operating environment and short implementation timeframe, which were eventually corrected during the project restructuring. Implementation A. Factors subject to government and/or implementing agency control 37. The delays experienced between approval and the first disbursement were precipitated by political turnover of key decision-makers as well as resource and capacity gaps at the BSEIPH. This led to delays in fiduciary processes and in establishing the project implementation unit (PIU) within the BSEIPH. The BSEIPH Secretary of State who was a major champion during the first year and a half of the project changed in March 2020, and the Minister of the Interior changed twice over the course of implementation. This led to challenges in sustaining political ownership of project activities and to administrative bottlenecks and procurement delays due to: (i) setting up the PIU with a qualified procurement specialist; and (ii) issues with approving signature specimens and signing of contracts by the Minister of Interior. With each period of institutional turnover, it took time to ensure understanding of complex project goals and activities among new actors. Despite being deemed as the most adequate implementing agency given its expertise and mandate on disability issues, the BSEIPH had never implemented a World Bank project and there was a steep learning curve regarding administrative and fiduciary processes and requirements, fiduciary responsibility was therefore assigned to the Ministry of Interior’s PIU. No project funding was provided prior to effectiveness to facilitate preparation. During project start- up, institutional capacity was insufficient to manage multiple concurrent activities, including hiring the PIU, establishing fiduciary processes, and signing major contracts to quickly get project activities off the ground. These issues were exacerbated by institutional turnover and political instability which contributed to procurement delays. 38. The lack of clear governance and priorities prior to the adoption of the PNPPS in 2020 resulted in challenges in implementation. While the BSEIPH is located within the MAST, the Fund for Economic and Social Assistance (FAES) is a 21 “Help people” in Creole. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 21 of 38 separate, semi-autonomous social assistance agency. Initially, the relationships between BSEIPH, FAES, and MAST were unclear with overlapping mandates, limited communication, and no cohesive social protection policy at the time of project approval. Coordination was difficult and the decline in social protection funding over the course of the 2010s led to competition and reduced information sharing as multiple institutions sought out scarce resources to maintain their activities. These challenges exacerbated issues with donor coordination and engagement as GoH agencies were unable to reconcile different donor objectives or approaches and how they might fit together. FAES had been responsible for the Kore Fanmi activities and was envisioned as a key partner to the BSEIPH to support linkages with social services in the original project design, however the coordination was never formalized. When implementation began, FAES was often not engaged, in part due to limited institutional commitment but also insufficient financial and human resources and capacity. The establishment of the SIMAST as a unified social registry and recognition of MAST as the leading entity in social protection through the PNPPS helped streamline coordination. For example, the MAST played an active role in supporting the implementation of Component 1 to ensure the disability module and survey methodology were in line with the overall SIMAST strategy and guidelines, and to support negotiations with the data collection service provider. B. Factors subject to World Bank control 39. The World Bank faced challenges in the early stages of implementation to ensure adequate supervision, however, support improved over time. The difficult operating environment in Haiti requires extremely strong and proactive management. During the early stages of implementation, supervisory support provided to the BSEIPH could have been strengthened in order to more proactively identify and resolve issues as they arose. The Social Protection Global Practice (GP) increased its resources dedicated to supervision following effectiveness, which sped up implementation alongside more focused human resources. Supervision become more hands-on and proactive, including frequent face to face missions between August 2019 and March 2020, followed by regular virtual check-ins and missions in the context of the COVID-19 pandemic and restrictions on travel. The speed of implementation and results in the final year of the project can be partially explained by enhanced vigilance in supervision and rapid troubleshooting of issues as they arose. The project restructuring reflected proactive management to ensure realistic objectives could be achieved and was a result of the process to identify qualified firms to carry out project activities. This required a re-appraisal of project achievement including in response to COVID-19. The timing of the restructuring was reasonable considering the short project timeframe, however, was also influenced by a lack of resources (notably dedicated staff) early on in implementation. With regards to project reporting, the first ISR was published in March 2020, nearly a year and a half after project approval. However, starting in March 2020, ISRs were timely, of high quality, and adequately raised key issues to be addressed. C. Factors outside the control of government and/or implementing agencies 40. Haiti is an extremely difficult operating context and several external factors affected project implementation. Peyi lok (“country lockdown” in Creole) began in July 2018, less than one month after project approval, in response to civil unrest resulting from rising fuel prices and which gained momentum due to corruption allegations related to management of Petrocaribe funds. Protests peaked throughout the summer and fall of 2018, when schools, courts, businesses, public services, and economic production largely shut down. Protests continued throughout 2019, in the fall of 2020, and into spring 2021. The continued volatility led to an extremely unpredictable environment in which day-to- day implementation of activities was difficult at best and impossible at times. Haiti has become increasingly insecure with a rise in kidnappings and gang violence that further limited movement and conduct of project activities, ranging from ensuring staff safety to come to work every day, to proceeding with field activities despite continued security threats. Macroeconomic conditions were also highly variable. In particular, the fluctuation of the exchange rate created a major The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 22 of 38 challenge and months of delay in finalizing negotiations for the survey contract to conduct the data collection under Component 1. In the case of several procurements, it was challenging to find firms to perform the requested work due to the strict requirements in ensuring accessibility and safety for persons with disabilities, low capacity and availability of qualified firms, and a lack of trust in GoH institutions including issues with payment that prevented some firms from responding to procurement notices. 41. Finally, the first cases of COVID-19 were announced in Haiti on March 19, 2020 and a State of Emergency was declared on the same day. The quarantine, social distancing and movement restrictions put in place to reduce the spread of COVID-19 in Haiti limited project activities. Most notably, data collection under Component 1 and implementation of professional trainings under Component 2 were delayed due to limitations on in-person activities. All trainings had to be conducted in person due to their manual focus (baking, motorcycle repair, massage therapy for instance), as well as difficulties associated with connectivity in Haiti (for instance for the web design training). Adjustments, such as social distancing and use of protective gear, were required to proceed with the registration of persons with disabilities, and trainings had to be delayed until in-person activities could resume. In the absence of COVID-related constraints, the actuals at closure for two out of four of the final PDIs (notably PDI 1 - persons with disabilities documented in the national registry and PDI 4 - persons with disabilities who participate in training and obtain certification) would have been higher, although it is difficult to assess the exact magnitude of this difference. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME Quality of Monitoring and Evaluation Overall Rating: Substantial A. M&E Design: Modest 42. The challenges faced during project preparation led to shortcomings in M&E design. The PDO and results framework were overly ambitious and did not adequately reflect the country context at the time of approval. Project activities and indicators did not fully align with the PDO. In particular, the focus in the PDO on increasing access of persons with disabilities to social services was not reflected in the project theory of change and did not have linked activities or indicators in the results framework to measure progress. Length of implementation was insufficient to increase access of persons with disabilities to employment opportunities, and the original theory of change did not identify the crucial link between access to employment services and actual employment. The distribution of PDIs and IRIs was unbalanced, as there were no PDIs linked to Component 2. However, it is important to note that many of the shortcomings related to M&E were associated with Component 2, whereas Component 1 activities, indicators, and PDO (identify persons with disabilities) were well designed and clear. The project design did include hiring of an M&E specialist within the PIU, to facilitate accurate and regular reporting. B. M&E implementation: Substantial 43. Despite the challenges noted during the design phase, M&E data were consistently collected and analyzed throughout implementation to track project activities with sufficient institutional backing. Component 3, “Project management, monitoring and evaluation, and knowledge dissemination” ensured that there were sufficient resources available to collect and analyze available data. An M&E specialist was hired to the PIU and embedded within the BSEIPH. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 23 of 38 M&E implementation was regularly discussed during World Bank supervision missions to ensure understanding of key concepts and reliability of data. The registry of persons with disabilities and the job placement service’s MIS were the two main sources of data used to monitor progress and assess outcomes. These were updated regularly based on rollout of project activities, particularly in the final year of the project to keep up with the rapid pace of implementation. These data sources are now fully embedded within the GoH – the registry is incorporated into the SIMAST, and the job placement service is functional at the BSEIPH – which will facilitate continued tracking of key outcomes after project closing. A mid-term review (MTR) was conducted in December 2020 to assess progress and identify issues or additional adjustments following the project restructuring. The only shortcomings in M&E implementation were linked to early project delays, which meant that reporting on progress in the ISRs did not begin until March 2020. C. M&E utilization: High 44. M&E utilization was quite strong and helped to correct errors made during the design stage as well as adapt to challenges that arose during implementation. With support from the World Bank, the PIU closely tracked progress against the results framework and key changes to the M&E framework were carried out during the project restructuring (discussed in the Efficacy section above). Progress towards outcome and intermediate indicators were discussed regularly during supervision meetings with the World Bank and appropriate course corrections were made. While the MTR noted significant progress made towards final targets following project restructuring, it also highlighted the issue of sustainability and how key BSEIPH activities would continue to be financed following project closure. Furthermore, the communication trainings were designed to help BSEIPH staff better disseminate project objectives and results in order to increase awareness of disability among the general population. Data collected through the registry will inform implementation of future social protection projects in Haiti, including the ASPIRE project. Analytical work based on the registration data collected on persons with disabilities is ongoing and expected to be included in the upcoming Systematic Country Diagnostic for Haiti, which will inform future World Bank programming in the country. Overall, the BSEIPH is better equipped to engage with partners and continue to work towards inclusion of persons with disabilities. Environmental, Social and Fiduciary Compliance 45. The project was rated C for environmental and social safeguards. 22 No safeguards policies were triggered. Fiduciary compliance was sufficient with no issues reported beyond initial delays in contract execution for the reasons mentioned above. Starting March 2020 until closing, the PIU embedded in the BSEIPH included a procurement specialist and a financial management specialist. These individuals received regular support from the World Bank team and effectively managed all project related fiduciary aspects. 46. Compliance with the project’s Financial Management (FM) arrangements were maintained throughout project implementation ending with a Moderately Satisfactory FM rating and a Moderate FM risk rating. The disbursement ratio for the project reached 94 percent despite several exogenous factors affecting implementation, as mentioned above. Interim financial reports (IFRs) and financial audits were submitted as requested by the financing agreement and were accepted by the Bank. Some IFRs were delayed but it did not affect the capacity to provide timely and reliable provision of information required to manage and monitor the execution of project´s activities. The recruitment of a financial management specialist since the inception dedicated exclusively to the project and regular supervision missions from the Bank allowed the PIU to maintain adequate capacities to timely address shortcomings that arose during implementation. 22 The ‘C‘ rating indicates that at appraisal, the project was not expected to have any significant, large scale, or irreversible adverse environmental or social impacts. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 24 of 38 Bank Performance Overall Rating: Moderately Satisfactory A. Quality at Entry: Moderately Unsatisfactory 47. During project initiation in 2013, the Bank completed necessary due diligence to identify and prepare an innovative approach to meet the needs of the disabled population in Haiti. The project design was strategically relevant, technically sound, and had potential to contribute significantly to social and economic inclusion outcomes for persons with disabilities. However, the long delay between initiation and approval led to challenges which were not adequately addressed by the World Bank and contributed to insufficient quality at entry. While the use of trust funds was approved by the PHRD management team, shifting priorities within the GoH and the World Bank Country Management Unit (CMU) in Haiti caused the project to sit idle for several years. By the time it was taken up by all stakeholders in 2018, the PHRD management team was not willing to approve major changes to the project (such as reduced targets, elimination of activities or objectives, or increased project timeframe) as this would lead to further delays. As a result, the project proceeded according to the design developed in 2013 despite significant changes in the operating context as discussed in previous sections. While many of these factors were beyond World Bank control, relevant Bank teams could have more effectively managed key stakeholders (notably the PHRD management team), increased engagement with the GoH during preparation, and proactively identified and resolved issues during the period while the grant was idle, in order to lessen the impacts of certain setbacks that arose during the early years of implementation and were ultimately corrected during the June 2020 restructuring. As discussed, M&E design was also inadequate which led to reduced quality at entry. B. Quality of Supervision: Satisfactory 48. Despite the challenges at entry, World Bank supervision over the course of implementation was proactive and effective and ensured the achievement of key development outcomes. There was a strong focus on development impact, particularly with regards to institutional strengthening of the BSEIPH and institutional coordination between BSEIPH and MAST. The Bank team was instrumental in building buy-in for the SIMAST as the central social registry in Haiti and supporting the BSEIPH to register persons with disabilities and improve targeting of social services. Institutional and policy support encouraged the establishment of the Sectoral Tables for Social Protection (Tables Sectorielles de Protection Sociale, or TSPS), which serve as the key coordination mechanism within the social protection sector, as well as the adoption of the PNPPS in 2020. Bank supervision included support to donor coordination in the social protection sector, including the United Nations’ World Food Program (WFP), European Union (EU), UNICEF, and Inter-American Development Bank (IDB), particularly for expansion of the SIMAST and development of the PNPPS, as well as to streamline activities between the BSEIPH and MAST. The continued focus on development impact is expected to contribute to supporting adequate transition arrangements and operation of activities after grant closing. 49. The PIU and key BSEIPH stakeholders reported high levels of satisfaction with Bank supervision. This was noted in the last year of the project when implementation accelerated significantly. This built momentum for continuation of activities after project closing, including efforts by the BSEIPH to identify new funding sources. At the same time, early delays in startup, recruitment of PIU, and difficulties in procurement can be partially attributed to less than adequate supervision by the World Bank and difficulties in managing client relationships to ensure smooth rollout of project activities. Challenges in the Haitian context, notably low levels of institutional resources and capacity, highlight the need for more intensive supervisory efforts by World Bank teams beyond what is seen in other country contexts. Once these early challenges in supervision were identified, the Bank team increased support provided to the PIU through missions The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 25 of 38 (including frequent virtual missions due to COVID-19 restrictions), increased availability of key staff including task team leaders, and an increased staff and consultant presence in Haiti when circumstances allowed. Furthermore, while no ISRs were completed prior to March 2020, the quality of Bank performance reporting improved significantly thereafter and following the project restructuring. Risk to Development Outcome 50. Given the project’s focus on institutional strengthening and active stakeholder engagement, the risk to development outcome is deemed to be moderate. Institutional support to the BSEIPH was a core tenet of project design and was prioritized throughout project implementation. The PNPPS includes a vision for integrating persons with disabilities into all spheres of life and lays out specific mechanisms to achieve these objectives. The expansion of the SIMAST remains a central priority of the GoH and development partners, and the disability module will allow persons with disabilities to continue to be registered and reached by social services and programs. Many persons with disabilities were able to gain meaningful employment following the training activities and with support from the job service, which suggests this can be a model to promote employment alongside awareness raising with employers of the 2012 Disability Law. The BSEIPH is seeking out additional funding sources to continue the training and registration activities and the ASPIRE project will support related activities. At the same time, Haiti remains an extremely difficult and volatile operating context and BSEIPH struggles to continue activities without adequate staffing and funding. Recent events have further deteriorated the operating context, including the presidential assassination on July 7, 2021, the magnitude 7.2 earthquake in southwest Haiti on August 14, 2021 alongside a general deterioration in the security situation. In the wake of these events, coordination and investment in adaptive social protection have increased somewhat to reach vulnerable populations. However, continued political instability, institutional turnover, and the high threat of natural disasters pose large risks to development outcomes. The strengthening of policy and program infrastructure in the social protection sector would help maintain achievement of outcomes over the longer term. V. LESSONS LEARNED AND RECOMMENDATIONS 51. Fragile operating contexts such as Haiti require intensive engagement and supervision to ensure project success with dedicated staff resources and time. Effective implementation depended on adaptive management and strong coordination between the World Bank team, the PIU, BSEIPH and other GoH actors. While the PIU was located in the central BSEIPH office, projects that include department-level or further decentralized activities should have a stronger permanent field presence. A focus on institutional strengthening is essential, including building capacity at decentralized levels, engaging regularly with government representatives and donor partners, and building the evidence base for effective interventions. Specialized tools can be used to ensure remote supervision, such as through the World Bank’s Geo-Enabled Monitoring and Supervision (GEMS) approach or third-party monitoring when mobility is restricted for PIU staff to travel to the field, as was the case throughout implementation. 52. More attention is required during project design and in defining the project theory of change to ensure that proposed activities are clearly linked to expected outcomes and with assumptions and risks defined. Many issues arose due to an overly ambitious project design in contrast to a difficult operating context, limited availability of social services and employment opportunities, and high levels of vulnerability and exclusion of the target population. Developing realistic expectations and assumptions during the project design phase needs to be accompanied by adaptive management practices and flexibility in project implementation to respond to disruptive events or changing circumstances. For example, given the volatile macroeconomic context which led to exchange rate fluctuations and difficulties in negotiating key contracts, projects might consider building in arrangements during project preparation with The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 26 of 38 the support of procurement teams, or at least discuss with key actors during appraisal how these issues might be handled if they do arise. Adequate budgets should also be made available for World Bank teams to ensure preparation and closing of RETFs, for instance to complete ICRs. The level of project ambition can be seen in project components that aimed to address both social protection and labor gaps in Haiti. This points to the need to fully flesh out project results chains during preparation to ensure that all project components can be managed during implementation and adapted in the case of operational roadblocks or institutional transition. 53. Many lessons were learned related to the technical implementation, which can inform ways to implement a disability inclusive surveys in Haiti as well as other contexts. Beyond the utilization of a standardized disability survey module (the extended set of questions developed by the Washington Group), the PIU developed a methodology to ensure the data collection methodology would be adequate. This adaptation included a questionnaire piloting workshop with people with disabilities with different types of disabilities, a specific training on the issue of disability for data collection staff, close collaboration with Organizations for People with Disabilities (ODPs) and local authorities to include them in data collection staff (representing 40 percent of facilitators and enumerators), the inclusion of sign language interpreters, a communication campaign through various medias, and the targeting of specific areas in which persons with disabilities lived (camps in the West department for example). 54. From an operational standpoint, projects focusing on vulnerable populations, such as persons with disabilities, requires additional time to identify qualified service providers. Procurement processes were particularly complex due to a lack of providers, and often required lengthy negotiations to identify options for adjustments to implement disability- inclusive activities. Consultations can build on the expertise of specialized organizations (such as Humanity and Inclusion in the case of Haiti) and support engagement with providers to assess the feasibility of service adjustments (such as designing a disability friendly training curriculum or data collection activity with sign language interpreters). 55. The importance of needs assessments and sectoral studies to define and refine activities in data-poor environments cannot be overlooked. The combination of weak evidence at entry and a low-capacity operating environment exacerbated the challenges the project faced early on, and a full accounting of these issues would indicate the need for longer implementation timeframes in Haiti. The interconnected yet distinct social protection and labor components required efforts to build the evidence base to ensure effective and relevant interventions. For Component 1, this involved developing appropriate survey tools and methodologies for data collection, including the disability module to the SIMAST, and accessing a hard-to-reach population while ensuring institutional compatibility between the BSEIPH and the MAST. For Component 2, this involved conducting an in-depth study on potential employment opportunities for persons with disabilities to inform the training design and refinements to make the job placement service more effective. 56. Social protection and labor programs should be matched with complementary investments in promoting availability of social services, as well as employment opportunities. The link between registration of vulnerable groups and access to social services is not automatic in any operating context. Proactive management and engagement with partners can help to ensure that social services are available, and contingency plans should be established in case service provision deteriorates, as was seen in the Kore Fanmi network. Furthermore, with regards to employment, the potential of self-employment in Haiti and in other similar contexts should not be overlooked. Beneficiaries of skills trainings were able to integrate more easily into the labor market by developing their own income-generating activities, as opposed to gaining employment with existing public or private sector employers. Providing technical or financial support alongside trainings can facilitate success in self-employment or the creation of small businesses, as well as support job retention and sustainability of employment outcomes for persons with disabilities. The short implementation timeframe led to a The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 27 of 38 reduction in the scope of employment-related objectives between the original and revised PDO, however this was reflective of project-specific constraints rather than relevance or feasibility if adequate time and resources are provided. 57. Finally, the project contributes several lessons learned in terms of sustainability, particularly given the small project size and short implementation timeframe. A project of this complexity and in a challenging context such as Haiti needed a longer implementation timeframe, not only to carry out project activities as designed but also to ensure sustainability of outcomes, as well as monitoring of the employment situation of the beneficiaries. Given the short project timeframe and capacity constraints at the BSEIPH, there was not sufficient time prior to project closure to follow-up with participants in the trainings or job placement service to assess job placement performance more thoroughly, including types of jobs (such as short- or long-term employment, or common sectors). At the same time, the project has helped advance the disability and social protection agendas in Haiti, including through the disability module linked to the SIMAST registry, which will be scaled up through the World Bank funded ASPIRE project and earthquake response activities. Additional efforts and resources will be required to expand coverage of the social registry and ensure greater inclusion of persons with disabilities over time, including streamlined efforts across government and partners to integrate the disability module into future SIMAST surveys. Institutional strengthening of the BSEIPH and MAST at national and departmental levels to establish the foundations of an adaptive social safety net should be priorities for continued engagement in the social protection sector, as guided by the PNPPS, and sustainability of project-supported outcomes. Given the high levels of political instability and disaster risk in Haiti, continued investment and engagement between key government actors and development partners is essential to address the needs of the poorest and most vulnerable. . . The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 28 of 38 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To increase Persons with Disabilities access to employment services Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Number of PwD who participate in training and obtain certification Number 0.00 0.00 200.00 157.00 05-May-2020 05-Oct-2018 30-Jun-2021 30-Jun-2021 Comments (achievements against targets): 157 Persons with Disabilities out of the target of 200 participated in training and obtained certification for an achievement rate of 78.5 percent. As of the end of the project, an additional 60 individuals were actively participating in an IT training and expected to receive certification which would bring the total number of individuals trained to 217. Training was conducted in skill areas identified as ‘high-demand’ based on an employment study, and 74 individuals had gained employment (including self-employment) within one month of completing the training reflecting an employment rate of 47 percent. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 29 of 38 Number of PwD who participate in the job placement service Number 0.00 1000.00 500.00 679.00 05-Oct-2018 05-Oct-2018 30-Jun-2021 30-Jun-2021 Percentage of PwD who participate in the job placement service who are female Percentage 0.00 50.00 50.00 44.77 05-Oct-2018 05-Oct-2018 30-Jun-2021 30-Jun-2021 Comments (achievements against targets): The revised target of 500 Persons with Disabilities was surpassed as 679 Persons with Disabilities participated in the BSEIPH job placement service for an achievement rate of 136 percent. Increasing awareness of and participation in the job placement service was a major success of the project. However, 42.3 percent of participating Persons with Disabilities in the job placement service were female, falling slightly short of the 50 percent target. Additional efforts can be made to conduct outreach to female Persons with Disabilities and ensure they participate equally. Objective/Outcome: To identify persons with disabilities (PwD) Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion PwD documented in the national registry Number 0.00 100000.00 50000.00 53784.00 05-Oct-2018 05-Oct-2018 30-Jun-2021 30-Jun-2021 Comments (achievements against targets): The revised target of 50,000 Persons with Disabilities documented in the national registry was exceeded as 53,453 individuals were registered for an achievement rate of 106.9 percent. Not only was the target surpassed, but the development of disability survey module will allow for detailed information on disability to be included in the national social registry SIMAST in the future. The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 30 of 38 A.2 Intermediate Results Indicators Component: Piloting and promoting a model to increase labor market access for Persons with Disabilities Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Human resources trained on employment opportunities for persons with disabilities Number 0.00 1000.00 30.00 40.00 05-Oct-2018 05-Oct-2018 30-Jun-2021 30-Jun-2021 Comments (achievements against targets): 40 human resource representatives were trained on employment opportunities for Persons with Disabilities compared to the target of 30 for an achievement rate of 133.3 percent. The success was due to a proactive outreach and communications strategy to identify and engage with employers and human resource departments. The project helped to raise awareness among employers of the legal framework that sets a quota for employment of Persons with Disabilities, but also how organizations and businesses can better leverage the skills that Persons with Disabilities offer. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Employers included in the job placement service database Number 0.00 100.00 30.00 40.00 05-Oct-2018 05-Oct-2018 30-Jun-2021 30-Jun-2021 Comments (achievements against targets): The proactive employer outreach strategy is also reflected as 40 employers were included in the job placement service database compared to the target of 30 for an achievement rate of 133.3 percent. The expansion of the job placement service on the side of employers as well as Persons with Disabilities enhances the potential for employers to identify Persons with Disabilities with skills or training certification in areas of need. Note: Original Cumulative Target Values: YR1: N/A. YR2: 30; YR3: 60; YR4: 100. Original End Target: 100. The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 31 of 38 Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Awareness raising events implemented Number 0.00 5.00 3.00 7.00 05-Oct-2018 05-Oct-2018 30-Jun-2021 30-Jun-2021 Comments (achievements against targets): Despite the challenges of COVID-19 and political unrest, project staff were able to implement four awareness raising events compared to the target of three events for an achievement rate of 133.3 percent. Participation in the awareness raising events was high with media coverage. The awareness raising events constituted a core aspect of the project’s communications strategy to raise awareness regarding disability in Haiti, but also helped to link Persons with Disabilities with potential employers. Note: Original Cumulative Target Values: YR1: 1; YR2: 2; YR3: 3; YR4: 5. Original End Target: 5. Component: Project Management M&E and knowledge dissemination Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Number of BSEIPH staff and consultants receiving communication training Number 0.00 10.00 10.00 10.00 05-Oct-2018 05-Oct-2018 30-Jun-2021 26-May-2021 Comments (achievements against targets): The communications strategy were a core component of project implementation and will be essential for BSEIPH to continue to raise awareness among Persons with Disabilities, employers, and the general population, and to expand the reach of their services. The target of 10 BSEIPH staff and consultants receiving communication training was ultimately reached with an achievement rate of 100 percent. This indicator aimed to measure the efforts to increase The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 32 of 38 BSEIPH’s capacity to conduct effective outreach and communication which is a key part of their mandate. Note: Original Cumulative Target Values: YR1: 10; YR2: n/a; YR3: n/a; YR4: n/a. Original End Target: 10. Indicator Name Unit of Measure Baseline Original Target Formally Revised Target Actual Achieved at Completion Awareness raising events implemented Number 0.00 5.00 3.00 7.00 05-Oct-2018 05-Oct-2018 30-Jun-2021 30-Jun-2021 Comments (achievements against targets): Despite the challenges of COVID-19 and political unrest, project staff were able to implement four awareness raising events compared to the target of three events for an achievement rate of 133.3 percent. Participation in the awareness raising events was high with media coverage. The awareness raising events constituted a core aspect of the project’s communications strategy to raise awareness regarding disability in Haiti, but also helped to link Persons with Disabilities with potential employers. Note: Original Cumulative Target Values: YR1: 1; YR2: 2; YR3: 3; YR4: 5. Original End Target: 5. The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 33 of 38 . . ANNEX 2. PROJECT COST BY COMPONENT Components Amount at Approval (US$M) Actual at Project Closing (US$M) Percentage of Approval (US$M) Registration of persons with disabilities 0.71 1.24 31 Piloting and promoting a model to increase labor market access for persons with disabilities 1.14 0.60 49 Project Management M&E and knowledge dissemination 0.45 0.45 19 Total 2.30 2.29 100.00 The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 34 of 38 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS (1) Did the project achieve its objectives from your point of view? 1. The general objective of the project has been to identify people with disabilities and improve their access to Social Services and Employment, and measuring the degree of achievement of the objectives refers us above all to looking at the impact indicators, the targets, and related measures. To facilitate the process, this summary table of the objectives, targets and results actually achieved is presented below. Table 3.1: Summary of Objectives, Targets, and Actual Results Achieved Goals Unit of measure Target Results persons with disabilities documented in the national registry Person 50,000.00 53,453 Number of persons with disabilities who participate in training and obtain certification Person 200.00 157 23 Number of persons with disabilities who participate in the job placement service Person 500.00 679 Percentage of persons with disabilities who participate in the job placement service who are female Person 50.00 44.77% Human resources trained on employment opportunities for persons with disabilities Person 30.00 40 Employers included in the job placement service database Business 30.00 40 Awareness raising events implemented Event 3.00 7 2. Taking into account the results presented in the previous table, we can conclude that, overall, the project was able to achieve or even exceed most of its objectives. Instead of 50,000 people with 23 Lot 2 of training in Computer Graphics and Web Design, Repair of cell phones, tablets and laptops is still in progress as of June 30, 2021 and has around 60 beneficiaries. PROJECT TO PROMOTE THE INCLUSION OF DISABLED PEOPLE IN SOCIAL PROTECTION AND EMPLOYMENT PROGRAMS PAASSEPH MINISTÈRE DE L’INTÉRIEUR ET DES COLLECTIVITÉS TERRITORIALES (MICT) UNITÉ DE COORDINATION DE PROJET (UCP) Bureau of the Secretary of State for the Integration of Persons with Disability Project Implementation Unit (PIU) The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 35 of 38 disabilities, 53,453 have been identified and registered and the data is available and ready to be integrated into the SIMAST database. This is the most complex activity of the project in terms of preparation, planning, and operation. Thus, the availability of these data represents today a leap forward for the BSEIPH and the Haitian government, which face a glaring lack of statistics on the population in general and on people with disabilities in particular. These data will definitely facilitate the targeting of people with disabilities within the framework of social protection and civil protection programs. They will allow BSEIPH to better understand the situation of its target group and to undertake actions contributing to the sustainable improvement of the living conditions of people with disabilities. Obviously, the impact will be limited as these data have not been collected nationally. However, this activity has nonetheless made it possible, through the lessons learned, to point the way forward. 3. The specific objective of the project, through its component 2, was to promote a model to increase access to the labor market for people with disabilities. For this component, we can conclude that the objective is achieved with a quantitative downside. Indeed, to start, a study on the promising sectors in terms of training and employment was carried out to determine the best options that would make it possible to achieve this objective, taking into account the constraints linked to the situation of the target group, the very short timeframe available for the implementation of activities, etc. From there, we could understand the importance of focusing on short trainings and subjects that are not commonly taught. 4. This choice made it possible to graduate approximately 157/200 disabled people with a post- training employment rate of 58.5 percent approximately one (1) month after graduation, while approximately 60 other disabled people are still in training as of June 30, 2021. In this sense, if from a quantitative point of view, there is still a margin to reach the target over time, from a qualitative point of view, we believe that the model developed is relevant and what has been achieved can provide information on the choices to be made for increasing the access of people with disabilities to the labor market. (2) What were the successes of the project? 5. Considering the particularly difficult context for implementing activities, marked by the Covid-19 pandemic, government instability, insecurity, kidnapping, conflict between armed gangs and episodes of all-out blocking (peyi-lòk), retaining all indicators measured, we can consider the project to be a success when we add up the cumulative total for all indicators, which gives a percentage of 122 percent. 6. Taken individually, the survey on the identification and registration of persons with disabilities can be considered a success in so far as we manage both to exceed the target but also to issue a receipt to each person registered in the database. This receipt can therefore serve as a precursor to a national disability card. It is equally important to note that this process could have been carried out in partnership with representatives of organizations of / for people with disabilities. 7. We can also consider the employment rate of 58.5 percent for the first 127 training program beneficiaries to be a success. An important element to consider is the fact that we have been able to accommodate virtually all types of disabilities except people with intellectual disabilities. The project was even able to help around thirty (30) people with visual impairments to complete the program and graduate. The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 36 of 38 Table 3.2: Breakdown for final courses OPTION CLASS TOTAL WORKFORCE WOMEN MEN TOTAL Amount % Amount % Cooking and Pastry- making 19 15 79% 4 21% 12.1019% Motorcycle Repair 32 0 0 32 100% 20.3821% Agricultural Technology 22 12 55% 10 45% 14.0127% Processing of Agricultural Products 54 36 67% 18 33% 34.3949% Massage therapy 30 19 63% 11 36% 19.1082% Total 157 82 52.22% 75 47.77% 100% 8. In terms of events and awareness, the project has been a real success. This is because this objective was achieved by over 200 percent over the target, and 7 events to raise awareness and promote the talents of young disabled people which have taken place, which is 4 more than the target. Despite the difficult context, we were able to reach 4 departments with our events, namely, the North (Forum on Employment and Professional Orientation), the West (Employment Fair and breakfast debate with companies), the South-East (graduation and fair) and the South (graduation and fair). 9. The organization of events and activities has enabled directors and executives of institutions to have a more accurate perception of the potential and capacities of people with disabilities, of their responsibilities in relation to the law of March 2012 and of the support available to them facilitating integration of the quota of disabled people in their institutions. On the basis of the feedback received, one can expect a better participation of institutions, all sectors included, in the process of creating suitable opportunities for people with disabilities. (3) What activities did not go well and why? 10. Almost all the activities went well but we still want to report that: For component 1, while collection and availability of data was a great success, a crucial element that does not directly depend on the project team could not be completed, namely the creation of the access portal in the national social security registry (SIMAST) for the BSEIPH. The reason for this is that SIMAST made this activity conditional on the availability of the survey results for identification and registration of persons with disabilities. Since the data will become available only at the very end of the project, this activity therefore remains at the discretion of the SIMAST team and is not subject to encouragement or incentivizing from the project team for its implementation. 11. For component 2, training in Cell Phone, Tablet and Laptop Repair, and Computer Graphics and Web Design could not be completed due to the resurgence of Covid-19 cases across the country. This training will therefore be continued as part of another project financed by the Bank, knowing that the activities were approximately 70 percent completed as of June 30, 2021. The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 37 of 38 12. We must also mention the fact that we launched a procurement process that did not come to fruition, namely the selection of a provider for training and socioeconomic integration (job placement and / or the development of income-generating activities) for thirty (30) disabled people in the fields of sewing, industrial sewing machine operation, bead work, and sequinning. 13. As part of this Request for Proposals (RFP), one (1) of the providers acknowledged receipt of correspondence and expressed interest in submitting a proposal. By the deadline for receipt of bids, which was set for December 21, 2020 at 11:59 p.m. Haiti time, for electronic transmission, the UCP-BSEIPH had not received any bids. 14. Based on this observation, the UCP-BEIPH made the following assumptions: a. Contract content and disability issue: Providers may not have the means to provide services according to the needs expressed in the terms of reference and the accessibility of services to address the disability issue. b. Institutional capacity and availability: Only one provider acknowledged receipt of the invitation to submit a proposal. Some institutions may not see the emails sent to them. The holiday season and the closure of administration may have made it more difficult for institutions to prepare a proposal. c. Mistrust: Providers may be reluctant to engage with a state entity even with funding from a multilateral bank, especially in the particularly difficult health and political context in Haiti. (4) What was the performance of the Bank in this project? 15. The Bank provided exceptional performance in terms of support and monitoring under this project. The response time and treatment of emergencies, assistance in the decision-making process, and the flexibility of the Bank team greatly facilitated the work of the UCP-BSEIPH, the progress of activities and the achievement of the objectives of the project. (5) What are the lessons learned for the future? 16. Despite the difficult economic situation in Haiti, marked by socio-political unrest, the health crisis and significant volatility of the national currency against the dollar, the UCP-BSEIPH was able to achieve the majority of the objectives set through contracting partners essential to achieving expected results. It drew on the experience and lessons learned to tackle these unexpected crises with far-reaching consequences in the short and medium terms and to make progress in achieving the key indicators. This first experience turned out, however, to be enriching for the BSEIPH, which was able to draw useful lessons for the design of programs to promote the inclusion of people with disabilities in different spheres of national life. The World Bank Promoting Inclusion of PwDs in Social Protection and Employment Programs (P145468) Page 38 of 38 17. Among other lessons learned that we believe are important to relate, we can mention that: • Most service providers, despite their seniority and experience, often do not have enough organizational and managerial skills to effectively conduct activities, especially when it comes to working with a specific group; • It is easier to achieve results when the project team is significantly represented in the field and maintains a high level of connection to the different categories of stakeholders; • Training beneficiaries can be integrated more easily into the labor market through the implementation of income-generating activities rather than as an employee of a business; • Technical and financial support for training beneficiaries can greatly facilitate them in developing their economic and financial independence through self-employment; • Agility, a high degree of responsiveness and shared points of view between the Bank and Project teams are key to the success of activities, especially in a context of crises and violence.