Pwomosyon Enklizyon Moun Andikape nan Pwogram Pwoteksyon Sosyal ak Travay

Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005641 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF 18710 ON A SMALL GRANT IN THE AMOUNT OF USD 2,294,320 MILLION FROM THE POLICY AND HUMAN RESOURCES DEVELOPMENT (PHRD) FUND TO THE REPUBLIC OF HAITI FOR PROMOTING INCLUSION OF PERSONS WITH DISABILITIES S IN SOCIAL PROTECTION AND EMPLOYMENT PROGRAMS (P145468) December 17 , 2021 Social Protection & Jobs Global Practice Latin America And Caribbean Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Regional Vice President : Carlos Felipe Jaramillo Country Director : Lilia Burunciuc Regional Director : Luis Benveniste Practice Manager: Pablo Gottret Task Team Leader (s) : Caroline Tassot, Briana N. Wilson ICR Main Contributor : Emma Schwirck Willenborg CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2021 ) Currency Unit = Haitian Gourde ( HTG ) US$ 1 = HTG 93,50 ABBREVIATIONS AND ACRONYMS BSEIPH Office of the Secretary of State for the Integration of persons with disabilities ( Bureau du Secrétaire d’État à l’Intégration des Personnes Handicapées ) CPF Country Partnership Framework FAES Economic and Social Assistance Fund ( Fonds d’Assistance Économique et Sociale ) GDP Gross Domestic Product GEMS Geo - Enabled Monitoring and Supervision GoH Government of Haiti ICR Implementation Completion and Results Report IRI Intermediate Results Indicator ISR Implementation Status Report M&E Monitoring and evaluation MAST Ministry of Social Affairs and Labor ( Ministère des Affaires Sociales et du Travail ) MIS Management Inf ormation System MTR Midterm Review NGO Non - government Organization OAS Organization of American States OPD Organizations of persons with disabilities PNPPS National Policy for Social Protection and Promotion ( Politique Nationale de la Protection et Promotion Sociale) PPIPDSPEP Project to Promote the Inclusion of People with Disabilities in Social Protection and Employment Programs PDI Project Development Indicator PDO Project Development Objective PHRD Policy and Human Resources Development Trust Fund PIU Project Implementation Unit PLR Performance and Learning Review RETF Recipient - Executed Trust Fund SIMAST Social Registry of the Ministry of Social Affairs and Labor ( Système d’Information du MAST) TSPS Social Protection Sectoral Table ( Tables Sectorielles de Protection Sociale ) TTL Task Team Leader UNICEF United Nations’ Children’s Fund USAID United States Agency for International Development WFP United Nations’ World Food Program TABLE OF CONTENTS DATA SHEET ................................ ................................ ....... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ................................ ....................... 4 II. OUTCOME ................................ ................................ ................................ .................... 12 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 20 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 22 V. LESSONS LEARNED AND RECOMMEND ATIONS ................................ .............................. 25 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ................................ ........................... 28 ANNEX 2. PROJECT COST BY COMPONENT ................................ ................................ ........... 33 ANNEX 3. RECIPIENT, CO - FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 34 The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 1 of 38 DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P145468 Promoting Inclusion of Persons with Disabilities in Social Protection and Employment Programs Country Financing Instrument Haiti Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Ministry of Economy and Finance Office of the Secretary of State for the Integration of the Persons with Disabilities Project Development Objective (PDO) Original PDO The development objective is to identify persons with disabilities (PwD) and increase their access to social services and to employment opportunities. Revised PDO The development objective is to identify persons with disabilities (PwD) and increase their access to employment services The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 2 of 38 FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF - 18710 2,294,320 2,294,320 2,152,776 Total 2,294,320 2,294,320 2,152,776 Total Project Cost 2,294,320 2,294,320 2,152,776 KEY DATES Approval Effectiveness Original Closing Actual Closing 28 - Jun - 2018 05 - Oct - 2018 30 - Jun - 2020 30 - Jun - 2021 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed ( US$M) Key Revisions 28 - Jun - 2020 0.26 Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Implementation Schedule Other Change(s) 09 - Sep - 2020 0.32 KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Moderately Satisfactory Substantial The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 3 of 38 RATINGS OF PROJECT PERFORMANCE IN ISRs No. Date ISR Archived DO Rating IP Rating Actual Disbursements (US$M) 01 18 - Mar - 2020 Moderately Unsatisfactory Moderately Unsatisfactory 0.26 02 17 - Dec - 2020 Moderately Satisfactory Moderately Satisfactory 0.57 03 17 - May - 2021 Moderately Satisfactory Moderately Satisfactory 1.31 04 07 - Jun - 2021 Satisfactory Satisfactory 1.80 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Carlos Felipe Jaramillo Country Director: Anabela Abreu Lilia Burunciuc Director: Michal J. Rutkowski Luis Benveniste Practice Manager: Pablo Gottret Pablo Gottret Task Team Leader(s): Snjezana Plevko, Aissatou Maisha Dicko Caroline Anne Isabelle Tassot, Briana N. Wilson ICR Contributing Author: Emma Schwirck Willenborg The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 4 of 38 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context at Appraisal A. Country Context 1. At appraisal, Haiti’s population faced high levels of poverty, low levels of human development and heightened risk to political crises, natural disasters, and other shocks. The devastating January 12, 2010 earthquake that struck just outside of Port - au - Pri nce killed 220,000 people and displaced 1.5 million. As of 2012, nearly 60 percent of Haiti’s population was living below the national poverty line and 24 percent (2.5 million) was extremely poor. 1 In 2015, Haiti was ranked 168 out of 187 countries accord ing to the United Nations Human Development Index based on social indicators such as literacy, life expectancy, infant and maternal mortality, and child malnutrition. Recurrent shocks exacerbated Haiti’s underlying socioeconomic drivers of poverty and vuln erability, threatened livelihoods, hampered service delivery, and worsened human development outcomes. In October 2016, category four Hurricane Matthew hit Haiti’s south - west coast, affecting over 2 million people and reconstruction needs were assessed at 25 percent of GDP. 2 2. Persons with disabilities were among the most at - risk groups in Haiti to both poverty and shocks and faced low human development outcomes . At the same time, minimal reliable nationally representative estimates of disability prevalence existed. According to Handicap International a nd the National Network of Associations for the Integration of Persons with Disabilities , 800,000 individuals had some form of disability in 2003. The latest census conducted in 2003 estimated the prevalence of disability at 1.6 percent of the population, based on questions about severe impairments. 3 One quarter of the 280,000 persons injured during the 2010 earthquake were severe ly injur ed and seven percent had at least one permanent disability due to physical or mental factors. 4 Furthermore, public infr astructure was severely damaged which rendered the environment less accessible to individuals with disabilities and disrupted informal, community - based networks of support. As of 2014, approximately 172,000 people, including at least 900 persons with disab ilities , were still living in camps. 5 3. A lack of national - level coordination and limited evidence base hampered the Government of Haiti’s (GoH) ability to identify and reach persons with disabilities with services and support . Although the GoH did prioritize the issue of disability, institution al capacity was low and donor interventions were often fragmented. Some resources for persons with disabilities were available, however a lack of data meant that needs were not well understood and often overlooked by public agencies. Anecdotal evidence was consistent with other low - income countries, as persons with disabilities and their households were more likely to be poor when compared to other households, and persons with disabilities themselves were more likely to live in extreme poverty. Unemployment of persons with disabilities also tends to be higher due to barriers to entering the workplace, such as competition for limited job opportunities, lack of accessibility, and limited access to education or training programs. In Haiti, there was a need to s trengthen institutional 1 World Bank World Development Indicators. 2 Government of Haiti, World Bank, and Inter - American Development Bank, 2016. Rapid Damage and Loss Assessment of Hurricane Matthew. 3 World Bank, 2011. Report on Disability. Severe disability is defined as “the equivalent of disability inferred for conditi ons such as quadriplegia, severe depression, or blindness.” 4 London School of Hygiene and Tropical Medicine and Handicap International, 2012. Representation and evaluation of disability in Haiti. 5 Temporary camps were established for individuals displaced due to the 2010 earthquake. Source: Handicap International, 2014. Haiti: Situation update – Jan. 2014. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 5 of 38 capacity, create a system to identify eligible beneficiaries, and systemize interventions to improve efficiency and ensure existing resources reach those most in need. B. Sectoral and Institutional Context 4. In 2007, the Office of the Secretary of State for the Integration of Persons with Disabilities (BSEIPH) was established by Presidential decree under the Ministry of Social Affairs and Labor (MAST) . T herefore, t he BSEIPH ha d its own mandate but contribute d to overarching objectives for social protection coverage in Haiti as defined by the MAST . BSEIPH’s objectives were to implement a national policy to support the integration of persons with disabilities , improve the ir living conditions, raise awareness among the public about disability, strengthen the capacity of organizations working in the fie ld of disability, and put in place a legal framework for the protection of persons with disabilities . At appraisal, the Secretary of State of the BSEIPH was a long - time advocate regarding issues of disabilit y and had support from the President. While the a gency lacked resources, institutional capacity, and had a limited presence outside of Port - au - Prince, the BSEIPH had taken initial steps to create a job placement service to support employment opportunities for persons with disabilities . A full - time staff member had been hired to develop employment services, organize job fairs, and establish a website to better link persons with disabilities and prospective employers. However, capacity gaps persisted, including limited outreach, poor coordination with other social ministries, and low capacity to monitor, evaluate, and use data on persons with disabilities to plan and target potential benefits and programs. 5. Following the 2010 earthquake, Haiti’s Parliament enacted a law that required improved social and econ omic inclusion of persons with disabilities . Schools and public buildings were required to be accessible and two percent of jobs in companies with more than 20 employees had to be reserved for persons with disabilities . The textile industry had also agreed to employ an unspecified number of persons with disabilities . In March 2012, Haiti approved additional legislation to integrate persons with disabilities into public programs and services. In October 2012, the President and the Prime Minister signed a government declaration to mainstream the disability agenda across ministries, demonstrating support for the disability agenda by the highest level of the Gov ernment. However, a lack of resources, technical expertise and coordination, including to identify and register persons with disabilities and link them with services, were preventing these high - level objectives from being met. Employers were also largely u naware of issues of accessibility and the employment potential of workers with disabilities. 6. At appraisal, several non - state actors were involved in the field of disability in Haiti. However, uncoordinated and unregulated service provision contributed to significant variation in the quality and reach of services provided. At the time, the Organization of American States (OAS) supported legal frameworks for the inclusion of persons with disabilities ; the United States Agency for International Development (U SAID) was strengthening legal frameworks for disability and supporting rehabilitation and reintegration of persons with disabilities ; and several NGOs (e.g., Christian Blind Mission, Humanity and Inclusion ) were providing direct services to persons with di sabilities . No agency was developing a registry or systematically providing job support to persons with disabilities , and the government did not have a strong role in setting policy and coordinating assistance from donors, development organizations, and pr ivate sector providers working in the disability space. However, t he national social registry SIMAST, managed by the MAST, was established in 2013 with household - level demographic and socioeconomic information to facilitate targeting of social assistance p rograms in Haiti. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 6 of 38 Project Development Objectives (PDOs) 7. The Project Development Objective was to identify persons with disabilities and increase their access to social services and to employment opportunities. 8. Three priority actions were identified at the time of appraisal to better assess the needs of persons with disabilities and reach them with adequate services and support . Thes e included: (1) establish a registry of persons with disabilities to assess the needs of persons with disabilities and support more effective planning and targeting of social programs and services; (2) strengthen government institutions to foster an effective design and implementation of disability policies and programs; (3) improve employment access and retention to economically empower persons with disabilities , as well as their household members and the communities where they live. 9. The operation aimed to support the GoH’s efforts to identify and register persons with disabilities and support their access to social services and employment opportunities. A mapping of the original Results Chain underlying the original operation is shown in Figure 1. Component 1 (registrat ion of persons with disabilities) s upport ed the BSEIPH to develop a registration system for persons with disabilities to identify their needs and improve the targeting of social services. T he registration system for persons with disabilities was designed to link to the national social registry SIMAST , which is managed by the MAST and used for the targeting of social protection programs . Establishing a context - adapted and accessible system would allow for the continual registration and updating of information on persons with disabilities. Institutional support to the BSEIPH at cent ral and departmental levels would ensure staff have the capacity to manage the registry. Component 1 also included the identification and registration of persons with disabilities in the system to support more accurate targeting of social services and support based on identified needs , and enhance access to health, education, social protection services. Component 2 ( p iloting and promoting a model to increase labor market access for persons with disabilitie s ) consisted of support to the existing job placement service at BSEIPH through trainings, counseling, fairs, and the development of a management information system (MIS) for the job placement database. These activities aimed to support the institutional a nd staff capacity at the BSEIPH to support job - seeking persons with disabilities, while also supporting persons with disabilities to gain the skills and connections with employers to better link them to the labor market. This component also included awareness - raising efforts with employers to promote awareness and enhance employment of person with disabilities. These efforts would in turn increase access of persons with disabilities to employment opportunities. 10. Project objectives and approach were influenced by the source of financing as well as other planned World Bank projects. Financing was from the Policy and Human Resources Deve lopment (PHRD) Trust Fund , which identified two priority areas: (1) addressing the capacity building needs of government organizations ; and (2) supporting the introduction of scalable methods, led by government or other development actors, to improve the well - being of persons with disabilities. Furthermore, the project was designed as complementary to the IDA - financed project, Improving M aternal and Child Health and Nutrition through Integrated Social Services (PASMISSI, P123706). The original PASMISSI design included a component to increase access of poor and vulnerable households to health and social protection services by scaling up the Kore Fanmi network of community agents and to support the GoH to develop a national social registry (which would later become integrated into the SIMAST). Therefore, the PHRD project aimed to complement these activities by promoting the inclusion of perso ns with disabilities in the registry and support their access to health and nutrition services . The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 7 of 38 Key Expected Outcomes and Outcome Indicators 11. PDO achievement was to be captured by the following PDO Indicators (PDIs) : • PDI 1 – Persons with disabilities documented in the national registry (number) • PDI 2 – BSEIPH offices that receive capacity building in the registry The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 8 of 38 12. Achievement of intermediate outcomes was captured by the following Intermediate Results Indicators (IRIs) . For the purposes of this report, IRIs are used as proxies for PDIs in certain outcome areas where a PDI is not defined. • IRI 1 – Human resources trained to identify persons with disabilities (number) • IRI 2 – P ersons with disabilities who participate in the job placement service (number) • IRI 3 – P ersons with disabilities who participate in the job placement service who are female (percentage) • IRI 4 – Employers included in the job placement service database (number) • IRI 5 – BSEIPH staff and consultants receiving communication training (number) • IRI 6 – Awareness raising events implemented (number) Components 13. Component 1: Registration of persons with disabilities ( US$709,000 , equivalent to 30.9 percent of total grant amount). The objective of this component was to provide reliable data on persons with disabilities in Haiti. The component supported the development and application of a context - adapted, disability - specific survey alongside communication efforts to raise awareness of the registration campaign. The project aimed to register 100,000 persons with disabilities in the national social registry over a three - year period . Geographic coverage of the survey was partial and inc luded metropolitan Port - au - Prince as well as 50 communes with legacy social registry data to allow for an initial identification of households. Activities were to be carried out in coordination with key social ministries and other public agencies. The two main sub - components included the development of a registration system for persons with disabilities and identification and registration of pers ons with disabilities . 14. Component 2: Piloting and promoting a model to increase labor market access for persons with disabilities ( US$1,138,700 , equivalent to 49.6 percent of total grant amount). This component supported the job placement service for perso ns with disabilities , a database operated by the BSEIPH that link s persons with disabilities to job opportunities in the formal sector as well as skills training, counseling, and job fairs. Both persons with disabilities and employers are registered in the job placement service to facilitate the job search and recruitment efforts. Under the law for the integration of persons with disabilities in Haiti , at least two percent of companies’ positions were required to be fille d by persons with disabilities . This component aimed to help persons with disabilities gain employment opportunities and companies to comply with existing regulations. Additionally, all persons with disabilities using the service were advised on available job offers identified through consultation and meetings with local employers. This sub - component aimed to train employers to raise their awareness o f disability and help them identify actions to take to increase employment of persons with disabilities . 15. Component 3: Project management, monitoring & evaluation, and knowledge dissemination ( US$446,620, equivalent to 19.5 percent of total grant amount). This component aimed to strengthen the BSEIPH’s capacity for effective outreach and communi cation, as well as registry management and quality control. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 9 of 38 Significant C hanges D uring I mplementation A. Restructuring 16. The project underwent one Level 2 restructuring in June 2020. The restructuring aimed to address exogenous disruptions such as repeated political unrest and the COVID - 19 pandemic and sector - specific constraints including limited availability of social services and employment opportunities for persons with disabilities . The main changes included: (1) Extension of the project closing date from June 30 , 2020 to June 30 , 2021; (2) Revision of the PDO and project title to align with revised timeframe and constraints imposed by the COVID - 19 pandemic; (3) Revision of the Results Framework; (4) Revision of the component funds allocation; and (5) Revision of the disbursement category allocation. B. Revised PDO and Project Title 17. The operation’s PDO, closing date and project title were modified during the restructuring in June 2020 . The original PDO to ‘ identify persons with disabilities and increase their access to social services and employment opportunities’ was revised to ‘identify persons with disabilities and increase their access to employment services . ’ The project closing date was extended by one year from June 30, 2020 to June 3 0, 2021 and the project title ‘Improving Access to Social Services & Employment Opportunities for P ersons with D isabilities ’ was revised to ‘Promoting Inclusion of P ersons with D isabilities in Social Protection and Employment Programs . ’ These changes are s ummarized below and discussed in the sections that follow . • ‘ Access to social services’ was removed from the PDO for two reasons. First, social services were not available as anticipated during appraisal. The GoH - led Kore Fanmi initiative which aimed to improve service delivery through a network of community agents , including with financing from the World Bank PASMISSI project (see above), was not successfully scaled up and eventually discontinued. 6 T he original project design also did not link the objective of increased access to services with defined project components, activities, or indicators, which represented a flaw in the original project theory of change. • ‘Access to employment opportunities’ was changed to ‘employment services’ due to project specific constraints notably the overly ambitious project design and short implementation timeframe which was not sufficient to achieve final employment outcomes . A longer project timeframe with more robust demand - side support, such as apprenticeship programs or more intensive employer engagement, would have a greater potential to contribute to meaningful , long - term employment for persons with disabilities in Haiti . C. Revised Indicators 18. PDO indicators (PDIs) and Intermediate Results Indicators (IRIs) were modified to reflect the changes introduced under the project restructuring (see Table 1) . The main modifications at the PDI level included: i) the elimination of original PDI 2 due to the lack of staff and other resources in decentralized BSEIPH offices to perform registry management; ii) the inclusion of the original IRIs 2 and 3 as revised PDIs 2 and 3; and iii) the addition of PDI 4. These adjustments were made to ensure activities under Component 2 were reflected in th e results framework. Finally, targets were revised downward for certain indicators to correct for an overly ambitious project design alongside delays in implementation, including in the context of the COVID - 19 pandemic which limited in - person data collecti on and training activities . These changes are reflected in Table 1 below and described in more detail in the Efficacy section. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 10 of 38 Table 1. Changes in PDO Indicators (PDIs) and Intermediate Results Indicators (IRIs) Original PDO Indicators (PDI) Target Changes Revised PDIs Target PDI 1. persons with disabilities documented in the national registry (number) 100,000 Revised target PDI 1. persons with disabilities documented in the national registry (number) 50,000 PDI 2. BSEIPH offices that receive capacity - building in the registry (number) 5 Dropped -- -- Added as PDI (prior IRI); revised target PDI 2. persons with disabilities who participate in the job placement service (number) 500 Added as PDI (prior IRI) PDI 3. persons with disabilities who participate in the job placement service who are female (percentage) 50% Added/New PDI 4. persons with disabilities who participate in training and obtain certification (number) 200 Original Intermediate Results Indicators (IRI) Target Changes Revised IRIs Target IRI 1. Human resources trained to identify persons with disabilities (number) 1,000 Revised indicator and target IRI 1. Human resources trained on employment opportunities for persons with disabilities (number) 30 IRI 2. persons with disabilities who participate in the job placement service (number) 1,000 Changed to PDI; Revised target IRI 3. persons with disabilities who participate in the job placement service who are female (percentage) 50% Changed to PDI IRI 4. Employers included in the job placement service database (number) 100 Revised target Became IRI 2. No change to wording. 30 IRI 5. BSEIPH staff and consultants receiving communication training (number) 10 No change Became IRI 3. No change to wording. 10 IRI 6. Awareness raising events implemented (number) 5 Revised target Became IRI 4. No change to wording. 3 Other Changes 19. Changes to the component and disbursement category allocations were made during the restructuring. Component 1 allocation increased from US$ 709,000 to US$ 1,244,320 , whereas Component 2 decreased from US$ 1,138,700 to US$ 600,000. Component 3 increased slightly from US$ 446,620 to US$ 450,000. Component 1 cost allocation was increased due to higher than estimated costs of the registration survey, which required fielding the SIMAST module in addition to the disability survey ( discussed in the Efficiency section below). Revision of the disbursement 6 Kore Fanmi (KF) had been introduced as a pilot project in the Central Plateau department and involved mapping of social services (using the KF management information system) and support for a local network of community agents. However, the KF approach faced challenges in achieving scale due to resource and capacity constraints and was ultimately phased out . The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 11 of 38 category allocation reflected the need for higher disbursements for Consulting Services for the survey and lower disbursements for Training and Workshops and Goods, due to reduced eq uipment needs. Synergies with the social registry SIMAST system allowed the BSEIPH to access equipment without support from this grant, while maintaining the training activities. Revised Theory of Change 20. The changes to the PDO and results framework are reflected in the revised theory of change (Figure 2). While project activities did not fundamentally change with the restructuring, intermediate and PDO outcomes and indicators were revised to better reflect what the original design intended to achieve, as well as to reflect challenging local conditions due to COVID - 19 and political unrest. N o PDO indicators were specified under Component 2 in the original project design, so the revised theory of change includes new PDO indicators that were added to better reflect outcome - level progress under Component 2 . Under Component 2, the focus was shifted to enhance access to employment services The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 12 of 38 rather than employment opportunities to better reflect existing project activities and feasible outcomes given the short project timeframe. Finally, the new PDO removed reference to increased access to social services, so the associated intermediate outcome and PDO outcome are removed from t he revised theory of change. II. OUTCOME Assessment of Achievement of E ach Objective/Outcome A. Relevance of PDOs Overall Rating: High 21. The relevance of the operation’s revised PDO is deemed High. The PDO is aligned with the WB’s Country Partnership Framework (CPF) for the Republic of Haiti FY16 - 19 (Report No. 98132 - HT) discussed by the Board of Directors on September 29, 2015 and the Haiti Performance and Learning Review (PLR) of the CPF (Report N o. 124812 - HT). 7 CPF Area of Focus 2 (Human Capital) aimed to support poverty reduction through investments in health and education and by closing gaps in service delivery. The P roject contributed to this objective by improving the targeting of social prot ection services to reach persons with disabilities and by enhancing their access to social and employment services. The project contributed to CPF Areas of Focus 1 (Inclusive Growth) through employment support for persons with disabilities to promote a mor e inclusive and accessible labor market. In addition, the 2015 Systematic Country Diagnostic (SCD) for Haiti identified improved targeting in social protection as a priority in the face of shocks and vulnerability. Project activities contributed to several cross - cutting SCD priorities, including strengthening state capacity, generating more and better jobs, reducing vulnerability, and developing evidence to inform planning and targeting of policies and programs. Finally, the Project supported the CPF cross - cutting theme on governance by enhancing efficiency of service delivery through improved targeting of persons with disabilities using the social registry and by supporting the BSEIPH to produce relevant data, manage the social protection sector, and set ev idence - based policy priorities. 22. T he PDO is consistent with domestic priorities and continues to be highly relevant to Haiti's growing social protection agenda . With the creation of BSEIPH in 2007 and passage of the 2012 law requiring better inclusion of persons with disabilities , the Project represented a crucial step towards building GoH capacity to better identify and support persons with disabilities in a co ntext of high vulnerability. Additionally, the National Policy of Social Protection and Promotion (PNPPS) that was adopted in 2020 prioritizes support for persons with disabilities including improved coordination between MAST and the BSEIPH and recognized the SIMAST social registry as the foundation for the targeting of social protection programs. B. Achievement of PDOs (Efficacy) 23. The following considerations should be noted regarding the methodological approach adopted for this Implementation Completion and Results Report (ICR): • The changes introduced under the restructuring in June 2020 required utilization of the split methodology. The relative weights reflect the disbursements made during the pre - and post - restructuring periods out of total disbursements (11.4 percent and 88.6 percent respectively). 7 The CPF period was originally set from 2015 to 2019. However, after completion of the PLR in 2018, CPF milestones were extend ed until 2021 while a new CPF is prepare d. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 13 of 38 • Efficacy for the pre - restructuring period is assessed based on the original PDO, which has been unpacked into three PDOs: identification of persons with disabilities (PDO 1); access to social serv ices (PDO 2); and access to employment opportunities (PDO 3). Performance is assessed against the original targets in the Project Results Framework. • Efficacy for the post - restructuring period is assessed based on the revised PDO, which has been unpacked i nto two PDOs: identification of persons with disabilities (PDO 1) and access to employment services (PDO 3). Performance is assessed against the revised targets in the Project Results Framework. Table 2. Summary of Efficacy Ratings – Pre and Post Restructuring Periods Outcomes Efficacy Rating Pre - restructuring (Original) Post - restructuring (Revised) PDO 1 : to identify persons with disabilities Substantial High PDO 2 : to increase access of persons with disabilities to social services Negligible N/A (Dropped) Original PDO 3 : to increase access of persons with disabilities to employment opportunities Revised PDO 3 : to increase access of persons with disabilities to employment services Modest High Pre - restructuring Rating: Modest 24. Efficacy for PDO 1 during the pre - restructuring period is assessed as Substantial . PDO 1 included two key activities: (1) development of a registration system for persons with disabilities , and (2) identification and registration of persons with disabilities . By project closure, PDI 1 was over partially achieved with 53, 453 persons wit h disabilities 8 registered in the SIMAST (see Table 3). A survey instrument was developed to capture information on persons with disabilities and was adapted to the local context , Creole language, and specificities of conducting surveys with persons with disabilities (including sign language interpreters and proxy interviews). 9 The questionnaire was based on the extended set of survey questions recommended by the Washington Group 10 in addition to questions on education , labor market status , a nd assistance received based on disability status . The survey tool was designed to link to the SIMAST survey in order to include persons with disabilities in the social registry . T he core social registry module was also administered to collect information on household characteristics . Data collection was delayed due to political unrest, COVID - 19 restrictions on in - person activities, and complexities in preparing the disability module as described above . A database manager hired by the Project received training in the SIMAST registry. 8 Surveys were conducted for all levels of disability (including severe, moderate, and mild). 9 The process for piloting, adapting, and deploying the survey tool included: (1) a workshop to pilot the questionnaire with pe ople with different types of disabilities; (2) an inclusive data collection methodology; (3) specific training on issues for data collection staff; and (4 ) close collaboration with local authorities and Organizations for Persons with Disabilities (OPDs) for data collection. The BS EIPH worked closely with the National Food Security Coordination (CNSA) to conduct the process. 10 The Washington Group on Disability Statistics (WG) was established under the United Nations Statistical Commission to address the need for cross - nationally compar able population - based measures of disability. Its mandate is to promote international coordination for health statistics by focusing on disability data collection for censuses and national surveys. The extended survey includes d etailed modules on types of disability , as well as needs in terms of assistive devices. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 14 of 38 Table 3. Status of original PDO 1 Indicators at closure Indicator Actual at Closure Original Target PDI 1: persons with disabilities documented in the national registry (number) 53, 453 100,000 PDI 2: BSEIPH offices that receive capacity - building in the registry (number) Dropped (only the central office received training ) 5 25. Efficacy for PDO 2 during the pre - restructuring period is assessed as Negligible . PDO 2 was ‘to increase access of persons with disabilities to social services.’ By including persons with disabilities in the social registry (SIMAST) , it was assumed that they would be reached with existing social services and support. While progress was made towards including persons with disabilities in the registry , improving access to social services would have required a stronger supply side intervention to ensure that services wer e made available to persons with disabilities . The original project design did not include any specific activities or indicators to assess progress towards achievement of PDO 2, and at the same time overestimated the availability of social services for per sons with disabilities . Therefore, achievement of this objective was negligible, and PDO 2 was dropped during the project restructuring. 26. Efficacy for PDO 3 during the pre - restructuring period is assessed as Modest . PDO 3 was ‘to increase access of persons with disabilities to employment opportunities’ and consisted of two key activities: (1) support the job placement service at BSEIPH and (2) raise awareness among employers to promote employment of persons with disabilities . There were no PDO indicators (PDIs) included in the original project design 11 to measure progress at the outcome level for this component of the PDO , so intermediate results indicators (IRIs) are used instead. By project closure, IRI targets for PDO 3 wer e either achieved or partially achieved (see Table 4). Therefore, the operation was expected to partly achieve its original objectives under PDO 3. The gaps between actual at closure and the original target are explained by an overly ambitious proj ect desi gn given the short implementation timeframe and difficult operating context in Haiti , which are discussed further in Section 3 below. Table 4. Status of original PDO 3 Intermediate Results Indicators (IRI) at closure Indicator Actual at Closure Original Target IRI 1 (original). Human resources trained to identify persons with disabilities (number) IRI 1 (revised). Human resources trained on employment opportunities for persons with disabilities (number) 40 1,000 IRI 2. persons with disabilities who participate in the job placement service (number) 679 1,000 IRI 3. persons with disabilities who participate in the job placement service who are female (percentage) 44.8% 50% IRI 4. Employers included in the job placement service database (number) 40 100 IRI 5. BSEIPH staff and consultants receiving communication training (number) 10 10 IRI 6. Awareness raising events implemented (number) 7 5 Post - restructuring Rating: Substantial 27. Efficacy for PDO 1 during the post - restructuring period is assessed as High. PDO 1 remained unchanged from the pre - restructuring period, however the end target was revised from 100,000 to 50,000 persons with disabilities documented in the national registry . The revised target reflected conditions in the country at the time of restructuring 11 Only two PDIs were included in the original project design, and both were associated with Component 1 (PDO 1 – to identify persons with disabilities). This is reflected in the original theory of change (Figure 1). The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 15 of 38 (discussed in Section 3 below) and accounted for the longer process of developing the disability survey module , and was therefore more realistic given the short project timeframe. However, c onsiderable progress was made in the final year of implementation. Outreach was conducted through a communications campaign led by a service provider , and by using existing lists of persons with disabilities available from the BSEIPH , other organizations including Organizations for People with Disabilities (OPDs) , and local authorities . By project closure, 53, 453 persons with disabilities were documented in the national registry . 12 To facilitate registration, BSEIPH introduced several innovations including a system to issue unique beneficiary cards to registered persons with disabilities , a media presence to encourage enrolment (through radio, T witter, Facebook, billboards and posters), and piloting of self - registration for persons with disabilities using kiosks, either in - person, via dependents or organizations that support persons with disabilities . The self - registration was used for pre - identi fication purposes and followed up with home visits conducted by enumerators to validate the information provided and conduct the full survey. The original PDI 2 ( BSEIPH offices that receive capacity - building in the registry) was dropped due to the short project implementation timeframe and lack of staff and resources in the decentralized BSEIPH offices , however the central BSEIPH office received capacity building . Table 5. Status of revised PDO 1 Indicators at closure Indicator Actual at Closure Revised Target PDI 1: persons with disabilities documented in the national registry (number) 53, 453 50,000 28. Efficacy for PDO 3 during the post - restructuring period is assessed as High . PDO 3 changed from the pre - restructuring period to reflect increased access of persons with disabilities to employment services rather than employment opportunities. The revised PDO 3 , indicators and targets, were in line with the conditions in the country at the time of restructuring and the short implementation timeframe. One PDI was added and two IRIs were elevated to PDIs to ensure that progress towards PDO 3 was sufficiently captured at the outcome level, based on the two major activities under Component 2 (support to job placement service and provision of training). The target for PDI 2 (original IRI 2) was reduced from 1,000 to 500 persons with disabilities participating in the job placement service due t o the difficult country conditions , and several end targets were revised to the original IRIs as well. PDI 4 was added at restructuring to measure the number of persons with disabilities who participate d in training and obtain ed certification. Requiring certification added an element of rigor to the training while signal ing the higher value of the training to potential employers. M any individuals who participated in the job placement service and/or trainings did go on to obtain p roductive employment (including self - employment) , indicating that final employment outcomes were achieved beyond the scope of the revised PDO . 29. Significant progress was made in the final year of implementation for PDO 3. Most revised targets were surpassed thanks to a proactive outreach strategy to employers and human resources departments and awareness - raising events to promote the integration of persons with disabilitie s in the labor market. O utcomes beyond the PDIs wer e achieved, including increas ed access to employment as envisaged in the original PDO. The BSEIPH job placement database expanded rapidly in the final year to include 679 individuals (exceeding the revised target of 500 ), including those who completed the trainings or participated in the job fairs, coaching workshops, or other BSEIPH services and activities. Approximately 45 percent of individuals who participated in the job placement service were female , which fell slightly short of the 50 pe rcent target. Through the job placement service, 27 persons with disabilities were matched with employment opportunities and 12 received scholarships to continue their studies. Training opportunities were provided 12 It is important to note that the survey did not have full geographic coverage in Haiti but covered 50 communes already includ ed in the existing SIMAST social registry as well as select commu nes in the Port - au - Prince metropolitan area. Therefore, while the number of persons with disabilities registered is significant and exceeded final revised targets, additional efforts will be needed to ensure full coverage of the social registry and disabil ity module to register the full population of individuals with some form of disability in Haiti. The World Bank Promoting Inclusion of persons with disabilitiess in Social Protection and Employment Programs (P145468) Page 16 of 38 for 157 individuals to improve their job - related skills in agricultural production and product transformation, motorcycle repair, cooking and baking, cellphone repair, infographics, massage therapy, as well as in employment search techniques. The training programs were carrie d out by service providers and designed for individuals with all types of disabilities except intellectual disabilities. Of the 157 individuals who completed training and obtained certification, 74 went on to create their own income - generating activities . Training was conducted in skill areas identified as ‘high - demand’ based on an employment study, which is evidenced by employment within one month of nearly one half of those trained. By the end of the project, an additional 60 individuals had completed most of the IT training with certification and distribution of job starter kits pending . 13 A nother funding source was proactively identified to allow training completion . Table 6. Status of revised PDO 3 Indicators at closure Indicator Actual at Closure Revised Target PDI 2: persons with disabilities who participate in the job placement service (number) 679 500 PDI 3. persons with disabilities who participate in the job placement service who are female (percentage) 44.8% 50% PDI 4. persons with disabilities who participate in training and obtain certification (number) 157 200 C. Efficienc y (implementation efficiency and economic efficiency) Rating : Substantial 30. Overall, efficiency is rated as Substantial. While the operation’s efficiency was overestimated during the design stage as discussed in Section 3 below , actual efficiency during implementation align ed with what would be expected in the social sectors in Haiti. With regards to implementation efficiency , the time between approval and first disbursement (12.6 months) wa s not uncommon for projects in Haiti and can be largely explained by external factors including institutional turnover and political instability (see Section 3). The restructuring corrected for initial shortcomings in project design (including overly ambit ious objectives and targets given