Performance Evaluation Report: Local Value Chain Enhancement (LEVE) Activity

PERFORMANCE EVALUATION REPORT LOCAL VALUE CHAIN ENHANCEMENT (LEVE) ACTIVITY JULY 2019 This publication was prepared independently by Social Impact, Inc. at the request of the United States Agency for International Development. PATRICK ADAMS FOR RTI INTERNATIONAL USAID/HAITI LOCAL ENTERPRISE VALUE CHAIN ENHANCEMENT ACTIVITY (LEVE) FINAL PERFORMANCE EVALUATION Final Report USAID/Haiti Office of Economic Growth Evaluation Mechanism No: AID-521-A-15-00009 Cover Photo Credit: RTI International This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Zuri Linetsky (Team Leader), Phanol Philippe (Assistant Team Leader), and Paul Rivera (ROI Expert) on behalf of Social Impact, Inc. Contact: Jennifer Mandel, Chief of Party 2300 Clarendon Blvd, Suite 1000 Arlington, VA 22201, USA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The five-year Local Enterprise and Value Chain Enhancement (LEVE) Activity sought to achieve four objectives across three sectors (agriculture, apparel and textiles, and construction): to enable MSMEs to work together to mutually create value; provide MSME’s with access to a productive and skilled labor pool; improve the sustainability of Haitian businesses benefiting from LEVE; and identify and improve synergies among existing activities and interventions. Based on key informant interviews with 45 respondents and four focus group discussions with students at technical and vocational schools that benefited from LEVE support, the Activity was a qualified success. The Activity achieved all four of its objectives in the agriculture and the apparel and textile sectors. However, an issue beyond LEVE’s control – specifically the sector’s informality – hindered its ability to achieve its objectives in the construction sector. LEVE achieved its objectives in two sectors because of its ‘light-touch’ Activity design, which was predicated upon facilitating business linkages, training new laborers, and organizational capacity building. These interventions empowered local business leaders to grow their operations, and, because they do not require direct LEVE support to continue, are likely to be sustainable over the long- term. The Activity’s adaptive management approach was critical to its success; LEVE tailored its interventions to individual businesses as well to the dynamic Haitian business environment. Longitudinal surveys with LEVE beneficiaries will be essential to evaluate the extent of LEVE’s long-term impact and sustainability. CONTENTS TABLES AND FIGURES I ACRONYMS II EXECUTIVE SUMMARY III INTRODUCTION III EVALUATION METHODOLOGY III DATA ANALYSIS III METHODOLOGICAL LIMITATIONS IV FINDINGS AND CONCLUSIONS IV CONCLUSIONS VI CONCLUSIONS VII RECOMMENDATIONS VII INTRODUCTION 1 BACKGROUND OF LOCAL CONTEXT 1 AGRIBUSINESS VALUE SECTOR 2 APPAREL AND TEXTILE SECTOR 2 CONSTRUCTION VALUE CHAIN 3 ACTIVITY OVERVIEW 3 KEY ASSUMPTIONS 3 EVALUATION PURPOSE & QUESTIONS 4 EVALUATION METHODOLOGY 5 DATA COLLECTION 5 LITERATURE REVIEW 5 KEY INFORMANT INTERVIEWS 5 FOCUS GROUP DISCUSSIONS 6 RETURN ON INVESTMENT ANALYSIS 6 DATA ANALYSIS 7 METHODS 8 EVALUATION LIMITATIONS 8 FINDINGS AND CONCLUSIONS 9 EVALUATION QUESTION 1: TO WHAT EXTENT AND IN WHAT WAYS WAS LEVE’S APPROACH SUCCESSFUL IN ACHIEVING ITS FOUR OBJECTIVES? 9 FINDINGS 9 CONCLUSIONS 19 EVALUATION QUESTION 2: TO WHAT EXTENT AND IN WHAT WAYS HAVE LEVE ACTIVITIES RELATED TO THE MAIN VALUE CHAINS GENERATED MULTIPLIER EFFECTS?20 FINDINGS 20 CONCLUSIONS 26 EVALUATION QUESTION 3: TO WHAT EXTENT AND IN WHAT WAYS HAS LEVE’S FACILITATION APPROACH BEEN EFFECTIVE IN EMPOWERING SUSTAINABLE LOCAL FIRMS/ENTITIES? 27 FINDINGS 27 CONCLUSIONS 33 RECOMMENDATIONS 34 ANNEXES 36 ANNEX A: EVALUATION SCOPE OF WORK 37 ANNEX B: DATA COLLECTION TOOLS 46 KEY INFORMANT INTERVIEW GUIDE FOR LEVE BENEFICIARIES 46 KEY INFORMANT INTERVIEW GUIDE FOR ACTIVITY IMPLEMENTORS (RTI) 49 FOCUS GROUP DISCUSSION GUIDE FOR USAID AND OTHER IPS 52 KEY INFORMANT INTERVIEW GUIDE FOR TECHNICAL AND VOCATION TRAINING CENTERS 58 FOCUS GROUP DISCUSSION GUIDE FOR TVET STUDENTS 60 ANNEX C: SOURCES OF INFORMATION 62 ANNEX D: DISCLOSURES OF CONFLICT OF INTEREST 64 ANNEX E: EVALUATION TEAM MEMBERS 67 i | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV TABLES AND FIGURES TABLE 1: KEY INFORMANT INTERVIEWS BY RESPONDENT GROUP AND LOCATION 6 TABLE 2: BASELINE VALUES FOR ROI ANALYSIS 25 FIGURE 1: RESPONDENT ASSESSMENT OF LEVE’S OVERALL WORK 9 FIGURE 2: RESPONDENT ASSESSMENT OF LEVE’S TRAINING AND CAPACITY BUILDING 10 FIGURE 3: BASELINE AND ALTERNATIVE SCENARIOS FOR LEVE ROI 25 FIGURE 4: LEVE ROI BY QUARTER 26 FIGURE 5: ACCUMULATION OF LEVE COSTS AND JOBS 26 USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | ii ACRONYMS A&T Apparel and Textile ADIH Association des Industries d’Haiti AmCham American Chamber of Commerce CIDA Canadian International Development Agency CLP Chanje Lavi Plante CNAIH Collège National des Ingénieurs et Architectes Haïtiens COP Chief of Party COR Contracting Officer’s Representative EGAD Economic Growth and Agriculture Development ESS Evaluation and Survey Services ET Evaluation Team EQ Evaluation Question FCR Findings, Conclusions, and Recommendations FGD Focus Group Discussion GDP Gross Domestic Product G o H Government of Haiti HACCP Hazard Analysis Critical Control Point HELP Haiti Economic Lift Program HOPE Haitian Hemispheric Opportunity through Partnership Encouragement ICC International Code Council INEPCO Institut National pour le Développement et la Promotion de la Couture INFP National Institute of Vocational Training IP Implementing Partner JAA J .E. Austin and Associates KII Key Informant Interview LEAD Leveraging Effective Application of Direct Investments LEVE Local Enterprise Value Chain Enhancement LOE Level of Effort LOP Life - of - project M&E Monitoring and Evaluation MATCON International Conference On Materials For The Millennium MCI Ministry of Commerce and Industry MEL Monitoring, Evaluation & Learning MSME M icro - , S mall , and M edium E nterprise MTPTC Ministry of Public Works, Transport and Communications NGO Non - Governmental Organization PANSEH National Support Program for the Structuring of Haitian Entrepreneurship ROI Return on Investment RTI Research Triangle International SARA Stand - Alone Roads Activity SI Social Impact , Inc. SOP Standard Operating Procedure SOW Scope of Work STTA Short - term Technical Assistance TASC Textile and Apparel Service Center TVET Technical and Vocational Education and Training USAID United States Agency for International Development USD United States Dollars USG United States Government iii | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY INTRODUCTION The United States Agency for International Development (USAID) requested that Social Impact, Inc. (SI)’s Haiti Evaluation and Survey Services (ESS) project design and implement the final performance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. The primary stakeholders for this evaluation include USAID/Haiti, RTI International, and its principle partners: J.E. Austin and Associates (JAA), Papyrus, TetraTech, Haitian micro-, small, and medium enterprises (MSMEs), other donors supporting MSMEs in Haiti, Haitian private sector entities, and the Government of Haiti (GoH). This evaluation report’s purpose is to explain the evaluation team’s (ET) findings and conclusions regarding LEVE’s ability to meet its four objectives, provide a detailed Return on Investment (ROI) analysis, and clarify the extent to which LEVE’s work will be sustainable in the wake of the Activity’s end. This report also provides actionable recommendations to USAID about follow-on Activities. EVALUATION METHODOLOGY The ET used both qualitative and quantitative methods, including an in-depth desk review of Activity reports, focus group discussions (FGDs), key informant interviews (KIIs), and an ROI analysis, using relevant data that LEVE had already collected. 1. Desk Review: The ET conducted a review of documents produced by LEVE and partner institutions to understand the Activity design and implementation, as well as inform data collection. 2. Key Informant Interviews: The ET conducted one-on-one and small group KIIs with 45 individuals in Port-au-Prince, Cap-Haitian, and Saint-Marc using a purposive sampling strategy. 3. Focus Group Discussions: The ET conducted four FGDs with current students of Technical and Vocational Education and Trainings (TVETs). 4. ROI Analysis: The ET conducted an ROI analysis (or, profitability analysis) to calculate the ratio of the net present value of net benefits to the net present value of investment costs for the entire Activity implementation period. DATA ANALYSIS The team took detailed notes of all KIIs and FGDs, both by hand and using audio recordings of every interview in which respondents consented to being recorded. The ET transcribed all these notes into Microsoft Word documents and stored them on the SI server to ensure security. The ET reviewed and discussed the interviews it conducted, and recorded emergent themes identified in these meetings in a findings, conclusions and recommendations (FCR) matrix resembling a report outline. The ET transferred coded emergent themes into a tally sheet matrix to count the frequency with which themes occurred (in both KIIs and FGDs), with tallies disaggregated by stakeholder group, geographic area, and sex. The tally sheet matrix, as well as the FCR, were the substrate of this final report. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | iv METHODOLOGICAL LIMITATIONS The ET encountered several limitations while in the field. Most notably, country-wide protests interrupted fieldwork, which prevented the ET from conducting any face-to-face interviews over a ten- day period. To make up for lost time, the ET conducted several interviews in Port-au-Prince and Saint- Marc by phone, five interviews during the data analysis phase of fieldwork, as well as one interview after the formal out brief with USAID. Protests and political instability prevented the ET from visiting the Saint-Marc region, an area beset by security challenges. As a result of these circumstances, the ET cancelled its trip to the area and conducted all interviews by phone. The ET was also unable to conduct its planned FGD with TVET students in Saint-Marc. The ET also notes four additional limitations that affected this evaluation. First, beyond the sample of people the ET interviewed, the ET cannot generalize based on the qualitative data it collected for this evaluation. However, the ET compared this data with similar findings from the 2018 Aid for Trade assessment, which ESS also conducted, and the findings were consistent between the two reports, suggesting that the data the ET collected for this evaluation is internally valid. 1 Secondly, and relatedly, since the ET spoke to LEVE beneficiaries, LEVE staff, and implementing partners (IPs), it is possible that social desirability bias (telling the ET what respondents thought it wanted to hear to ensure they might benefit from follow-on USAID Activities) affected responses. However, given the alignment of findings between this evaluation, the Aid for Trade assessment, and the ROI analysis, the ET is confident that this form of bias did not notably impact interview responses. Third, the GoH was not an active participant in this evaluation, nor was it regularly involved in the LEVE Activity. Representatives from several key Ministries, including the National Institute of Vocational Training (INFP) and the Ministry of Commerce and Industry (MCI), were unwilling to meet with the ET. While the Ministry of Public Works, Transport and Communications (MTPTC) agreed to meet the ET, it reported no knowledge of the LEVE Activity, likely due to staff turnover. Finally, while LEVE provided it’s the ET M&E data, these were incomplete. By design, LEVE’s M&E systems did not collect the data necessary for the ET to conduct an accurate ROI analysis, an issue the ET addressed by approximating, or modeling, some of the key data points for the ROI analysis—much like LEVE conducted with its own ROI. Further, USAID was unable to collect relevant baseline data for LEVE before the Activity began, which the ET would have used to inform its ROI. 2 FINDINGS AND CONCLUSIONS Evaluation Question (EQ)1: To what extent and in what ways was LEVE’s approach successful in achieving its four objectives? The evaluation should specifically examine what factors supported and/or hindered its approach identifying the most promising sectors, the best approach to working with the Government of Haiti, and the private sector. An overwhelming majority of KII and FGD respondents believed LEVE was somewhat-to-very successful in achieving its objectives; over half believed LEVE was very successful. Similarly, over half of interviewees reported LEVE’s training and capacity building interventions (core components of the LEVE 1 Several of the questions were similar, and there were a number of respondents who participated in both evaluations, making a comparison between the two evaluations possible and appropriate. 2 USAID tried but was unable to obtain these data. According to the Economic Growth Office (EGAD), USAID released two separate solicitations for firms to collect the necessary baseline data, but no appropriate vendor could be identified before the LEVE activity began v | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV Activity) were somewhat to very impactful; a majority of these respondents reported LEVE training and capacity building support was very impactful. The Activity was successful in meeting its four objectives. Through its facilitation approach in the apparel and textile (A&T) and agribusiness sectors, LEVE drove synergies and the creation of business linkages among MSMEs, as well as between MSMEs and Lead Firms (Objective 1). Through its training, capacity building and back-office support interventions, LEVE increased the productivity of MSMEs and their staff. By working with TVETs, LEVE also improved the quality of new workers that were entering the A&T and construction value chains (Objective 2). A clear majority of interviewees said that LEVE’s interventions will be sustainable in the A&T and agricultural sectors (Objective 3), specifically because LEVE’s design underscored facilitation, helping to establish business linkages, drive business empowerment, and diversify existing company’s businesses. Finally, both LEVE staff and Activity beneficiaries noted that they worked effectively with other donors and USAID Activities to empower local actors (Objective 4). For example, LEVE improved Papyrus’ capacity as an implementing partner (IP). Unfortunately, due to systematic challenges in the construction sector, many of which were beyond LEVE’s control, including changes in USAID staffing and Activity implementation decisions, LEVE did not meet Objectives 1 and 4 in the construction sector. However, LEVE did, despite major challenges, develop a new plumbing code for Haiti, and contribute to training and placing more skilled workers in the construction sector. LEVE also supported Technical and Vocational Schools to train female welders and plumbers in the North, and solar panel technicians in Port-au-Prince (Objective 2). Due to their focus on building training capacity, these interventions are likely to be sustainable (Objective 3). Interviewees noted that future USAID Activities should consider identifying niche areas in which to build Haitian labor capacity to produce value added goods. Finally, interviewees reported that the most effective approach for working with the GoH is to build personal relationships with technical staff in Ministries relevant to any intervention, include these staff in the design of every USAID intervention, and maintain this relationship over the life of an Activity. CONCLUSIONS • LEVE was successful in achieving the four objectives of its facilitation approach, which allowed it to contribute to building business linkages, support training and capacity building for MSMEs, and empower MSMEs. • LEVE’s approach and the challenges it encountered working with the GoH highlighted a pathway toward a successful working relationship; new Activities must identify a consistent champion among technical staff in relevant Ministries with whom to cultivate an enduring personal relationship. • New Activities must help build a business-empowerment culture in the GoH, which many interviewees believe is lacking. LEVE, at least in the A&T sector, accomplished this due to the personal relationships between its staff and GoH officials. EQ 2: To what extent and in what ways have LEVE activities related to the main value chains generated multiplier effects? The evaluation should conduct a return on investment analysis using LEVE’s existing data. Then the evaluation should examine the multiplier effects, if any, that reflect the return on investment. The ROI analysis showed that LEVE’s job creation benefits must continue for a minimum of six quarters beyond the end of the Activity to achieve a break-even ROI value of 1.0. For the Activity USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | vi implementation period, the ET’s analysis of the direct job creation benefits yielded a baseline ROI value of 0.424 for the apparel sector, reflecting 99 percent of LEVE’s job creation results. LEVE did not create jobs in the agribusiness and construction sectors at the scale necessary for the ET to generate a credible ROI; thus, the ET excluded job creation in these sectors from its analysis. However, if job creation benefits in the apparel sector continue through the end of the Haiti Economic Lift Program (HELP) and Haitian Hemispheric Opportunity through Partnership Encouragement (HOPE) Acts in 2025, LEVE will yield a direct ROI value of 1.533. In the absence of data on LEVE’s secondary impacts, the ROI analysis alternatively utilized a range of income multiplier values from a previous apparel sector study in Haiti to account for indirect and induced benefits originating from LEVE's job creation interventions. When the ET used the multiplier results in a baseline ROI value of 0.543 for the Activity implementation period, ROI increased to 1.947 when projected to 2025. The study modeled scenarios utilizing a wide range of values for each relevant parameter, which never resulted in an ROI value above 0.70 during the Activity implementation period. When the study expanded benefits through 2025, the ET recorded ROI values above 1.30 for all justifiable variations of the model parameters. CONCLUSIONS • The ROI analysis may not accurately represent LEVE's impact or success. While the ROI analysis demonstrated the positive and increasing benefits of LEVE's job creation interventions over time, the time-to-build dimension of job creation and the necessary front-loading of costs created a downward bias in LEVE's ROI values, as the methodology placed greater emphasis on earlier events. • The main driver of LEVE’s long-term ROI is the continuity of benefits beyond the Activity implementation period. Elements the ROI analysis did not measure, like sustainability, network linkages, and technical assistance, will allow LEVE’s benefits to persist and expand over time— the long-term ROI. • For the ET to produce an accurate ROI, LEVE required a more rigorous and complete M&E system. EQ 3: To what extent and in what ways has LEVE’s facilitation approach been effective in empowering sustainable local firms/entities? Among other empowerment criteria, the evaluation should consider the level of leadership taken by the firms in the undertaken activities and success in terms of sales, revenue and job creation in the analysis. As detailed in EQ 1, LEVE’s work with TVETs was a key factor in empowering sustainable local firms and entities. More than a third of interviewees said that LEVE support to TVETs was positive-to-very positive. Most TVET respondents, including both directors and students, reported satisfaction with the training and job placement services they received, all of which LEVE facilitated. Additionally, LEVE pushed TVETs to operate more like businesses to fund their operations, given that state funding for technical and vocational schools was inadequate. This operational change may make TVETs more sustainable in the long-term. In terms of general empowerment, almost half of interview respondents said that LEVE beneficiaries were empowered to be leaders in their value chains and/or sectors. LEVE achieved this empowerment through local connections and business linkages, back-office support, access to capital, access to machinery, business diversification, and capacity development. These interventions were sustainable, as evidenced by beneficiary businesses that are currently replicating LEVE practices despite LEVE’s closure. The linkages that LEVE provided to short-term technical assistance (STTA) were also a critical and sustainable source of integration and support; firms that can afford it can continue to contract with STTA and maintain and/or expand upon the relationships that LEVE helped them establish. Respondents vii | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV also noted that LEVE interventions helped them manage through enduring political instability in Haiti, and even increased their business revenues. In addition to chronic political instability, there are two major challenges to business sustainability in Haiti: 1) A lack of access to credit at competitive interest rates; and, 2) LEVE’s closing and cessation of facilitation and technical assistance, which interviewees stated could limit/slow business growth in the short-term. Worker training in the A&T sector will potentially present the most pressing gap now that LEVE has ended, as LEVE was instrumental in maintaining Association des Industries d’Haiti (ADIH) and its vocational training center, Textile and Apparel Service Center’s (TASC’s) training capacity. CONCLUSIONS • LEVE’s interventions successfully empowered local businesses. • LEVE’s interventions were proven sustainable because they focused on developing systems and practices that LEVE beneficiaries are currently replicating and expanding in the absence of LEVE. • Current business linkages among LEVE beneficiaries after LEVE has closed, as well as ties to technical assistance providers, demonstrate that LEVE’s interventions are likely to be sustainable. • Those TVETs that can identify sources of independent funding (like producing and selling goods from the schools), and those that demonstrate the ability to place students in relevant industry jobs, are more likely to be sustainable businesses. RECOMMENDATIONS • Follow-on Activities should continue building on LEVE’s training and back-office support. • Follow-on Activities should follow LEVE’s light-touch facilitation approach. • Follow-on Activities should prioritize building personal relationships within the GoH at the technical-office level. • While there are significant gaps in construction, future Activities must identify capacity building niches for Haitian business and labor. • Any follow-on Activity will require more robust M&E systems, mechanisms for collecting ROI- relevant data from Activity outset, as well as a commitment from USAID to deliver baseline data, especially if USAID will require data-intensive analysis (like ROI). • USAID should commission a survey of LEVE beneficiaries, such as individuals who acquired new jobs or improved jobs, over the next five years to evaluate the Activity’s long-term impact. 1 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV INTRODUCTION The United States Agency for International Development (USAID)/Haiti requested that Social Impact, Inc. (SI)’s Haiti Evaluation and Survey Services (ESS) design and implement the final Performance Evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. The primary stakeholders for this evaluation include USAID/Haiti, RTI International, and its principle partners: J.E. Austin and Associates, Papyrus, TetraTech, Haitian micro-, small, and medium enterprises (MSMEs), other donors supporting MSMEs in Haiti, Haitian private sector entities, and the Government of Haiti (GoH). The purpose of this final performance evaluation report was to present findings, conclusions and recommendations based on the evaluation team’s (ET’s) review of Activity documentation, as well as data gathered through key informant interviews (KIIs), focus group discussions (FGDs), and a return on investment analysis (ROI). This report details the methodology the ET used to conduct fieldwork and data analysis, the ET’s limitations and mitigation measures, and evaluation findings, conclusions and recommendations. BACKGROUND OF LOCAL CONTEXT As a result of political instability, trade embargos, and environmental catastrophes, Haiti has suffered through years of limited economic development, which has played a critical role in preventing the GoH from addressing domestic social and economic challenges. 3 Prior to the 2010 earthquake, Haiti faced high levels of unemployment, extreme income inequality, and a limited tax base. The earthquake itself destroyed infrastructure, human capital, inventory and collateral—the assets underpinning Haitian businesses and, therefore, the entire Haitian economy—fundamentally undermining national economic stability. As of April 2019, more than 96 percent of the Haitian population has been exposed to natural disasters, including hurricanes, floods, and earthquakes. Currently, the Haitian economy provides only limited livelihood opportunities for its citizens. Haiti has a Gross Domestic Product (GDP) per capita of $870, and ranks 168 of 188 countries on the United Nation’s Development Program’s human development index. 4 An estimated six million Haitians live under the national poverty line of $2.41 United States Dollars (USD) per day, and more than 2.5 million people live below the extreme poverty line of $1.239 USD per day. 5 The United States has also reduced its aid budget for Haiti since 2017, and the Temporary Protected Status for Haitians (people allowed to live and work in the United States after war or a major natural disaster in their own countries) is set to end in January 2020. 6 Haitian politics contribute to local instability. The current President of Haiti was inaugurated on February 7, 2017, which returned the country to political order after Haiti had been without an elected 3 Daniel Runde, “Haiti’s Governance Problems Require Touch Love From Donors,” Forbes, April 17, 2015, accessed May 7, 2019, https://www.forbes.com/sites/danielrunde/2015/04/17/haitis-governance-problems-require-tough-love-from- donors/#13657df82917 . 4 United Nation Develop Programs, “Human Development Report: Haiti,” accessed May 19, 2019, http://hdr.undp.org/en/countries/profiles/HTI . 5 The World Bank, “The World Bank in Haiti: Overview,” April 5, 2019, accessed May 7, 2019, https://www.worldbank.org/en/country/haiti/overview . 6 Maureen Taft-Morales, “Haiti’s Political and Economic Conditions: In Brief, “ Congressional Research Service,” December 1, 2017, accessed May 6, 2019, https://fas.org/sgp/crs/row/R45034.pdf . See also, Franco Ordeñez and Jacqueline Charles, “Trump Administration Extends TPS for Haitian and Three other Groups,” McClatchy Dc Bureau, March 1, 2019, accessed May 7, 2019, https://www.mcclatchydc.com/news/politics-government/white-house/article226943724.html . USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 2 President for a year. The new President, Jovenel Moïse, was elected with only 21 percent of the vote, meaning he had almost no political mandate. His administration has since seen a cholera outbreak, stagnant economy, enduring political protests caused by a poor economy, and public calls for an end to government corruption. Moïse arrived under the shadow of an ongoing Haitian government investigation into his possible involvement in money laundering and irregular loan arrangements, allegations Moïse has denied. In May 2019, Haiti’s Superior Court of Auditors and Administrative Disputes alleged that Moïse and other officials embezzled millions of dollars. 7 These corruption allegations led to protests and riots that affected the ET’s ability to collected data for this evaluation. AGRIBUSINESS VALUE SECTOR Agribusiness is a critical business sector in Haiti and can be an engine for national economic growth. According to the Haiti Statistics and Informatics Institute, nearly 50 percent of the Haitian population work as farmers or skilled laborers in agriculture or fishing. 8 This represents roughly a ten percent decrease from the 2008 census. There are an estimated one million farmers in Haiti, each working an average of 1.5 hectares (ha) of land, which accounts for some 25 percent of national GDP. Haiti’s agriculture sector supplies 50 to 60 percent of the food consumed in Haiti. 9 Public or private institutions have never invested enough in this sector to reach the level required to establish a national industrial agriculture foundation. Historically, Haitian farmers have undertaken exploitative agriculture, which has eroded the very agricultural lands that the population needs to survive. Due to various political, economic, and social crises, the GoH has been forced to import large quantities of often-subsidized agricultural goods, making Haiti “one of the most open agricultural sectors in the world.” 10 Competition between domestic producers and cheaper imported agricultural products, as well as the GoH’s lack of planned and systematic investment in the agricultural sector, has hindered the development of a more sustainable and competitive agricultural sector. 11 APPAREL AND TEXTILE SECTOR Overall, the apparel and textile (A&T) sector accounted for more than 50,000 jobs in January 2018 and produced more than $1 billion worth of exports in 2015. 12 There are two aspects of the apparel and textile sector in Haiti: Mass industrial production, and small-scale stitchers and designers. The mass industrial production value chains employ the majority of labor in this sector, accounting for more than 30,000 jobs. 13 Several small sewers and designers make goods for the uniform market (schools, etc.), designer haute couture, tourist wear, and costumes for Haitian celebrations like carnival. Additionally, several companies work for high-end tailors. The smaller-scale sub-sector is often only semi- commercial, and sells products by attaching ethical or sympathy (charity) labels to its goods. 14 A key factor facilitating the expansion of the Haitian apparel and textiles sector is the Haitian Hemispheric Opportunity through Partnership Encouragement (HOPE)/ Haiti Economic Lift Program (HELP) legislation, which allows Haitian producers to obtain duty-free preferences in the Unites States. 7 Congressional Research Service, “Haiti’s Political and Economic Conditions,” July 1, 2019, https://fas.org/sgp/crs/row/R45034.pdf , p. 1. 8 Haitian Institute of Statistics and Informatics, “Economic Characteristics: Main Occupation,” Ministry of Economics and Finances , 2016, accessed May 8, 2019, http://www.ihsi.ht/rgph_resultat_ensemble_CE.htm . 9 USAID/Haiti, “Value Chain Assessment Annex 3. Agribusiness Sector Assessment: Local Enterprise and value Chain Enhancement (LEVE) Project,” April 2014, p. 5. 10 Ibid, p. 5. 11 Ibid, p. 5. 12 These are the most up to date data. Haiti Open Data, “Textiles and Apparel Industry,” accessed May 8, 2019, http://opendata.investhaiti.ht/pulbare/textiles-and-apparel-industry . 13 USAID/Haiti, “Value Chain Assessment Annex 1. Apparel and Textiles Sector Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. 2. 14 Ibid. 3 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV However, while Haiti’s apparel and textile industry has grown, growth has been slow and localized to low value-added goods like T-shirts and other cut-and-sew products. CONSTRUCTION VALUE CHAIN The construction value chain in Haiti encompasses two main parts: buildings (residencies, hospitals, schools, hotels, restaurants, etc.) and infrastructure (roads, bridges, tunnels, power lines, pipelines etc.). These sectors possess a boom-or-bust quality due to the enduring threat of natural disasters like the 2010 earthquake. As with Haiti’s agricultural sector, lack of GoH investment in infrastructure development is caused by lack of borrowing capacity and high interest rates. The lack of GoH dollars precludes large-scale investment in major public works projects. 15 USAID/Haiti’s 2014 Value Chain Assessment concluded, “…The construction industry cannot rely on public works as a major source of growth, except for projects such as road paving, schools, and hospitals, which traditionally grow at a steadier rate. In most developing countries, public works infrastructure is a key job creator for the construction industry sector.” 16 There is also variation in construction sector capacity within Haiti; in Cap-Haitian, the sector is much less developed than in the capital. However, in Cap-Haitian, tourism- related building, new hotel rooms, and the international airport, as well as the continued growth of beach resorts and the expansion of the industrial park, drive sector growth. 17 ACTIVITY OVERVIEW Limited economic development has long hindered Haiti from addressing social and economic issues. Even before the January 2010 earthquake, Haiti faced high levels of unemployment, income inequality, and a limited tax base. The earthquake exacerbated these problems, destroying infrastructure, human capital, and inventory, further increasing economic instability. In response, USAID has supported Haiti’s economic development by rebuilding damaged infrastructure, promoting policies that lead to a business- friendly environment, expanding access to financial products, and aiding MSMEs. USAID launched the 34,265,488 USD LEVE Activity (contract number: AID-521-C-14-00001) to increase economic security. USAID designed LEVE to create jobs in target industries/sectors in and around Port-au-Prince, Saint-Marc, and Cap-Haitian by creating more inclusive and productive value chains. Implemented by Research Triangle Institute International between December 2013 and June 2019, LEVE targeted MSMEs to help improve and enhance their capacity and worked with technical and vocational schools to assist them to improve their courses and job placement processes. In Port-au- Prince, LEVE assisted all three of its target sectors: agribusiness, construction, and the A&T industry. In and around St. Marc, LEVE focused particularly on the agribusiness sector, and in the Cap-Haitian corridor, it assisted the construction and A&T sectors. LEVE’s Development Hypothesis stated that if Haitian MSMEs in high potential sectors engaged with other value chain actors to mutually create value in response to market demand, and if they were supported by a productive labor pool with skills and competencies relevant to the target sectors, then those value chains would be more inclusive and productive, thereby leading to increased job creation in Haiti. KEY ASSUMPTIONS 15 USAID/Haiti, “Value Chain Assessment Annex 2. Construction Industry Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. 5. 16 Ibid. 17 Ibid. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 4 The LEVE Activity made several assumptions in its design. First, the Activity assumed that these three sectors (agribusiness, construction, and the A&T industry) could support sustainable long-term employment for Haitian workers. LEVE made this assumption based on the idea that these are “high- potential sectors” capable of generating broad-based employment in the three development corridors supported by the United States Government (USG). 18 Second, the Activity assumed that the raw materials necessary to allow these sectors to grow, and thereby generate large-scale employment, were readily available. Relatedly, the Activity assumed that there was interest from the GoH and the sectors themselves in creating more employment opportunities. Third, the Activity assumed that working with Lead Firms in certain sectors and training to produce a more skilled labor force would produce more inclusive and higher-quality job creation – i.e., a trickle- down effect for all sectors that LEVE covered. To address the Haitian employment context in the three sectors of interest through the three development corridors, LEVE established four key indicators of success: 19 • Increase in sales (volume and USD value) in target value chains; • Percentage or USD value increase in investment in target value chains; • Percentage increase in productivity (modified to fit each targeted value chain); and • Number of new jobs (full-time equivalents) created in target sector in designated corridors. USAID estimated during the design phase that LEVE would create 8,000 – 14,000 jobs (as measured by full time equivalents (FTEs)) during the Activity’s three-year base period; the Mission later reduced this figure to 6,000. The Activity also initially planned to increase the investments of 155 businesses. EVALUATION PURPOSE & QUESTIONS The purpose of this final performance evaluation was to orient USAID/Haiti’s new generation of support to MSMEs by analyzing how successful LEVE was in achieving results relative to each of the Activity’s components. This final evaluation answers the evaluation questions (EQs) below: 1. To what extent and in what ways was LEVE’s approach successful in achieving its four objectives? The evaluation should specifically examine what factors supported and/or hindered LEVE’s approach to identifying the most promising sectors and what factors determined the best approach to working with the GoH and the private sector. 2. To what extent and in what ways have LEVE activities that are related to the main value chains generated multiplier effects? To answer this question the evaluation should conduct a return on investment (ROI) analysis using LEVE’s existing data. Then the evaluation should examine the multiplier effects, if any, that reflect the ROI. 3. To what extent and in what ways has LEVE’s facilitation approach been effective in empowering sustainable local firms/entities? Among other empowerment criteria, the evaluation should consider the level of leadership taken by the firms and their success in terms of sales, revenue, and job creation. 18 LEVE contract, section C. 19 LEVE contract, section C, p. 12. 5 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV EVALUATION METHODOLOGY The ET used a mixed-method, primarily qualitative, evaluation design consisting of four principal data collection methods: Document review, KIIs, FGDs, and an ROI analysis. The team assessed LEVE’s quantitative data through a review of the Activity’s performance monitoring documents (e.g., original and updated versions of LEVE’s M&E Plan, indicator tracking tables in its quarterly and annual reports), and an ROI analysis (see below). 20 The ET triangulated this quantitative data with qualitative data collected through its KIIs, FGDs and review of additional background documents. DATA COLLECTION LITERATURE REVIEW The ET reviewed documents LEVE and partner institutions produced to understand LEVE’s design and implementation, extract findings relevant to the EQs, and inform data collection protocol to develop instruments that appropriately supplemented or cross-checked information in background documents. The ET used the following categories of documents; Annex B contains full descriptions of documents: • Annual Reports: FY14 - FY18 • Quarterly Reports: FY14 – Q2 and Q3; FY15 – Q1, Q2, and Q3; FY16 – Q1, Q2, and Q3; FY 17 – Q1, Q2, and Q3; FY 18 – Q1, Q2, and Q3; FY19 – Q1 • M&E and work Plans: FY 14 - 19 • LEVE workforce assessment report • LEVE Apparel and Textile, Agribusiness and Construction sector value chain assessments • Data used for the RTI International’s internal ROI Analysis KEY INFORMANT INTERVIEWS The ET used a purposive sampling approach to identify candidates for KIIs and ensured balanced geographic, sector, and gender representation to the extent possible. The ET identified an initial list of respondents from LEVE documentation, which LEVE staff supplemented by LEVE staff to help fill any potential data gaps, as well as obtain relevant contact information. The ET conducted KIIs with 45 key stakeholders including 28 males and ten females in Port-au-Prince, four males in Cap-Haitian, and two males and one female in Saint-Marc. KII participant groups included LEVE beneficiaries in the construction, agribusiness, and apparel sectors, with an emphasis on construction sector actors due to LEVE’s limited success in this sector, TVET officials, LEVE and other implementing partners (RTI International, Papyrus), officials from relevant GoH agencies, and USAID/Haiti. 20 The ET acknowledges the assertions within the recent USAID/OIG report about flaws and omissions in LEVE’s performance data, some of which the ET discusses above (see preliminary findings for EQ 1, information gaps under EQ 3). While the it approached the data with these assertions in mind, the ET found it necessary to analyze this data not only to validate (or invalidate) the issues detailed in the OIG report, but also to utilize validated performance data to triangulate these results with its qualitative (e.g., KII, FGD) findings. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 6 TABLE 1: KEY INFORMANT INTERVIEWS BY RESPONDENT GROUP AND LOCATION RESPONDENT GROUP PORT - AU - PRINCE SAINT MARC CAP HAITIAN LEVE BENEFICIARIES Construction 4 0 0 Agribusiness 5 3 0 Apparel & Textiles 7 0 1 TVETs (one non - beneficiary) 3 1 3 USAID 2 0 0 IMPLEMENTING PARTNERS 12 0 0 GoH 4 0 0 KII TOTAL (45) 37 4 4 FOCUS GROUP DISCUSSIONS The ET conducted four FGDs with current students and recent graduates of LEVE-supported TVETs, including two in Port-au-Prince and two in Cap-Haitian. The ET sampled FGDs participants utilizing a convenience sampling strategy. At school visits, the ET selected FGD participants by asking TVET administrators to identify six to ten students who were beneficiaries of LEVE support; because TVET identified these students, it is possible that social desirability bias affected evaluation findings. Of the four FGDs, one contained entirely female respondents, while the three others included both men and women. The ET designed FGDs to gather information from current and recent students about their challenges acquiring the skills necessary to secure and maintain sustainable employment in Haiti’s job market, and the degree to which LEVE’s TVET support had provided them with skills and connections to companies seeking employees with those skills. The ET developed data collection protocols for FGDs (see Annex C), like those for KIIs, that both addressed and were derived from the EQs. RETURN ON INVESTMENT ANALYSIS ROI analysis involves two primary components: effectiveness and profitability. Effectiveness refers to an assessment of the extent to which an activity or investment met its objectives. The ET’s response to EQ 1 fully addresses effectiveness, and for methodological purposes, the ROI analysis included here is synonymous with profitability analysis. The ET calculated ROI as a simple ratio of net benefits to investment costs. To accurately determine benefits, LEVE needed to provide quarterly data disaggregated by sex and sector on: 1) the number of new jobs LEVE created among the target beneficiary population as a result of the Activity; 2) measures of beneficiary wages; 3) beneficiaries’ time worked per week or month as appropriate; and, 4) employment duration. However, LEVE only provided data on job creation. Cost information came from data on USAID disbursements for LEVE. These data included the total amount disbursed by quarter, and account for all costs LEVE incurred to implement interventions. To standardize the value of benefits and costs over time, the ET selected an appropriate discount rate. For activities where beneficiaries were private individuals, discount rates closely paralleled prevailing interest rates. As a conservative estimate, the ET applied to the ROI the historical average of interest 7 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV rates in Haiti during the 2014-2019 period of Activity implementation and adjusted all monetary values to net present value at the time of Activity initiation. 21 The ET then calculated ROI as the ratio of the net present value of net benefits to the net present value of investment costs for the entire period of Activity implementation. Based on the results of EQ 1, the ET also deemed it appropriate to include a forward estimation of benefits based on average job creation by firm with an appropriate decay rate 22 for new job creation after Activity sunset. The ET similarly converted these additional estimated benefits to net present value to calculate expected ROI. The ET verified the robustness of results via sensitivity analyses by varying the effects on calculated ROI of changing key parameters within the model. MULTIPLIER ANALYSIS: In applied development work, analysis that includes spillover effects, like a multiplier, increases the likelihood of introducing subjectivity into analytical results and weakens the reliability of attribution between Activity interventions and purported results. Once analysis hypothesizes Activity impacts beyond the target beneficiaries, its scope is no longer exclusively about job creation, as in the case of LEVE, and analysis must then consider a broad set of potential secondary effects. Spillover effects are, thus, not typically part of ROI analysis, and the ET does not recommend them in this case. EQ 2 nonetheless specifies an analysis of multiplier effects due to LEVE Activities. Without performance data to support LEVE's spillover effects, the ET carried out multiplier analysis utilizing income multipliers adopted from a previous study on the Haitian apparel sector. The validity of the ROI multiplier results thus depend on the relevance of the utilized income multipliers. Methodologically, the multiplier functioned as an inflation factor to the net direct benefits of the ROI calculation, and the ET incorporated adjusted values into the standard equation. The ET reports the generated ROI value as an estimate, noting the diminished certainty of attribution stemming from hypothesized parameter estimates. DATA ANALYSIS The team took detailed notes of all KIIs and FGDs—both by hand and based on audio recordings of every interview in which respondents consented to being recorded. The ET transcribed all these notes into Microsoft Word documents and stored them on the SI server to ensure security. The ET discussed the interviews it conducted and recorded emergent themes from these meetings in a findings, conclusions and recommendations (FCR) matrix that resembles a report outline. The ET recoded these emergent themes and transferred them into a tally sheet matrix to count the number of times the same theme occurred (in both KIIs and FGDs), with tallies disaggregated by stakeholder group, geographic area, and sex. The tally sheet matrix, as well as the FCR, were the substrate of this final report. 21 Interest rates and discount rates both estimate time preference; the relative value individuals place on wellbeing and consumption in the present versus the future, a key element in net present value calculation. In a 2018 publication, Cicowiez and Filippo find that “Haiti is a relatively impatient country,” such that “the effective rate of time discount of Haiti is greater than the interest rate” (p. 12). Nominal interest rates in Haiti have been stable at 20 percent since 2016, and a number this high will tend to reduce the net present value of benefits. 22 The decay rate for new jobs will include parameters to account for the extent to which LEVE met its project objectives and the commensurate expected continued impetus on job creation, as well as estimates of average employment duration to generate expected net new job creation in the period after project sunset. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 8 METHODS The ET employed several data analysis methods to identify key findings from the collected data, draw conclusions, and make recommendations for Activity follow-up or future potential programming. The ET utilized the following analysis methods: 1. Cont