Performance Evaluation Report: Local Value Chain Enhancement (LEVE) Activity

PERFORMANCE EVALUATION REPORT LOCAL VALUE CHAIN ENHANCEMENT (LEVE) ACTIVITY JULY 2019 This publication was prepared independently by Social Impact, Inc. at the request of the United States Agency for International Development. PATRICK ADAMS FOR RTI INTERNATIONAL USAID/HAITI LOCAL ENTERPRISE VALUE CHAIN ENHANCEMENT ACTIVITY (LEVE) FINAL PERFORMANCE EVALUATION Final Report USAID/Haiti Office o f Economic G rowth Evaluation Mechanism No: AID-521-A-15-00009 Cover P hoto Credit: R TI International This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Zuri Linetsky (Team L eader), Phanol Philippe (Assistant Team Leader), a nd Paul Rivera (ROI Expert) on behalf of Social Impact, Inc. Contact: Jennifer Mandel, Chief of Party 2300 Clarendon Blvd, Suite 1000 Arlington, VA 22201, USA D ISCLAIMER The authors’ views expressed in this publication do not necessarily r eflect the views o f the United St ates Agency for I nternational Development o r the United S tates Go vernment. ABSTRACT The five-year Local Enterprise and Value Chain Enhancement (LEVE) Activity sought to achieve four objectives across three sectors (agriculture, apparel and textiles , and construction): to enable MSMEs to work together to mutually create value; provide MSME’s with access to a productive and skilled labor pool; improve the sustainability of Haitian businesses benefiting from LEVE; and identify and improve synergies among exis ting activities and interventions. Based on key informant interviews with 45 respondents and four focus group discussions with students at technical and vocational schools that benefited from LEVE support, the Activity was a qualified success. The Activity achieved all four of its objectives in the agriculture and the apparel and textile sectors. However, an issue beyond LEV E’s control – specifically the sector’s informality – hindered its ability to achieve its objectives in the construction sector. LEVE achieved its objectives in two sectors because of its ‘light-touch’ Activity design, which was predicated upon facilitating business linkages, training new laborers, and organizational capacity building. These interventions empowered local business leaders to grow their operations, and, because they do not require direct LEVE support to continue, are likely to be sustainable over the long - term. The Activity’s adaptive management approach was critical to its success; LEVE tailored its interventions to individual businesses as well to the dynamic Haitian business environment. Longitudinal surveys with LEVE beneficiaries will be essential to evaluate the extent of LEVE’s long-term impact and sustainability. CONTENTS TABLES AND FIGURES I ACRONYMS II EXECUTIVE SUMMARY III INTRODUCTION III EVALUATION METHODOLOGY III DATA ANALYSIS III METHODOLOGICAL LIMITATIONS IV FINDINGS AND CONCLUSIONS IV CONCLUSIONS VI CONCLUSIONS VII RECOMMENDATIONS VII INTRODUCTION 1 BACKGROUND OF LOCAL CONTEXT 1 AGRIBUSINESS VALUE SECTOR 2 APPAREL AND TEXTILE SECTOR 2 CONSTRUCTION VALUE CHAIN 3 ACTIVITY OVERVIEW 3 KEY ASSUMPTIONS 3 EVALUATION PURPOSE & QUESTIONS 4 EVALUATION METHODOLOGY 5 DATA COLLECTION 5 LITERATURE REVIEW 5 KEY INFORMANT INTERVIEWS 5 FOCUS GROUP DISCUSSIONS 6 RETURN ON INVESTMENT ANALYSIS 6 DATA ANALYSIS 7 METHODS 8 EVALUATION LIMITATIONS 8 FINDINGS AND CONCLUSIONS 9 EVALUATION QUESTION 1: TO WHAT EXTENT AND IN WHAT WAYS WAS LEVE’S APPROACH SUCCESSFUL IN ACHIEVING ITS FOUR OBJECTIVES? 9 FINDINGS 9 CONCLUSIONS 19 EVALUATION QUESTION 2: TO WHAT EXTENT AND IN WHAT WAYS HAVE LEVE ACTIVITIES RELATED TO THE MAIN VALUE CHAINS GENERATED MULTIPLIER EFFECTS? 20 FINDINGS 20 CONCLUSIONS 26 EVALUATION QUESTION 3: TO WHAT EXTENT AND IN WHAT WAYS HAS LEVE’S FACILITATION APPROACH BEEN EFFECTIVE IN EMPOWERING SUSTAINABLE LOC AL FIRMS/ENTITIES? 27 FINDINGS 27 CONCLUSIONS 33 RECOMMENDATIONS 34 ANNEXES 36 ANNEX A: EVALUATION SCOPE OF WORK 37 ANNEX B: DATA COLLECTION TOOLS 46 KEY INFORMANT INTERVIEW GUIDE FOR LEVE BENEFICIARIES 46 KEY INFORMANT INTERVIEW GUIDE FOR ACTIVITY IMPLEMENTORS (RTI) 49 FOCUS GROUP DISCUSSION GUIDE FOR USAID AND OTHER IPS 52 KEY INFORMANT INTERVIEW GUIDE FOR TECHNICAL AND VOCATION TRAINING CENTERS 58 FOCUS GROUP DISCUSSION GUIDE FOR TVET STUDENTS 60 ANNEX C: SOURCES OF INFORMATION 62 ANNEX D: DISCLOSURES OF CONFLICT OF INTEREST 64 ANNEX E: EVALUATION TEAM MEMBERS 67 i | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV TABLES AND FIGURES TABLE 1: KEY INFORMANT INTERVIEWS BY RESPONDENT GROUP AND LOCATION 6 TABLE 2: BASELINE VALUES FOR ROI ANALYSIS 25 FIGURE 1: RESPONDENT ASSESSMENT OF LEVE’S OVERALL WORK 9 FIGURE 2: RESPONDENT ASSESSMENT OF LEVE’S TRAINING AND CAPACITY BUILDING 10 FIGURE 3: BASELINE AND ALTERNATIVE SCENARIOS FOR LEVE ROI 25 FIGURE 4: LEVE ROI BY QUARTER 26 FIGURE 5: ACCUMULATION OF LEVE COSTS AND JOBS 26 USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | ii ACRONYMS A&T Apparel and Textile ADIH Association des Industries d’Haiti AmCham American Chamber of Commerce CIDA Canadian International Development Agency CLP Chanje Lavi Plante CNAIH Collège National des Ingénieurs et Architectes Haïtiens COP Chief of Party COR Contracting Officer’s Representative EGAD Economic Growth and Agriculture Development ESS Evaluation and Survey Services ET Evaluation Team EQ Evaluation Question FCR Findings, Conclusions, and Recommendations FGD Focus Group Discussion GDP Gross Domestic Product GoH Government of Haiti HACCP Hazard Analysis Critical Control Point HELP Haiti Economic Lift Program HOPE Haitian Hemispheric Opportunity through Partnership Encouragement ICC International Code Council INEPCO Institut National pour le Développement et la Promotion de la Couture INFP National Institute of Vocational Training IP Implementing Partner JAA J.E. Austin and Associates KII Key Informant Interview LEAD Leveraging Effective Application of Direct Investments LEVE Local Enterprise Value Chain Enhancement LOE Level of Effort LOP Life-of-project M&E Monitoring and Evaluation MATCON International Conference On Materials For The Millennium MCI Ministry of Commerce and Industry MEL Monitoring, Evaluation & Learning MSME Micro-, Small, and Medium Enterprise MTPTC Ministry of Public Works, Transport and Communications NGO Non-Governmental Organization PANSEH National Support Program for the Structuring of Haitian Entrepreneurship ROI Return on Investment RTI Research Triangle International SARA Stand-Alone Roads Activity SI Social Impact, Inc. SOP Standard Operating Procedure SOW Scope of Work STTA Short-term Technical Assistance TASC Textile and Apparel Service Center TVET Technical and Vocational Education and Training USAID United States Agency for International Development USD United States Dollars USG United States Government iii | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY INTRODUCTION The United States Agency for International Development (USAID) requested that Social Impact, Inc. (SI)’s Haiti Evaluation and Survey Services (ESS) project design and implement the final pe rformance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity . The primary stakeholders for this evaluation include USAID /Haiti, RTI International, and its principle partners: J.E. Austin and Associates (JAA), Papyrus, TetraTech, Haitian micro-, small, and medium enterprises (MSMEs), other donors supporting MSMEs in Haiti, Haitian private sector entities, and the Government of Haiti (Go H). This evaluation report’s purpose is to explain the evaluation team’s (ET) findings and conclusions regarding LEVE’s ability to meet its four objectives, provide a detailed Return on Investment (ROI) analysis, and clarify the extent to which LEVE’s work will be sustainable in the wake of the Activity’s end. This report also provides actionable recommendations to USAID about follow-on Activities. EVALUATION METHODOLOGY The ET used both qualitative and quantitative methods, including an in-depth desk review of A ctivity reports, focus group discussions (FGDs), key informant interviews (KIIs), and an ROI analysis, using relevant data that LEVE had already collected. 1. Desk Review: The ET conducted a review of documents produced by LEVE and partner institutions to understand the Activity design and implementation, as well as inform data collection. 2. Key Informant Interviews: The ET conducted one-on-one and small group KIIs with 45 individuals in Port -au-Prince, Cap-Haitian, and Saint-Marc using a purposive sampling strategy. 3. Focus Group Discussions: The ET conducted four FGDs with current students of Technical and Vocational Education and Trainings (TVETs). 4. ROI Analysis: The ET conducted an ROI analysis (or, profitability analysis) to calculate the ratio of the net present value of net benefits to the net present value of investment costs for the entire Activity implementation period. DATA ANALYSIS The team took detailed notes of all KII s and FGDs, both by hand and using audio recordings of every interview in which respondents consented to being recorded. The ET transcribed all these notes into Microsoft Word documents and stored them on the SI server to ensure security. The ET reviewed and discussed the interviews it conducted, and recorded emergent themes identified in these meetings in a findings, conclusions and recommendations (FCR) matrix resembling a report outline. The ET transferred coded emergent themes in to a tally sheet matrix to count the frequency with which themes occurred (in both KIIs and FGDs), with tallies disaggregated by stakeholder group, geographic area, and sex. The tally sheet matrix, as well as the FCR, were the substrate of this final report. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | iv METHODOLOGICAL LIMITATIONS The ET encountered several limitations while in the field. Most notably , country-wide protests interrupted fieldwork, which prevented the ET from conducting any face-to-face interviews over a ten- day period. To make up for lost time, the ET conducted several interviews in Port-au-Prince and Saint - Marc by phone, five interviews during the data analysis phase of fieldwork, as well as one interview after the formal out brief with USAID. Protests and political instability prevented the ET from visiting the Saint-Marc region, an area beset by security challenges. As a result of these circumstances, the ET cancelled its trip to the area and conducted all interviews by phone. The ET was also unable to conduct its planned FGD with TVET students in Saint -Marc. The ET also notes four additional limitations that affected this evaluation. First, beyond the sample of people the ET interviewed, t he ET cannot generalize based on the qualitative data it collected for this evaluation. However, the ET compared this data with similar findings from the 2018 Aid for Trade assessment, which ESS also conducted, and the findings were consistent between the two reports, suggesting that the data the ET collected for this evaluation is internally valid. 1 Secondly, and relatedly, since the ET spoke to LEVE beneficiaries, LEVE staff, and implementing partners (IPs), it is possible that social desirability bias (telling the ET what respondents thought it wanted to hear to ensure they might benefit from follow-on USAID Activities) affected respon ses. However, given the alignment of findings between this evaluation, the Aid for Trade assessment, and the ROI analysis, the ET is confident that this form of bias did not notably impact interview responses. Third, the GoH was not an active participant in this evaluation, nor was it regularly involved in the LEVE Activity. Representatives from several key Ministries, including the National Institute of Vocational Training (INFP) and the Ministry of Commerce and Industry (MCI), were unwilling to meet with the ET . While the Ministry of Public Works, Transport and Communications (MTPTC) agreed to meet the ET, it reported no knowledge of the LEVE Activity, likely due to staff turnover. Finally, while LEVE provided it’s the ET M&E data, these were incomplete. By design, LEVE’s M&E systems did not collect the data necessary for the ET to conduct an accurate ROI anal ysis, an issue the ET addressed by approximating, or modeling, some of the key data points for the ROI analysis—much like LEVE conducted with its own ROI. Further, USAID was unable to collect relevant baseline data for LEVE before the Activity began, which the ET would have used to inform its ROI . 2 FINDINGS AND CONCLUSIONS Evaluation Question (EQ)1: To what extent and in what ways was LEVE’s approach successful in achieving its four objectives? The evaluation should specifically examine what factors supported and/or hindered its approach identifying the most promising sectors, the best approach to working with the Government of Haiti, and the private sector. An overwhelming majority of KII and FGD respondents believed LEVE was somewhat-to-very successful in achieving its objectives; over half believed LEVE was very successful. Similarly, over half of interviewees reported LEVE’s training and capacity building interventions (core components of the LEVE 1 Several of the questions were similar, and there were a number of respondents who participated in both evaluations, making a comparison between the two evaluations possible and appropriate. 2 USAID tried but was unable to obtain these data. According to the Economic Growth Office (EGAD), USAID released two separate solicitations for firms to collect the necessary baseline data, but no appropriate vendor could be identified before the LEVE activity began v | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV Activity) were somewhat to very impactful; a majority of these respondents reported LEVE training and capacity building support was very impactful. The Activity was successful in meeting its four objectives. Through its facilitation approach in the apparel and textile (A&T) and agribusiness sectors, LEVE drove synergies and the creation of business linkages among MSMEs, as well as between MSMEs and Lead Firms (Objective 1). Through its training, capacity building and back -office support interventions, LEVE increased the productivity of MSMEs and their staff. By working with TVETs, LEVE also improved the quality of new workers that were entering the A&T and construction value chains (Objective 2). A clear majority of interviewees said that LEVE’s interventions will be sustainable in the A&T and agricultural sectors (Objective 3), specifically because LEVE’s design underscored facilitation, helping to establish business linkages, drive business empowerment, and diversify existing company’s businesses. Finally, both LEVE staff and Activity beneficiaries noted that they worked effectively with other donors and USAID Activities to empower local actors (Objective 4). For example, LEVE improved Papyrus’ capacity as an implementing partner (IP). Unfortunately, due to systematic challenges in the construction sector, many of which were beyond LEVE’s control, including changes in USAID staffing and Activity implementation decisions, LEVE did not meet Objectives 1 and 4 in the construction sector. However, LEVE did, despite major challenges , develop a new plumbing code for Haiti, and contribute to training and placing more skilled workers in the construction sector. LEVE also supported Technical and Vocational Schools to train female welders and plumbers in the North, and solar panel technicians in Port-au-Prince (Objective 2). Due to their focus on building training capacity, these interventions are likely to be sustainable (Objective 3). Interviewees noted that future USAID Activities should consider identifying niche areas in which to build Haitian labor capacity to produce value added goods. Finally, interviewees reported that the most effective approach for working with the GoH is to build personal relationships with technical staff in Ministries relevant to any intervention, include these staff in the design of every USAID intervention, and maintain this relationship over the life of an Activity. CONCLUSIONS • LEVE was successful in achieving the four objectives of its facilitation approach, which allowed it to contribute to building business linkages, support t raining and capacity building for MSMEs, and empower MSMEs. • LEVE’s approach and the challenges it encountered working with the GoH highlighted a pathway toward a successful working relationship; new Activities must identify a consistent champion among technical staff in relevant Ministries with whom to cultivate an enduring personal relationship. • New Activities must help build a business-empowerment culture in the GoH, which many interviewees believe is lacking. LEVE, at least in the A&T sector, accomplished this due to the personal relationships between its staff and GoH officials. EQ 2: To what extent and in what ways have LEVE activities related to the main value chains generated multiplier effects? The evaluation should conduct a return on investment analysis using LEVE’s existing data. Then the evaluation should examine the multiplier effects, if any, that reflect the return on investment. The ROI analysis showed that LEVE’s job creation benefits must continue for a minimum of six quarters beyond the end of the A ctivity to achieve a break- even ROI value of 1.0. For the Activity USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | vi implementation period, the ET’s analysis of the direct job creation benefits yielded a baseline ROI value of 0.424 for the apparel sector, reflecting 99 percent of LEVE’s job creation results. LEVE did not create jobs in the agribusiness and construction sectors at the scale necessary for the ET to generate a credible ROI; thus, the ET excluded job creation in these sectors from its analysis. However, if job creation benefits in the apparel sector continue through the end of the Haiti Economic Lift Program (HELP) and Haitian Hemispheric Opportunity through Partnership Encouragement (HOPE) Acts in 2025, LEVE will yield a direct ROI value of 1.533. In the absence of data on LEVE’s secondary impacts, the ROI analysis alternatively utilized a range of income multiplier values from a previous apparel sector study in Haiti to account for indirect an d induced benefits originating from LEVE's job creation interventions. When the ET used the multiplier results in a baseline ROI value of 0.543 for the Activity implementation period, ROI increased to 1.947 when projected to 2025. The study modeled scenarios utilizing a wide range of values for each relevant parameter, which never resulted in an ROI value above 0.70 during the Activity implementation period. When the study expanded benefits through 2025, the ET recorded ROI values above 1.30 for all justifiable variations of the model parameters. CONCLUSIONS • The ROI analysis may not accurately represent LEVE's impact or success. While the ROI analysis demonstrated the positive and increasing benefits of LEVE's job creation interventions over time, the time-to-build dimension of job creation and the necessary front-loading of costs created a downward bias in LEVE's ROI values, as the methodology placed greater emphasis on earlier events. • The main driver of LEVE’s long-term ROI is the continuity of benefits beyond the Activity implementation period. Elements the ROI analysis did not measure, like sustainability, network linkages, and technical assistance, will allow LEVE’s benefits to persist and expand over time— the long-term ROI. • For the ET to produce an accurate ROI, LEVE required a more rigorous and complete M&E system. EQ 3: To what extent and in what ways has LEVE’s facilitation approach been effective in empowering sustainable local firms/entities? Among other empowerment criteria, the evaluation should consider the level of leadership taken by the firms in the undertaken activities and success in terms of sales, revenue and job creation in the analysis. As detailed in EQ 1, LEVE’s work with TVETs was a key factor in empowering sustainable local firms and entities. More than a third of interviewees said that LEVE support to TVETs was positive -to-very positive. Most TVET respondents, including both directors and students, reported satisfaction with the training and job placement services they received, all of which LEVE facilitated. Additionally, LEVE pushed TVETs to operate more like businesses to fund their operations, given that state funding for technical and vocational schools was inadequate. This operational change may make TVETs more sustainable in the long-term. In terms of general empowerment, almost half of interview respondents said that LEVE beneficiaries were empowered to be leaders in their value chains and/or sectors. LEVE achieved this empowerment through local connections and business linkages, back -office support, access to capital, access to machinery, business diversification, and capacity development. These interventions were sustainable, as evidenced by beneficiary businesses that are currently replicating LEVE practices despite LEVE’s closure. The linkages that LEVE provided to short-term technical assistance (STTA) were also a critical and sustainable source of integration and support; firms that can afford it can continue to contract with STTA and maintain and/or expand upon the relationships that LEVE helped them establish. Respondents vii | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV also noted that LEVE interventions helped them manage through enduring political instability in Haiti, and even increased their business revenues. In addition to chronic political instability, there are two major challenges to business sustainability in Haiti: 1) A lack of access to credit at competitive interest rates; and, 2) LEVE’s closing and cessation of facilitation and technical assistance, which interviewees stated could limit/slow business growth in the short-term. Worker training in the A&T sector will potentially present the most pressing gap now that LEVE has ended, as LEVE was instrumental in maintaining Association des Industries d’Haiti ( ADIH) and its vocational training center, Textile and Apparel Service Center’s (TASC’s) training capacity. CONCLUSIONS • LEVE’s interventions successfully empowered local businesses. • LEVE’s interventions were proven sustainable because they focused on developing systems and practices that LEVE beneficiaries are currently replicating and expanding in the absence of LEVE. • Current business linkages among LEVE beneficiaries after LEVE has closed, as well as ties to technical assistance providers, demonstrate that LEVE’s interventions are likely to be sustainable. • Those TVETs that can identify sources of independent funding (like producing and selling goods from the schools), and those that demonstrate the ability to place students in relevant industry jobs, are more likely to be sustainable businesses. RECOMMENDATIONS • Follow-on Activities should continue building on LEVE’s training and back -office support. • Follow-on Activities should follow LEVE’s light-touch facilitation approach. • Follow-on Activities should prioritize building personal relationships within the GoH at the technical- office level. • While there are significant gaps in construction, future Activities must identify capacity building niches for Haitian business and labor. • Any follow-on Activity will require more robust M&E systems, mechanisms for collecting ROI- relevant data from Activity outset, as well as a commitment from USAID to deliver baseline data, especially if USAID will require data -intensive analysis (like ROI). • USAID should commission a survey of LEVE beneficiaries, such as individuals who acquired new jobs or improved jobs, o ver the next five years to evaluate the Activity ’s long-term impact. 1 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV INTRODUCTION The United States Agency for International Development (USAID)/Haiti requested that Social Impact, Inc. (SI)’s Haiti Evaluation and Survey Services (ESS) design and implement the final Performance Evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. The primary stakeholders for this evaluation include USAID /Haiti, RTI International, and its principle partners: J.E. Austin and Associates, Papyrus, TetraTech, Haitian micro-, small, and medium enterprises (MSMEs), other donors supporting MSMEs in Haiti, Haitian private sector entities, and the Government of Haiti (GoH). The purpose of this final performance evaluation report was to present findings, conclusions and recommendations based on the evaluation team’s (ET’s) review of Activity documentation, as well as data gathered through key informant interviews (KIIs), focus group discussions (FGDs), and a return on investment analysis (ROI). This report deta ils the methodology the ET used to conduct fieldwork and data analysis, the ET’s limitations and mitigation measures, and evaluation findings, conclusions and recommendations . BACKGROUND OF L OCAL CONTEXT As a result of political instability, trade embargos, and environmental catastrophes , Haiti has suffered through years of limited economic development, which has played a critical role in preventing the GoH from addressing domestic social and economic challenges. 3 Prior to the 2010 earthquake, Haiti faced high levels of unemployment, extreme income inequality, and a limited tax base. The earthquake itself destroyed infrastructure, human capital, inventory and collateral—the assets underpinning Haitian businesses and, therefore, the entire Haitian economy—fundamentally undermining national economic stability. As of April 2019, more than 96 percent of the Haitian population has been exposed to natural disasters, including hurricanes, floods, and earthquakes. Currently, the Haitian economy provides only limited livelihood opportunities for its citizens. Haiti has a Gross Domestic Product (GDP) per capita of $870, and ranks 168 of 188 countries on the United Nation’s Development Program’s human development index. 4 An estimated six million Haitians live under the national poverty line of $2.41 United States Dollars (USD) per day, and more than 2.5 million people live below the extreme poverty line of $1.239 USD per day. 5 The United States has also reduced its aid budget for Haiti since 2017, and the Temporary Protected Status for Haitians (people allowed to live and work in the United States after war or a major natural disaster in their own countries) is set to end in January 2020. 6 Haitian politics contribute to local instability. The current Pres ident of Haiti was inaugurated on February 7, 2017, which r eturned the country to political order after Haiti had been without an elected 3 Daniel Runde, “Haiti’s Governance Problems Require Touch Love From Donors,” Forbes, April 17, 2015, accessed May 7, 2019, https://www.forbes.com/sites/danielrunde/2015/04/17/haitis-governance-problems-require-tough-love-from- donors/#13657df82917. 4 United Nation Develop Programs, “Human Development Report: Haiti,” accessed May 19, 2019, http://hdr.undp.org/en/countries/profiles/HTI. 5 The World Bank, “The World Bank in Haiti: Overview,” April 5, 2019, accessed May 7, 2019, https://www.worldbank.org/en/country/haiti/overview . 6 Maureen Taft-Morales, “Haiti’s Political and Economic Conditions: In Brief, “ Congressional Research Service,” December 1, 2017, accessed May 6, 2019, https://fas.org/sgp/crs/row/R45034.pdf. See also, Franco Ordeñez and Jacqueline Charles, “Trump Administration Extends TPS for Haitian and Three other Groups,” McClatchy Dc Bureau, March 1, 2019, accessed May 7, 2019, https://www.mcclatchydc.com/news/politics-government/white-house/article226943724.html. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 2 President for a year. The new President, Jovenel Moïse, was elected with only 21 percent of the vote, meaning he had almost no political mandate. His administration has since seen a cholera outbreak, stagnant economy, enduring political protests caused by a poor economy, and public calls for an end to government corruption. Moïse arrived under the shadow of an ongoing Hai tian government investigation into his possible involvement in money laundering and irregular loan arrangements, allegations Moïse has denied. In May 2019, Haiti’s Superior Court of Auditors and Administrative Disputes alleged that Moïse and other officials embezzled millions of dollars. 7 These corruption allegations led to protests and riots that affected the ET’s ability to collected data for this evaluation. AGRIBUSINESS VALUE SECTOR Agribusiness is a critical business sector in Haiti and can be an engine for national economic growth. According to the Haiti Statistics and Informatics Institute, nearly 50 percent of the Haitian population work as farmers or skilled laborers in agriculture or fishing. 8 This represents roughly a ten percent decrease from the 2008 census. There are an estimated one million farmers in Haiti, each working an average of 1.5 hectares (ha) of land, which accounts for some 25 percent of national GDP . Haiti’s agriculture sector supplies 50 to 60 percent of the food consumed in Haiti. 9 Public or private institutions have never invested enough in this sector to reach the level required to establish a national industrial agriculture foundation. Historically, Haitian farmers have undertaken exploitative agriculture, which has eroded the very agricultural lands that the population needs to survive. Due to various political, economic, and social crises, the GoH has been forced to import large quantities of often-subsidized agricultural goods, making Haiti “one of the most open agricultural sectors in the world.” 10 Competition between domestic producers and cheaper imported agricultural products, as well as the GoH’s lack of planned and systematic investment in the agricultural sector, has hindered the development of a more sustainable and competitive agricultural sector. 11 APPAREL AND TEXTILE SECTOR Overall, the apparel and textile (A&T) sector accounted for more than 50,000 jobs in January 2018 and produced more than $1 billion worth of exports in 2015. 12 There are two aspects of the apparel and textile sector in Haiti: M ass industrial production, and small-scale stitchers and designers. The mass industrial production value chains employ the majority of labor in this sector, accounting for more than 30,000 jobs. 13 Several small sewers and designers make goods for the uniform market (schools, etc.), designer haute couture, tourist wear, and costumes for Haitian celebrations like carnival. A dditionally, several companies work for high-end tailors. The smaller-scale sub-sector is often only semi- commercial, and sells products by attaching ethical or sympathy (charity) labels to its goods. 14 A key factor facilitating the expansion of the Haitian apparel and textiles sector is the Haitian Hemispheric Opportunity through Partnership Encouragement (HOPE)/ Haiti Economic Lift Program (HELP) legislation, which allows Haitian producers to obtain duty-free preferences in the Unites States. 7 Congressional Research Service, “Haiti’s Political and Economic Conditions,” July 1, 2019, https://fas.org/sgp/crs/row/R45034.pdf, p. 1. 8 Haitian Institute of Statistics and Informatics, “Economic Characteristics: Main Occupation,” Ministry of Economics and Finances, 2016, accessed May 8, 2019, http://www.ihsi.ht/rgph_resultat_ensemble_CE.htm . 9 USAID/Haiti, “Value Chain Assessment Annex 3. Agribusiness Sector Assessment: Local Enterprise and value Chain Enhancement (LEVE) Project,” April 2014, p. 5. 10 Ibid, p. 5. 11 Ibid, p. 5. 12 These are the most up to date data. Haiti Open Data, “Textiles and Apparel Industry,” accessed May 8, 2019, http://opendata.investhaiti.ht/pulbare/textiles-and-apparel-industry . 13 USAID/Haiti, “Value Chain Assessment Annex 1. Apparel and Textiles Sector Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. 2. 14 Ibid. 3 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV However, while Haiti’s apparel and textile industry has grown, growth has been slow and localized to low value-added goods like T-shirts and other cut-and-sew products. CONSTRUCTION VALUE CHAIN The construction value chain in Haiti encompasses two main parts: buildings (residencies, hospitals, schools, hotels, restaurants, etc.) and infrastructure (roads, bridges, tunnels, power lines, pipelines etc.). These sectors possess a boom-or-bust quality due to the enduring threat of natural disasters like the 2010 earthquake. As with Haiti’s agricultural sector, lack of GoH investment in infrastructure development is caused by lack of borrowing capacity and high interest rates. The lack of GoH dollars precludes large-scale investment in major public works projects. 15 USAID/Haiti’s 2014 Value Chain Assessment concluded, “…The construction industry cannot rely on public works as a major source of growth, except for projects such as road paving, schools, and hospitals, which traditionally grow at a steadier rate. In most developing countries, public works infrastructure is a key job creator for the construction industry sector.” 16 There is also variation in construction sector capacity within Haiti; in Cap-Haitian, the sector is much less developed than in the capital. However, in Cap-Haitian, tourism- related building, new hotel rooms, and the international airport, as well as the continued growth of beach resorts and the expansion of the industrial park, drive sector growth. 17 ACTIVITY OVERVIEW Limited economic development has long hindered Haiti from addressing social and economic issues. Even before the January 2010 earthquake, Haiti faced high levels of unemployment, income inequality, and a limited tax base. The earthquake exacerbated these problems, destroying infrastructure, human capital, and inventory, further increasing economic instability. In response, USAID has supported Haiti’s economic development by rebuilding damaged infrastructure, promoting policies that lead to a business- friendly environment, expanding access to financial products, and aiding MSMEs. USAID launched the 34,265,488 USD LEVE Activity (contract number: AID-521-C-14-00001) to increase economic security. USAID designed LEVE to create jobs in target industries/sectors in and around Port-au-Prince, Saint-Marc, and Cap-Haitian by creating more inclusive and productive value chains. Implemented by Research Triangle Institute International between December 2013 and June 2019, LEVE targeted MSMEs to help improve and enhance their capacity and worked with technical and vocational schools to assist them to improve their courses and job placement processes. In Port-au- Prince, LEVE assisted all three of its target sectors: agribusiness, construction, and the A&T industry. In and around St. Marc, LEVE focused particularly on the agribusiness sector, and in the Cap- Haitian corridor, it assisted the construction and A&T sectors. LEVE’s Development Hypothesis state d that if Haitian MSMEs in high potential sectors engaged with other value chain actors to mutually create value in response to market demand, and if they were supported by a productive labor pool with skills and competencies relevant to the target sectors, then those value chains would be more inclusive and productive, thereby leading to increased job creation in Haiti. KEY ASSUMPTIONS 15 USAID/Haiti, “Value Chain Assessment Annex 2. Construction Industry Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. 5. 16 Ibid. 17 Ibid. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 4 The LEVE Activity made several assumptions in its design. First, the Activity assumed that these three sectors (agribusiness, construction, and the A&T industry) could support sustainable long-term employment for Haitian workers. LEVE made this assumption based on the idea that these are “high- potential sectors” capable of generating broad-based employment in the three development corridors supported by the United States Government (USG). 18 Second, the Activity assumed that the raw materials necessary to allow these sectors to grow, and thereby generate large- scale employment, were readily available. Relatedly, the Activity assumed that there was interest from the GoH and the sectors themselves in creating more employment opportunities. Third, the Activity assumed that working with Lead Firm s in certain sectors and training to produce a more skilled labor force would produce more inclusive and higher-quality job creation – i.e., a trickle- down effect for all sectors that LEVE covered. To address the Haitian employment context in the three sectors of interest through the three development corridors, LEVE established four key indicators of success: 19 • Increase in sales (volume and USD value) in target value chains; • Percentage or USD value increase in investment in target value chains ; • Percentage increase in productivity (modified to fit each targeted value chain); and • Number of new jobs (full-time equivalents) created in target sector in designated corridors. USAID estimated during the design phase that LEVE would create 8,000 – 14,000 jobs (as measured by full time equivalents (FTEs)) during the Activity’s three-year base period; the Mission later reduced this figure to 6,000. The Activity also initially planned to increase the investments of 155 businesses. EVALUATION PURPOSE & QUESTIONS The purpose of this final performance evaluation was to orient USAID /Haiti’s new generation of support to MSMEs by analyzing how successful LEVE was in achieving results relative to each of the A ctivity’s components. This final evaluation answers the evaluation questions (EQs) below: 1. To what extent and in what ways was LEVE’s approach successful in achieving its four objectives? The evaluation should specifically examine what factors supported and/or hindered LEVE’s approach to identifying the most promising sectors and what factors determined the best approach to working with the GoH and the private sector. 2. To what extent and in what ways have LEVE activities that are related to the main value chains generated multiplier effects? To answer this question the evaluation should conduct a return on investment (ROI) analysis using LEVE’s existing data. Then the evaluation should examine the multiplier effects, if any, that reflect the ROI. 3. To what extent and in what ways has LEVE’s facilitation approach been effective in empowering sustainable local firms/entities? Among other empowerment criteria, the evaluation should consider the level of leadership taken by the firms and their success in terms of sales, revenue, and job creation. 18 LEVE contract, section C. 19 LEVE contract, section C, p. 12. 5 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV EVALUATION METHODOLOGY The ET used a mixed-method, primarily qualitative, evaluation design consisting of four principal data collection methods: Document review, KIIs, FGDs, and an ROI analysis. The team assessed LEVE’s quantitative data through a review of the Activity’s performance monitoring documents (e.g., original and updated versions of LEVE’s M&E Plan, indicator tracking tables in its quarterly and annual reports), and an ROI analysis (see below). 20 The ET triangulated this quantitative data with qualitative data collected through its KIIs, FGDs and review of additional background documents. DATA COLLECTION LITERATURE REVIEW The ET reviewed documents LEVE and partner institutions produced to understand LEVE’s design and implementation, extract findings relevant to the EQs, and inform data collection protocol to develop instruments that appropriately supplemented or cross-checked information in background documents. The ET used the following categories of documents; Annex B contains full descriptions of documents: • Annual Reports: FY14 - FY18 • Quarterly Reports: FY14 – Q2 and Q3; FY15 – Q1, Q2, and Q3; FY16 – Q1, Q2, and Q3; FY 17 – Q1, Q2, and Q3; FY 18 – Q1, Q2, and Q3; FY19 – Q1 • M&E and work Plans: FY 14 - 19 • LEVE workforce assessment report • LEVE Apparel and Textile, Agribusiness and Construction sector value chain assessments • Data used for the RTI International’s internal ROI Analysis KEY INFORMANT INTERVIEWS The ET used a purposive sampling approach to identify candidates for KIIs and ensured balanced geographic, sector, and gender representation to the extent possible. The ET identified an initial list of respondents from LEVE documentation, which LEVE staff supplemented by LEVE staff to help fill any potential data gaps, as well as obtain relevant contact information. The ET conducted KIIs with 45 key stakeholders including 28 males and ten females in Port-au-Prince, four males in Cap-Haitian, and two males and one female in Saint-Marc. KII participant groups included LEVE beneficiaries in the construction, agribusiness, and apparel sectors, with an emphasis on construction sector actors due to LEVE’s limited success i n this sector, TVET officials, LEVE and other implementing partners (RTI International, Papyrus), officials from relevant GoH agencies, and USAID/Haiti. 20 The ET acknowledges the assertions within the recent USAID/OIG report about flaws and omissions in LEVE’s performance data, some of which the ET discusses above (see preliminary findings for EQ 1, information gaps under EQ 3). While the it approached the data with these assertions in mind, the ET found it necessary to analyze this data not only to validate (or invalidate) the issues detailed in the OIG report, but also to utilize validated performance data to triangulate these results with its qualitative (e.g., KII, FGD) findings. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 6 TABLE 1: KEY INFORMANT INTERVIEWS BY RESPONDENT GROUP AND LOCATION RESPONDENT GROUP PORT-AU-PRINCE SAINT MARC CAP HAITIAN LEVE BENEFICIARIES Construction 4 0 0 Agribusiness 5 3 0 Apparel & Textiles 7 0 1 TVETs (one non-beneficiary) 3 1 3 USAID 2 0 0 IMPLEMENTING PARTNERS 12 0 0 GoH 4 0 0 KII TOTAL (45) 37 4 4 FOCUS GROUP DISCUSSIONS The ET con ducted four FGDs with current students and recent graduates of LEVE-supported TVETs , including two in Port-au-Prince and two in Cap -Haitian. The ET sampled FGDs participants utilizing a convenience sampling strategy. At school visits, the ET selected FGD participants by as king TVET administrators to identify six to ten students who were beneficiaries of LEVE support; because TVET identified these students, it is possible that social desirability bias affected evaluation findings. Of the four FGDs, one contained entirely female respondents, while the three others included both men and women. The ET designed FGDs to gather information from current and recent students about the ir challenges acquiring the skills necessary to secure and maintain sustainable employment in Haiti’s job market, and the degree to which LEVE’s TVET support had provided them with skills and connections to companies seeking employees with those skills. The ET developed data collection protocols for FGDs (see Annex C), like those for KIIs, that both address ed and were derived from the EQs . RETURN ON INVESTMENT ANALYSIS ROI analysis involves two primary components: effectiveness and profitability. Effectiveness refers to an assessment of the extent to which an activity or investment met its objectives. The ET’s response to EQ 1 fully addresses effectiveness, and for methodological purposes, the ROI analysis included here is synonymous with profitability analysis. The ET calculated ROI as a simple ratio of net benefits to investment costs. To accurately determine benefits, LEVE needed to provide quarterly data disaggregated by sex and sector on: 1) the number of new jobs LEVE created among the target beneficiary population as a result of the Activity; 2) measures of beneficiary wages; 3) beneficiaries’ time worked per week or month as appropriate; and, 4) employment duration. However, LEVE only provided data on job creation. Cost information came from data on USAID disbursements for LEVE. These data included the total amount disbursed by quarter, and account for all costs LEVE incurred to implement interventions. To standardize the value of benefits and costs over time, the ET selected an appropriate discount rate. For activities where beneficiaries were private individuals, discount rates closely paralleled prevailing interest rates. As a conservative estimate, the ET applied to the ROI the historical average of interest 7 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV rates in Haiti during the 2014- 2019 period of Activity implementation and adjusted all monetary values to net present value at the time of Activity initiation. 21 The ET then calculated ROI as the rati o of the net present value of net benefits to the net present value of investment costs for the entire period of Activity implementation. Based on the results of EQ 1 , the ET also deemed it appropriate to include a forward estimation of benefits based on average job creation by firm with an appropriate decay rate 22 for new job creation after Activity sunset. The ET similarly converted these additional estimated benefits to net present value to calculate expected ROI. The ET verified the robustness of results via sensitivity analyses by varying the effects on calculated ROI of changing key parameters within the model. MULTIPLIER ANALYSIS: In applied development work, analysis that includes spillover effects, like a multiplier, increases the likelihood of introducing subjectivity into analytical results and weakens the reliability of attribution between Activity interventions and purported results. Once analysis hypothesizes Activity impacts beyond the target beneficiaries, its scope is no longer exclusively about job creation, as in the case of LEVE, and analysis must then consider a broad set of potential secondary effects. Spillover effects are, thus, not typically part of ROI analysis, and the ET does not recommend them in this case. EQ 2 nonetheless specifies an analysis of multiplier effects due to LEVE Activities. Without performance data to support LEVE's spillover effects, the ET carried out multiplier analysis utilizing income multipliers adopted from a previous study on the Haitian apparel sector. The validity of the ROI multiplier results thus depend on the relevance of the utilized income multipliers. Methodologically, the multiplier functioned as an inflation factor to the net direct benefits of the ROI calculation, and the ET incorporated adjusted values into the standard equation. The ET reports the generated ROI value as an estimate, noting the diminished certainty of attribution stemming from hypothesized parameter estimates. DATA ANALYSIS The team took detailed notes of all KII s and FGDs—both by hand and based on audio recordings of every interview in which respondents consented to being recorded. The ET transcribed all these notes into Microsoft Word documents and stored them on the SI server to ensure security. The ET discussed the interviews it conducted and recorded emergent themes from these meetings in a findings, conclusions and recommendations (FCR) matrix that resembles a report outline. The ET recoded these emergent themes and transferred them into a tally sheet matrix to count the number of times the same theme occurred (in both KIIs and FGDs), with tallies disaggregated by stakeholder group, geographic area, and sex. The tally sheet matrix, as well as the FCR, were the substrate of this final report. 21 Interest rates and discount rates both estimate time preference; the relative value individuals place on wellbeing and consumption in the present versus the future, a key element in net present value calculation. In a 2018 publication, Cicowiez and Filippo find that “Haiti is a relatively impatient country,” such that “the effective rate of time discount of Haiti is greater than the interest rate” (p. 12). Nominal interest rates in Haiti have been stable at 20 percent since 2016, and a number this high will tend to reduce the net present value of benefits. 22 The decay rate for new jobs will include parameters to account for the extent to which LEVE met its project objectives and the commensurate expected continued impetus on job creation, as well as estimates of average employment duration to generate expected net new job creation in the period after project sunset. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 8 METHODS The ET employed several data analysis methods to identify key findings from the collected data, draw conclusions, and make recommendations for Activity follow- up or future potential programming. The ET utilized the following analysis methods: 1. Content Analysis: The ET intensively reviewed and systematically coded KII and FGD transcripts to identify and highlight the existence of certain concepts and their frequencies, within the data. The ET then summarized these standardized data to better understand contributions of LEVE activities to its objectives. 2. Gap Analysis: The ET utilized gap analyses to examine whether any aspects of LEVE fell short of anticipated performance, and the likely factors that contributed to these gaps. 3. Gender Analysis: The ET’s social analysis included gender-based findings and conclusions. The team worked across LEVE objectives to capture and compare Activity results as they specifically benefitted (or did not benefit) women and men. EVALUATION LIMITATION S The ET encountered several limitations while in the field. Most notably, country -wide protests interrupted fieldwork, which prevented the ET from conducting any face-to-face interview interviews over a ten- day period. To make up for lost time, the ET conducted several interviews in Port-au-Prince and Saint-Marc by phone, five interviews during the data analysis phase of fieldwork, as well as one interview after the formal out brief with USAID. Protests and political instability prevented the ET from visiting the Saint-Marc region, an area beset by security challenges. As a result of these circumstances, the ET cancelled its trip to the area and conducted all interviews by phone. The ET was also unable to conduct its planned FGD with TVET students in Saint-Marc. The ET also notes four additional limitations that affected this evaluation. First, beyond the sample of people the ET interviewed, the ET cannot generalize based on the qualitative data it collected for this evaluation. However, the ET compared this data with similar findings from the 2018 Aid for Trade assessment, which ESS also conducted, and the findings were consistent between the two reports, suggesting that the data the ET collected for this evaluation is internally valid. 23 Secondly, and relatedly, since the ET spoke to LEVE beneficiaries, LEVE staff, and implementing partners (IPs), it is possible that social desirability bias (telling the ET what it wants to hear in the hopes of benefiting from future interventions) affected responses. However, given the alignment of findings between this evaluation, the Aid for Trade assessment, and the ROI analysis, the ET is confident that this form of bias did not notably impact interview responses. Third, the Government of Haiti (GoH) was not an active participant in this evaluation, nor was it regularly involved in the LEVE Activity. Representatives from several key Ministries, including the National Institute of Vocational Training (INFP) and the Ministry of Commerce and Industry (MCI), were unwilling to meet with the ET. While the Ministry of Public Works, Transport and Communications (MTPTC) agreed to meet the ET, it reported no knowledge of the LEVE Activity, likely due to staff turnover. 23 Several of the questions were similar, and there were a number of respondents who participated in both evaluations, making a comparison between the two evaluations possible and appropriate. 9 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV Finally, while LEVE provided its M&E data to the ET, these data were incomplete. By design, LEVE’s M&E systems did not collect the data necessary for the ET to conduct an accurate ROI analysis, an issue the ET addressed by approximating, or modeling, some of the key data points for the ROI analysis—much like LEVE conducted its own ROI. Further, USAID was unable to collect relevant baseline data for LEVE before the Activity began, which the ET would have used to inform its ROI. 24 FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1: T O WHAT EXTENT AND IN WHAT WAYS WAS LEVE’S APPROACH SUCCESSFUL IN ACHIEVING ITS FOUR OBJECTIVES? FINDINGS A clear majority of respondents believed that LEVE was somewhat-to-very successful in achieving its objectives. 25 Respondents cited perceived success based on LEVE’s job creation for both MSMEs and Lead Firms, as well as increased revenue for MSMEs. 26 A large producer based in Cap -Haitian reported that LEVE played a critical role in the organization’s ability to hire and train 1,074 employees. According to this manufacturer, LEVE staff worked to resolve issues the company had with Haitian customs, provided a large block grant for the recruitment of new labor, as well as short -term technical assistance (STTA) to train new staff to meet the firm’s production standards. A smaller high-end manufacturer based in Port-au-Prince reported that due to LEVE’s support, it was able to source all the materials necessary to produce its goods in Haiti, cut production costs by a factor of three, and despite the political instability that limited domestic orders, continue to increase its revenue. A USAID official reported that the Activity was successful not only because it was able to create job s for MSMEs, Lead Firms, and increase the productive capacity of Haitian companies, LEVE was also effective in changing the mindset of businesses in Haiti, helping Haitian companies work together, build economies of scale, and create more value-added products. Moreover, this official noted that LEVE’s design allowed it to be respon sive to Haiti’s changing business environment and political situation. 24 USAID tried, but was unable to obtain these data. According to the Economic Growth Office (EGAD), USAID released two separate solicitations for firms to collect the necessary baseline data, but USAID could not identify an appropriate before the LEVE Activity began. 25 The ET asked interviewees to assess LEVE’s work in the sector most pertinent to their work, based on their experience working with LEVE. Respondents were asked to assess LEVE on a five-point Likert scale (very successful, somewhat successful, neutral, somewhat unsuccessful, very unsuccessful). These data provide systematic and comparable data points because the ET can assume that most people interviewed for this final performance evaluation share relatively common conceptions of what “successful” means. 26 The LEVE contract, Section C defined Lead Firms in 7 ways: (1) commercial linkages to many MSMEs; (2) financial strength to invest in or dedicate resources to expanding relations with MSMEs; (3) willingness to invest in or expand relations with MSMEs; (4) demand for their services; (5) ability to compete successfully in end markets for their products or services; (6) potential to influence other actors in their value chain; and, (7) acceptable track record and reputation as a business, p 18. 1 3 15 19 UnsuccessfulNeutralSomewhat Successful Very Successful How do you assess LEVE’s work? FIGURE 1: RESPONDENT ASSESSMENT OF LEVE’S OVERALL WORK USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 10 Three neutral respondents included an IP, a GoH official, and a TVET official. The IP and TVET respondents did not benefit from LEVE; the former said he was “not especially aware of LEVE ,” while the latter hoped to benefit from LEVE, but the assistance he required (millions of USD ) was not feasible from LEVE’s perspective. The GoH official reported LEVE was very active in the apparel and textiles (A&T) sector , but the Activity had not helped his office implement any studies on how to “take full advantage of the HOPE law.” Since a critical component of LEVE’s programming was focused on training workers in A&T, agriculture, and construction, as well as building Haitian businesses’ capacity, the ET asked interviewees to assess the impact of LEVE’s training and capacity building work. Figure 2 presents all categories interviewees chose. 27 The ET also asked respondents to rate perceived LEVE impact with a Likert scale question. Over half of interviewees said that the training and capacity building LEVE provided was somewhat-to-very impactful (respondents included GoH, actors from all three sectors, IPs, and USAID). One LEVE beneficiary from the agribusiness sector said LEVE’s training and capacity building was “very impactful because I did not spend all of my adult life in Haiti. I was not aware of the tax conditions in Haiti. LEVE put me in touch [with STTA] to learn better accounting practices, which helped me a lot.” The above discussion highlights respondents’ overarching reactions to LEVE and its interventions. The ET presents below a more detailed account of findings and conclusions for each of LEVE’s four objectives, addressing how and to what extent the Activity achieved success. Objective 1: Enable MSMEs to engage with other value chain actors to mutually create value Over half of the ET’s respondents characterized LEVE as a facilitation Activity that built linkages and synergies within the Haitian business community. Actors in the A&T and agribusiness sectors mutually added value through these business linkages. KII respondents often cited the example of the relationship LEVE established between Delice Tropical (jam and cookie producer) and SISA (pepper sauce maker). A representative from an agri -transformation business said, “LEVE created a network. A synergy developed between Delice Tropical and SISA… [they] were able to trade raw materials… at the jam level. SISA rejects some agricultural goods (like lime rinds) that Delice recovers and uses. This partnership has been win- win. There was information sharing also at the level of raw material suppliers.” LEVE drove similar network connections in the rice value chain, according to three KII interviewees. LEVE connected rice value chain actors, allowing these actors to build a relationship with the World Food Program, which ultimately sourced rice from Haitian growers. LEVE also connected the Movement des Planteurs de Bas Artibonite with Moulin Nan Noté—who also benefited from LEVE help with 27 The ET asked interviewees to assess LEVE’s training and accepted building work, based on their experience working with LEVE. Respondents were asked to assess LEVE on a five-point Likert scale (very positively impactful, positively impactful, no impact, negative impact, very negative impact). These data provide systematic and comparable data points because the ET can assume that most people interviewed for this final performance evaluation share relatively common conceptions of what “successful” means. 5 22 Somewhat ImpactfulVery Impactful How do you assess LEVE's training and capacity building? FIGURE 2: RESPONDENT ASSESSMENT OF LEVE’S TRAINING AND CAPACITY BUILDING 11 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV parboiled rice machinery—and the Association des Irrigants de Bas Maitre Rive D roite. The USAID LEAD Activity initiated this work. 28 The connections among these three actors facilitated the purchasing and refinement of locally grown rice by Moulin Nan Noté for local consumption. Beyond agri-transformation, LEVE worked with small designers in the A&T sector to help firms source materials locally. Owners of a firm producing luxury footwear cited the effect of this work, one of whom reported that due to LEVE, her company began sourcing all its raw materials in Haiti, navigating the political challenge s in Haiti and growing its revenue. “ We used to import material s, which cost us three times more than local production. LEVE allowed us to make… [all parts of our product] in Haiti. LEVE gave us capacity to transform raw materials to make a complete sandal [starting with the sole]. They allowed us access raw materials. It costs us less and we became more independent.” Small designers in different value chains in the A&T sector were also able to connect through LEVE’s facilitation events, according to LEVE staff, and began to work together, identifying avenues to create economies of scale to mutually benefit their production of goods. One of the long-term impacts of LEVE’s work in training and capacity building was building linkages between MSMEs and Lead Firms—five KII respondents identified MSME -Lead Firm linkages as a positive benefit of LEVE’s work. One example is the operational partnership between AGAprint (a Lead Firm) and Delong (an MSME) in the A&T sector, according to two LEVE staff members; the details of the partnership are not publicly available, but LEVE connected the actors. LEVE staff also noted that if the A&T sector in Haiti continues to move toward more value added (and in some cases, full package) manufacturing, “and will require supervisors and different types of skills, the small designers can fill this need and can be complementary to large apparel in terms of skill and knowledge and promote exports.” 29 An apt example of LEVE’s capacity for connecting MSMEs to Lead Firm s is LEVE’s relationship with Papyrus. The latter was a small woman-owned Haitian business at the outset of LEVE. Over five years, the two actors developed a very close and productive working relationship . A respondent from Papyrus (a LEVE beneficiary) said, “when we started with LEVE, we had seven employees with $750,000 in revenue. Now we are 90 employees with $6 million in annual revenue.” LEVE helped Papyrus move from a small, CEO-driven firm to a corporation with a Board of Directors. LEVE helped Papyrus structure its board and sell shares in the company. The connection between RTI/LEVE (which can be considered a Lead Firm ) and Papyrus proved instrumental for the latter. Papyrus staff noted , “We were allowed insight into project management from the perspective of a prime. We sat in a management unit meeting [at LEVE] in every Monday. The r unning of LEVE was discussed in front of us. We had input into certain programmatic decisions. Through this relationship we learned a lot about systems for monitoring and evaluation, grants management and program design.” Papyrus also reported that through linkages it built at LEVE events, it was able to design better proposals for new development projects in Haiti. For the designs of Konbit and Smallholder Alliance for Sorghum in Haiti (SMASH), Papyrus said that it adopted LEVE’s “light touch facilitation” approach through capacity building. According to several interviewees, the A&T sector remains a key area for continued investment from international development donors. The HOPE -HELP legislation has created the opportunity for Haiti to expand A&T exports to the American market at low cost, and major actors in the A&T sector (like 28 Pan American Development Foundation, “Empowering Haitian Rice Farmers,” August 15, 2016. https://www.padf.org/news/2016/3/3/empowering-haitian-rice-farmers. 29 Full package manufacturing refers to a firm’s capacity to offer a garment based on supplying all components of a garment including the sowing and fabric itself (also known as FOB). USAID/Haiti, “Value Chain Assessment Annex 1. Apparel and Textiles Sector Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. iii. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 12 Lululemon) have recognized Haiti as a valuable production point. According to LEVE staff, there are between 55 and 65 thousand jobs in the apparel and textiles sector in Haiti. “ However, if we can create the correct environment in this country, we can push this sector to 100,000 jobs. Maybe this country could sustain as many as 300,000 jobs if a lot of things came together. GMC (a Lead Firm) for example is moving towards (nearly) full package manufacturing and is investing in screen printing through AGA- print.” The ET notes a final point regarding LEVE enabling MSMEs to work with other actors in their value chain to mutually produce value. Interviewees in all respondent groups – except construction – said that LEVE’s ability to create linkages among businesses operating in Haiti was due to its staff. For example, nine respondents said that LEVE was effective because their managers had such deep and enduring roots in the Haitian business community . Their Chief of P arty (COP) grew up in Haiti, and their staff, especially in A&T, had been active in the sector for years. The A&T sector l ead whom several respondents referenced by name is well- known in the A&T sector because she formerly owned a factory. One Lead Firm beneficiary said, “In particular, Marie Louise Baker has helped a lot in unlocking situations particularly at the customs and government levels.” Objective 2: Increase MSMEs access to a productive labor pool with relevant skills and competencies As mentioned above, LEVE contributed to training and capacity development in Haiti, which it accomplished through its work with TVETs, and tailored assistance to MSMEs in all three sectors. Nearly a third of interviewees said that LEVE’s work with technical and vocational schools had a positive impact in creating a productive labor pool with relevant skills and competencies. LEVE connected TVET- trained students to construction sector contractors (like DFS) in the north, and some students obtained jobs and/or internships, according to FGD participants and a LEVE staff member. A pertinent example is LEVE’s work with TVETs in Cap -Haitian, where LEVE worked to link students trained in welding, plumbing and other construction- relevant skills to companies that planned to build 700 new hotel rooms. Unfortunately, companies did not ultimately build many of these rooms. Moreover, LEVE supported a job- placement officer at two TVETs in the north, which helped schools to attract more students and, consequently, more revenue, according to both a LEVE official and the job placement officer. As a more successful example, LEVE increased access to a productive labor pool (for MSMEs and Lead Firms) within the Textile and Apparel Service Center (TASC). LEVE worked with the Association des Industries d’Haiti (ADIH), which is the main coordinating body (akin to a Chamber of Commerce) for the A&T sector, to train new workers. Electra Sewing and Institut national pour le développement et la promotion de la couture (INEPCO) provided this training , which included basic factory behavioral training, as well as technical skill training. This program, according to LEVE respondents, trained several hundred new A&T sector workers every few months over the Activity’s lifetime. LEVE also helped beneficiaries obtain a Hazard Analysis Critical Control Point (HACCP) health certification to enable agri -transformation companies to export their products to North American markets. Caribbean Food Manufacturing and Sant Pwodiksyon Nourimamba obtained their HACCP certification due to LEVE’s intervention, while two aquaculture firms were in the processes of acquiring certification as LEVE closed. In fact, ten respondents in the agribusiness sector said that LEVE -supported trainings (in addition to HACCP certification support) were critical interventions. A beneficiary in the rice value chain said, “In addition to training, LEVE showed us how to prepare reports. An agronomist came to train us in report writing, sustainable management of the mill and accounting management.” 13 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV The LEVE/Papyrus/SMASH connection, discussed above, extends to a Lead Firm in the agribusiness sector, Brana. LEVE hired a short-term technical specialist to train Brana staff on how to operate a sorghum conditioning center. This linkage allowed the specialist to train all Brana staff, who are now training new staff on their own. This STTA connection has allowed Brana, which needs to purchase supplementary sorghum from American famers (due to quality issues with some Haitian production) , to mix its imported sorghum with Haitian-produced sorghum and resell the mixture back to American buyers. In construction, LEVE worked with a Haitian firm to develop a strategic plan for the business, which the firm indicated was helpful. However, due to the company owner’s death , the relationship with LEVE did not progress. One respondent who worked in the agriculture sector as an implementer said that due to LEVE, he noted many changes within farmer’s organizations despite a difficult business environment. “The MSMEs that LEVE helped in the agriculture sector have learned from the trainings they received to improve the quality of their work. They learned that hiring the right people is important. They have begun to adopt more modern business management techniques. They are not improvising; they are trying to find the right equipment and qualified staff. And using the right methods to recruit staff using basic business skills.” In total, over a third of evaluation respondents reported that LEVE contributed to the creation of new jobs and/or to the improvement of existing jobs. Several agricultural sector business es reported doubling the size of their workforce as a result of their work with LEVE, while a respondent in the tomato value chain said LEVE contributed to the creation of an estimated 60 new indirect jobs for individuals who transport fresh tomatoes but are not directly employed by the SHAISA factory. A&T firm respondents also reported large increases in the number and quality of their employees. Objective 3: I mprove sustainability of Haitian organizations serving target sectors and corridors USAID designed LEVE’s facilitation and training -based interventions to be “ light-touch,” according to a staff member. LEVE sought to positively impact the operations of Haitian businesses and improve the business environment, while ensuring that all LEVE interventions remained replicable and scalable after LEVE closed. A LEVE beneficiary in the agribusiness sector summarized her view of LEVE’s impact on the sustainability of Haitian businesses: “LEVE does not mother its beneficiaries. On the contrary, LEVE has made us autonomous and independent. LEVE knows that the beneficiaries it leaves are strengthened beneficiaries who will move forward. I know they will be proud with what they have left behind with Delice Tropical because the company is stronger and can continue by itself. LEVE never told us what to do. They especially showed us how to do it. That's the big difference.” The back-office support and business diversification ideas—all of which the ET discusses further in Evaluation Question 3— proved critical for Haitian business sustainability. The back-office support in terms of book-keeping, organization and business orientation have ensured that Haitian businesses can survive the dramatic vicissitudes of the Haitian economy. One agribusiness operator “In addition to training, LEVE showed us how to prepare reports. An agronomist came to train us in report writing, sustainable management of the mill and accounting management.” LEVE beneficiary in the rice value chain “LEVE never told us what to do. They especially showed us how to do it. That's the big difference.” LEVE Agribusiness sector beneficiary USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 14 elaborated exactly how LEVE accomplished this, saying “ LEVE strengthened my entrepreneurial abilities. LEVE …allowed us to be better structured, we know how to record income and cash. Thanks to LEVE orientations… we have meetings twice a month; meetings unite and strengthen us. LEVE showed us how to manage our cash flow and our accounting. Before LEVE, the busine ss was unstructured. Currently, we manage our activities better, we have records of the receipts, it is our strength.” Several LEVE beneficiaries in the A&T and agribusiness sectors echoed this sentiment. As this type of support does not require continuous direct LEVE intervention, it is sustainable if beneficiaries maintain the knowledge and systems they acquired through LEVE. These types of interventions also allowed business to plan ahead for potential economic or political upheaval, particularly by analyzing the state of their company’s finances at any given time. The ET found that LEVE did not achieve broad success in the area of construction . Despite this, LEVE’s efforts to build TVETs training and skill development capacity can be sustained without LEVE if TVETs remain financially solvent. LEVE worked with TVETs to help them become more sustainable financially by helping attract new students through job placement services and establishing small-scale production centers within TVETs, including furniture production and limited -run textile production at TASC. These small-scale production centers have the potential to sell student-produced goods, thereby generating income for the TVETs, and allowing TVET students to practice and refine their technical skills. These centers can also allow students to demonstrate their skills to industry. A decisive example of the sustainability of LEVE beneficiaries is its work with Papyrus . LEVE’s support to the latter helped it win a $13 million award from Global Affairs Canada to “upgrade the corn value chain in Haiti.” 30 Papyrus also reported that it won the SMASH II award for a five-year follow-on to its successful work on SMASH, and it designed its concept based on Papyrus’s work with LEVE. OBJECTIVE 4: IDENTIFY AND IMPROVE SYNERGIES AMONG EXISTING PROJECTS AND ACTIVITIES A minority of interviewees reported that LEVE was able to identify and improve upon synergies between existing projects and actors. Beginning in 2015, LEVE identified USAID’s and other international donors’ investments in the Haitian business community, and leveraged them substantially, according to LEVE’s yearly reporting. 31 Based on LEVE’s annual report, the Activity sought to continue building the capacity of businesses that had already received support from other donors. Therefore, LEVE worked with USAID Chanje Lavi Plante (CLP) to organize workshops on agricultural finance. LEVE collaborated with the Canadian International Development Agency (CIDA)-funded National Support Program for the Structuring of Haitian Entrepreneurship (PANSEH), a “project to produce the study on formalization of enterprises.” The Activity worked with the International Code Council on the new Plumbing code. Also, LEVE built on the work that PADF LEAD accomplished prior to LEVE’s start, specifically with firms in the A&T and agriculture sector that LEAD trained and provided equipment and targeted financial assistance. 32 One A&T sector respondent said, “When we started working with LEVE, the latter contacted USAID to get to know us better. I think some of our employees have contacts with LEAD and LEVE.” As noted above, in the rice value chain, LEVE built on work that LEAD conducted in the Saint-Marc region. 30 USAID/Haiti, “Local Enterprise and Value Chain Enhancement (LEVE) Project: FY 18 Annual Report (Including Q 4 Summary),” October 2018. 31 USAID/Haiti, “Local Enterprise and Value Chain Enhancement (LEVE) Project: FY 15 Annual Report (Including Q 4 Summary),” October 2015. 32 USAID/Haiti, “Local Enterprise and Value Chain Enhancement (LEVE) Project: FY 17 Annual Report (Including Q 4 Summary),” October 2017. 15 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV A construction sector respondent reported that his firm worked with USAID/HOME, through which HOME developed a working relationship with LEVE, allowing him to benefit from LEVE assistance. The synergy between HOME and LEVE predates this interaction. A LEVE staff member noted that LEVE organized the Activity design event for USAID Home with the future Contracting Officer’s Representative (COR). LEVE also attempted to work with HOME in the multi-plex housing value chain. LEVE helped design a technical assistance intervention for a beneficiary the ET interviewed , but as a result of various delays in startup, and the death of the firm’s CEO, the intervention never moved past the initial startup phase. Other examples in which LEVE worked with existing projects include LEVE’s close partnerships with Papyrus and SMASH. The former Chief of Party for both Feed the Future Chanje L avi Plante (CLP) and WINNER said that LEVE worked with CLP to help a strawberry grower set up greenhouses to increase the quality and quantity of production; the two also partnered to help an agribusiness acquire a small power tiller to plow land for rice fields. This respondent also noted that CLP and LEVE provide d tools to farmers in the rice value chain necessary for them to prepare to grow and harvest rice . LEVE staff reported that in addition to the aforementioned collaborations, LEVE worked with Global Affairs Canada (GAC). According to several staff, USAID told LEVE staff they had met their obligations for working with other activities, and therefore no longer needed to focus on this objective. “Our COR told us we could stop measuring this.” ADAPTIVE MANAGEMENT A quarter of KII respondents reported they believed adaptive management allowed LEVE to be successful. For LEVE, adaptive management “is about figuring out how and with whom to spend the right amounts of money for the right things, as well as providing support where and when it was required.” This approach, in the words of a USAID COP, fundamentally altered how it spent money. “Our spending was a be ll-curve, not linear like traditional USAID programs. This first year of the Activity was building our network and building our profile. Engagement was protracted because relationship development takes years.” Three sectoral assessments (one each in A&T, agribusiness and construction) formed the basis of LEVE’s approach to adaptive management. These assessments guided the work that LEVE conducted over the Activity’s life cycle. Additionally, LEVE used data from its Monitoring, Evaluation & Learning ( MEL) systems to “reinvent the activity and have a dynamic work planning process.” A LEVE staff member reported that the target outputs for many of LEVE’s programs were not static due to the Activity’s adaptive management approach; program targets constantly shifted to accommodate firms’ business conditions, as well as Haiti’s political situation. One LEVE beneficiary in the A&T sector said , “the experience of LEVE’s flexibility and adaptive programming was great. They listened to understand how to develop a program that could meet the needs of local companies. There were many meeting s for the development of tailored programming and trainings.” A USAID official echoed this, noting that LEVE’s adaptive management, as typified in its business coaching, was “key to survival. Also, LEVE had a feedback loop for learning. They tried to work with IDEJAN, but there was no willingness from them [so LEVE moved on]. LEVE worked with ADIH and Chambers of Commerce instead. They thought creatively about how to address the local challenges and bottlenecks.” Several respondents did not offer full-throated support of LEVE’s adaptive management approach. Specifically, two American government officials said that LEVE did not adapt to the situation on the ground to meet potential beneficiaries’ needs, but instead mandated that Haitian businesses adapt to LEVE. Additionally, one official stated that USAID wanted LEVE to work with informal businesses to help USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 16 them become formal, because according to the “best available data, the Haitian economy is primarily driven by informal MSMEs, which generate up to 90 percent of new jobs.” 33 In fact, a “fundamental objective of value chain programming,” and therefore of the LEVE activity, “is to move poor individuals, households, and MSMEs out of saturated, low -return activities and into higher-return, growing markets and eventually into the formal sector.” 34 This official noted that LEVE did not work to systematically formalize Haitian businesses: “LEVE could have studied this and found ways to address concerns and understand why businesses do not formalize. If I am doing another project now, I would love to know how to incentivize businesses to formalize.” 35 Construction In addition to the concerns these two officials raised about LEVE’s adaptive management approach, one may question why, if LEVE employed a flexible and responsive adaptive management approach, the Activity did not demonstrate any notable successes in the construction sector. One may further question why LEVE took several years t o tell USAID that it was unlikely to meet its goals in the construction sector. According to LEVE staff and another USAID IP, LEVE’s lack of success in the construction sector was largely due to factors beyond the Activity’s control. LEVE was proactive in this sector, but its efforts were stymied, in many cases by a lack of follow- through from its various partners, informality in the sector, and political instability. The ET note that LEVE M&E data and interviews with individuals involved in the Haitian construction sector – not LEVE staff – informed this description of the Haitian sector. Currently, the Haitian construction sector accounts for between 175,000-200,000 jobs, most of which are informal and temporary, making new job creation difficult. As much as 90 percent of the activity in the housing sector is informal, offering few points of entry for formal businesses. Two construction sector respondents reported that when a family builds a house, it often does so in stages, according to when it has the cash on hand to buy the necessary materials and hire the necessary labor . Haiti imports most of its construction materials ; therefore, material costs can account for 50- 70 percent of any construction project. As a result, Haitian construction businesses have limited opportunity to turn a profit. Finally, for large-scale building projects, businesses must import or lease heavy equipment. This phenomenon drives up the costs and complexity of large construction projects, as firms are required to possess the capacity to put up a financial bond to obtain necessary materials. Many Haitian firms cannot do so due to liquidity issues. Consequently, firms from the Dominican Republic that own their own machinery and equipment can come to Haiti for a single project, bring in the necessary equipment, and return home for less than it would cost a Haitian firm. Further, informality – when firms hire unskilled but experienced labor as needed – makes it difficult for interventions to identify opportunities for new formal job creation. These points are all systematic issues in the sector that undermine any Activity’s ability to operate in construction. Through its sector assessment, LEVE identified multiplex housing, rural roads and commercial buildings as value productive value chains that the Activity could work in. To this end, in 2014, LEVE led – at 33 LEVE contract, section C, p. 10. 34 Ibid, p. 15. 35 Many Haitian businesses do not formalize to avoid a higher tax burden, as well as a host of challenges associated with registering a new business, costing time and money. 17 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV USAID’s request – bidder training events with 14 Haitian construction firms, designed to teach firms how to bid on large-scale construction projects at, all in anticipation of USAID introducing a $30-$40 million construction project to address gaps in the Haitian road system called The Stand -Alone Roads Activity (SARA). Unfortunately, USAID never released a request for proposals. In 2014-2015, LEVE worked with the construction committee of the American Chamber of Commerce (AmCham) to organize a construction trade show (MATCON 2015). According to a LEVE staff member, the goal o f the event was to connect construction firms to form consortiums to bid jointly on large projects, and to stimulate ideas for better quality construction in Haiti. Political instability and elections delayed this event for a year (initially planned for 2014). Moreover, while the initial MATCON was a success, subsequent efforts by Am Cham to organize a follow-on event were not successful, despite continued LEVE support. Despite the trade show’s success (as well as LEVE’s work with TVETs), commercial construction firms were unwilling to work with TVETs to build skills. These firms focused on their businesses’ day-to-day management. Around Cap-Haitian, LEVE worked to train TVET students to find jobs in planned construction of seven new hotels. However, construction firms built only two new hotels during LEVE’s five-year lifecycle, undermining job creation in commercial building. While construction firms in the north hired some LEVE- supported TVET students (DFS, for example), firms simply did not need the newly trained laborers . In place of new job creation in this sector, after repeated failures, LEVE shifted its focus to improving the quality of construction sector jobs, according to LEVE staff. In this vein , in partnership with the International Code Council (ICC), the Collège National des Ingénieurs et Architectes Haïtiens (CNAIH) and MTPTC (Ministère des Travaux Publics, Transports et Communications), LEVE developed a Haitian plumbing code based on international standards. LEVE was not able to conduct significant programmatic follow-up on the new plumbing code due to government instability. Consequently , the GoH assumed responsibility to finalize and accept the new plumbing code and establish a training curriculum and certification system for TVET students. LEVE also worked with INFP on a nation-wide roll- out of a masonry training initiative. LEVE ultimately abandoned this due to a lack of initiative within INFP. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 18 Working with the Government of Haiti Half of interviewees said working with the GoH was hard because of turnover among Ministers (i.e. MCI), limited capacity, and lack of business culture. According to one IP, “In Haiti it is very difficult to work with the GoH because it changes so often… there have been 4-5 Prime Ministers since I have been here [over five years].” A separate IP said that over nearly a decade working on USAID-funded activities he worked with 4-5 separate governments, the incoherence of which made working with them difficult. “Haiti does not have a pro-business culture among civil servants. And sometimes they don’t really understand the importance of supporting business to create jobs and facilitate growth. We also have legal and regulatory framework that is totally obsolete…. I would also say when the public sector is thinking about talking to the private sector, they don’t consider the MSMEs, it is the big guys that you find [the GoH working with].” Three respondents reported that Ministerial- level interest in development programming can be somewhat capricious. They said that one M inister may take an interest in a program (e.g. an agricultural program), but if/when there is a change in personnel, the new Minister may not share that same enthusiasm. However, despite the difficulty, about a quarter of KII participants said that international development IPs must continue to work closely with the GoH. Once agriculture sector respondent said that his firm maintains a good relationship with the Ministry o f Agriculture, but he does not expect any assistance from the GoH. An IP in the agriculture sector said, “there are excellent people at the M inistry of Agriculture, they are active and productive and want to move forward, and through them we are able to develop collaboration with other departments in the Ministry.” A local consultant reported that working with technical staff within a given Ministry is critical because these staff experience less turnover and can serve as reliable advocates for development Activities within a Ministry. This respondent said that he looked forward to helping train agronomists in the Ministry of Agriculture to expand their capacity to understand development interventions: “The GoH have ten agronomists, and we want to bring them out to show them what we are doing. We want to strengthen their capacity through our work.” According to a third of KII respondents, to most effectively work with the GoH, Activities must have a champion within the Ministries technical offices with which they need to work. The key is to include technical staff at every step in the development, start-up and implementation of a development activity. Working with the same people consistently can help build up capacity within a given M inistry, increase interest in an activity, as well as help gain traction with the Minister. According to a consultant in the A&T sector, “everything in Haiti is about personal relationships, we all have the necessary phone number of someone to call to get things done. These are people who don’t change and know the system. Building relationships when you don’t have a problem is really important.” A US Embassy official echoed this sentiment: “the best way to work with GoH is to cultivate long-term relationships with technical staff. It is hard to meet with director general, but it is possible to get meetings with lower staff like head of public investment at MCI…. [it is necessary to] identify champions within ministries and continue to work with them.” Incorporating Women and Youth While not an explicit component of the LEVE contract (nor possessing its own set of indicators), LEVE worked toward the crosscutting issue of incorporating more women and youth into its three sectors of interest. LEVE’s work produced a direct and measurable effect on youth and women, as most TVET students are between 18 and 35 years old, and all TVETs cater to men and women, according to eight interviewees affiliated with TVETs. 19 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV Due to the challenges LEVE faced in the construction sector, there is little data available on women and youth in construction. From the ET’s own data, construction is a male-dominated sector in which older men with several years of work experience are more valued than younger, less experienced laborers. One KII respondent noted that women are indirectly involved in the construction sector because they tend to initiate food cart businesses outside construction sites to feed laborers. The ET conducted one all-female FGD with welding students from a Cap -Haitian TVET; these women reported being over- qualified for the available work in Haiti and indicated a desire to travel abroad to find more lucrative and challenging professional opportunities. More broadly, half of interviewees indicated that women and youth were key beneficiaries of LEVE in the A&T sector, and indirectly in the agriculture sector. These two groups benefitted indirectly in the latter sector because youth are increasingly uninterested in working on family farms, according to several KII respondents. The labor is difficult and pays little. One respondent noted that the technical training provided through LEVE could directly benefit youth interested in training and mechanization. “No one wants to work on father’s farm, as with all other places around the world. It is hard manual labor. The only way to move forward is low tech mechanization. So, I bring a mechanic from Quebec every three months and we travel around for one week, make repairs and diagnostics on machinery. We also do one to two full days of training on how to do preventative maintenance, and better operating of machines, and address the most common local repair issue. This is very popular; this is one of the most valuable things to do. There is a waiting list for this.” In the A&T sector, according to a GoH official, “almost 75 percent of workers are women. This is a phenomenon of Haiti.” A majority of individuals receiving training are also women. Women owned, operated, and staffed the vast majority of MSMEs with which LEVE worked. According to an IP, “one amazing thing we have seen is that in terms of small businesses, they are dominated by women, and the TASC is saturated with women.” 36 CONCLUSIONS LEVE was successful in achieving its four objectives. It achieved this success because its facilitation-based approach drove job creation in MSMEs and lead firms in the A&T and agriculture sectors. Objective 1: LEVE was successful in creating business linkages between MSMEs, as well as between MSMEs and lead firms. LEVE created these linkages, which are likely to continue to help Haitian businesses work together to mutually create value in their respective value chains and sectors. Objective 2: LEVE contributed to training and capacity development in Haiti, which it accomplished through its work with TVETs and other labor training centers, through back-office support to MSMEs, assistance in obtaining relevant certification, and connections to STTA . These types of support have increased Haitian businesses’ access to a sustainable and productive labor pool. Objective 3: LEVE’s “ light-touch” facilitation and training-based interventions are likely to be sustainable because USAID designed them to positively impact the operations of Haitian businesses and improve the business environment, while ensuring that all beneficiaries could replicate the support they received even after LEVE ended . 36 In 2009, the World Bank reported that 68 percent of the A&T sector laborers in Haiti were women. The World Bank, “Bringing HOPE to Haiti’s Apparel Industry,” November 2009, p.36, http://documents.worldbank.org/curated/en/715301468256506127/pdf/533160ESW0P11410Report10January2010.pdf. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 20 Objective 4: LEVE identified synergies between itself, other USAID Activities, and other international donor programs. In so doing, it was able to expand on work that preceded it to further build the business capacity of beneficiaries. It was so successful that USAID told LEVE it could stop tracking progress on this indicator well before Activity closeout. LEVE’s adaptive management approach is the underlying factor that enabled its success. The approach allowed LEVE to adapt to the Haitian context and design tailored interventions for each of its beneficiaries. Women and youth were direct and indirect beneficiaries of LEVE programming, and the benefits LEVE provided them are likely to continue as long as the A&T and agriculture sectors continue to be the backbone of the Haitian economy. LEVE was less successful in the construction sector. Systematic challenges in the construction sector, and challenges associated with collaborating with businesses and GoH Ministries involved in the sector has caused much of this lack of success. LEVE’s adaptive management techniques allowed the Activity some successes and helped it limit the time and effort spent working in this sector. Finally, it was difficult for LEVEL to w ork with the GoH. Future Activities require that staff carefully manage personal relationships with GoH and identify champions within technical offices of relevant Ministries. EVALUATION QUESTION 2: T O WHAT EXTENT AND IN WHAT WAYS HAVE LEVE ACTIVITIES RELATED TO THE MAIN VALUE CHAINS GENERATED MULTIPLIER EFFECTS ? FINDINGS USAID requested an ROI analysis to address EQ 2 and assess LEVE’s impacts with additional analysis of any multiplier effects. The available data, however, was insufficient to answer these questions thoroughly, particularly regarding multiplier effects. This analysis thus represents a parameter- based model with numerous alternative scenarios to examine the robustness of results. Data The ET based its ROI analysis on L EVE performance indicator and expenditure data. The ET calculated job creation benefits from LEVE activities utilizing the two quarterly indicators related to job creation: PL1: Number of jobs created in target sectors in designated corridors (FTE) and PL14: Number of new or improved jobs as a result of project activities. For comparability with LEVE’s internal ROI study, the ET utilized PL1 as the primary indicator, and PL14 as a check for robustness. LEVE provided cost data to indicate total amounts it billed USAID by year, divided into general categories of Labor, Fringe, and Other Direct Costs. For the purposes of the ROI analysis, the ET used the total of the three categories as the cost basis for LEVE. The analytical model required additional information that LEVE’s M&E data did not provide ; specifically, wages, income tax rates, estimates of pre-LEVE earnings, job loss data, and discount rates to account for time. For the purpose of building the ROI model, the ET assumed wages at the relevant minimum wage and converted to the U.S. dollar equivalent and acquired income tax rates from the Ministry of Economy and Finance. The ET found no systematic evidence of pre -LEVE earnings of beneficiaries; thus, the ET employs a range of $0 to $2 per day in its model, based on conversations with USAID and LEVE staff. The ET similarly did not acquire j ob loss or turnover data; its literature search yielded anecdotal 21 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV information suggesting four to six percent per year job loss in the Haitian apparel sector, and the ET used a best-case value of four percent in the model. The ROI utilized a time discount rate of 12 percent per year, consistent with standard USAID cost-benefit methodologies. Sectors LEVE sought to create jobs in three sectors: apparel, agribusiness, and construction. The performance indicator data, however, show that 99 percent of new jobs created through LEVE were in the apparel industry. Further, the number of jobs LEVE created in the agribusiness and construction industries are too small to have generated broad impact. Evidence from previous studies also suggests that average wages in agribusiness and construction may be marginally lower than the apparel industry. Taken together, there is little evidence to warrant an in-depth analysis by sector. Thus, the ROI analysis focused exclusively on the apparel sector to account for nearly all of LEVE's attributable and significant benefits. Limitations The lack of sufficient or appropriate data limited this ROI. USAID did not design L EVE for external assessment via impact evaluation or formal cost-benefit methodology, like ROI. There is no baseline data, USAID did not create the Activity with an experimental design, and there is no attempt to account for a counterfactual. LEVE M&E data is also incomplete, lacking necessary information on wages, taxes, previous earnings, and job loss. The ET instead estimated t hese data from official government sources, previous research, and anecdotal evidence from conversations with USAID and LEVE staff. Lacking information specific to particular individuals or firms, the ET also applied these data uniformly to all LEVE beneficiaries. As a result, the ET cannot determine whether the model represents the conditions faced by any particular individual or firm; rather, the ROI model presents a general assessment of LEVE's effect on the Haitian apparel sector. A significant portion of the ROI analysis also involves projections of benefits until 2025 under the ET’s assumption of stability in political conditions, continuing demand for Haitian apparel products on the world market, and stability in the exchange rate for the gourde. Recent political turmoil in Haiti, and two years of a continually declining exchange rate, call into question the realism of these context assumptions, and readers should interpret projections of ROI results beyond the life -of-project (LOP) period as best-case scenarios. Benefits Wages The study employs the minimum wage rate of 370 Haitian gourdes per day as a baseline wage, adjusted to 425 gourdes in FY2019 to reflect the increase in the minimum wage. Since the beginning of the project, the Haitian gourde has experienced significant and ongoing depreciation, such that the US dollar value of the daily wage has declined from $7.43 per day in 2015 to $6.44 per day in 2019, despite the increase in the minimum wage. For the purposes of modeling, the ET adjusted the nominal wage rate in gourdes annually by the US dollar exchange rate to obtain the relevant value. Taxes The analysis assumes LEVE created jobs in the formal sector, which are thus subject to applicable Haitian income tax rates. However, anecdotal information suggests that Haitian tax collection lacks rigor, enabling many individuals to effectively pay little or no tax. All evidence suggests that LEVE’s beneficiaries USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 22 generally fell into the lower income, ten percent tax rate band. 37 For the purpose of modeling, the ET assumed an effective tax rate of zero percent as the baseline, with alternative scenarios offering results for other tax rates. Previous Earnings ROI analysis assessed the net impact of an intervention on its intended beneficiaries, and thus required an estimate of pre-intervention earnings to derive the net effect of the project. LEVE did not directly collect such data, and published sources provided no useful estimates. However, both anecdotal evidence and communications with USAID and LEVE staff suggest previous wage estimates of $0- 2 per day. The model employs a baseline assumption of $1 per day, and alternative scenarios also model values of $0 and $2. Job Decay The ET based ROI analysis on the number of net new jobs LEVE created, and estimates of job loss are as important as counts of created jobs. The LEVE indicator database provides sparse estimates of jobs lost in a few quarters at only a handful of the larger firms and does not indicate whether the lost jobs were LEVE beneficiaries or previous employees. For modeling purposes, the ET did not utilize th is incomplete job loss data. While little rigorous quantitative research exists on the apparel sector in Haiti, a prominent apparel industry publication reports as a benefit of investing the Haitian apparel industry, “low levels of… turnover (between four and six per cent per year),” 38 based on a survey of factory managers. These values suggest the potential range of job loss parameters, and the ET assumed the most optimistic four percent value as a baseline. As LEVE data was reported quarterly, the ET assumed a one percent rate of job loss per quarter. Discount Rates The analysis applied discount rates to money-based data to normalize for the effects of time. Discount rates are also indicative of individual rates of time preference and the willingness to trade consumption in the future for consumption in the present. The model employed a standard 12 percent annual discount rate, despite evidence that “Haiti is a relatively ‘impatient’ country,” 39 where “borrowers are willing to pay 24-50 percent per month” 40 for access to credit, suggesting very high discount rate for individuals. For the purposes of modeling ROI, such values would introduce a heavy downward bias in the calculation of benefits, so the ET used the sector-accepted 12 percent instead for both cost and benefit calculations. Time Horizon Two-time horizons are relevant to this analysis. First, the LOP timeline for which performance data are available provides the timeline for directly attributable benefits of the project. Second, previous analyses 37 As per official income tax filing form posted on the Ministry of Economy and Finances (MEF) website: https://dgi.gouv.ht/wp-content/uploads/2018/01/DECLARATION-DEFINITIVE-DIMPOT-SUR-LE-REVENU-Version-Simplifiee- Categorie-Traitement-et-Salaires.pdf. 38 Can Haiti mark solid presence as ‘Promising’ apparel manufacturing hub. Apparel Resources. January 19, 2019. http://apparelresources.com/business-news/sourcing/can-haiti-mark-solid-presence- as-promising-apparel-manufacturing-hub/. 39 Cicowiez, Martin and Agustin Filippo (2018). A Simple Stylized Long-Run Growth Model for Haiti. Inter-American Development Bank, Technical Note IDB-TN-1485, September. 40 Lundahl, Mats (2011). Poverty in Haiti: Essays on Underdevelopment and Post Disaster Prospects. Palgrave. 23 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV of LEVE and the apparel industry in Haiti suggest that the HELP and HOPE Acts may contribute to the growth of the apparel industry in Haiti through 2025, providing a second relevant timeline. Ideally, LEVE will have created systems, networks, and job creation channels that will continue beyond LEVE’s own existence, thus allowing attributable job creation to continue after the end of the project. For the post- project period, the ROI applied the average number of new jobs created by each firm throughout the LOP period as the number of new jobs created in the first quarter of the post-project period. For every quarter there after, through the end of 2025, the ROI applies the average with a compounded ten percent decay rate per quarter, such that the number of new jobs attributable to LEVE diminishes over time. Secondary Effects Data showing direct evidence of LEVE’s secondary effects was not available, and the scope of this study did not include e x post collection of field-level data. An existing USAID 2017 study 41 did attempt to calculate a range of possible multipliers for the LEVE project. The study estimated an indirect multiplier of 1.28 and an indirect-plus-induced multiplier of 1.61. For the purposes of modeling, this study used a baseline 1.28 indirect multiplier and modelled the effects of increasing the multiplier over the relevant range of values. Costs LEVE provided cost data at annual intervals and divided into the categories of Labor, Fringe, and Other Direct Costs, as is standard for USAID projects. For the purposes of the analysis, the analysis includes annual total billed to USAID in the cost calculation. LEVE’s data indicate d a total expenditure of $32,227,900. Using the 12 percent discount rate, the present discounted value (PDV) with 2015 as the base year is $27,243,505. The ET’s calculations were also derived using a 20 percent discount factor for project costs, yielding a total cost of $24,934,467. Comparison to Internal LEVE ROI Study In 2018, LEVE conducted an internal ROI study employing a set of assumptions and modeling approach that differs significantly from the present study. However, the degree of consistency in the overall results of the two studies serves as evidence of the robustness of LEVE’s impact. The primary differences between the two studies are outlined below. Benefit Categories LEVE’s analysis included three categories of benefits: direct and indirect benefits through LEVE’s job creation activities, indirect benefits through LEVE’s investment promotion activities, and benefits to Haitian nationals employed by LEVE to implement the project. The present analysis employs only the first of these three categories, as it is the only benefit type for which there is direct evidence linked to LEVE’s intended activities and beneficiaries. Investment promotion activities only have an indirect impact on jobs and benefits, and without evidence linking LEVE’s activities to secondary impacts, the study 41 Local Enterprise and Value Chain Enhancement (LEVE) Project: Measuring Job Impacts in the Haitian Apparel Sector Technical Report. February 2017. USAID. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 24 could justify the inclusion of this category. 42 LEVE’s analysis also included as benefits a portion of the wages paid to Haitian nationals to implement LEVE. The ET excluded these wages from the present study because the study did not specify these individuals as intended beneficiaries of LEVE, and, more importantly, their inclusion confounds the calculation of costs and benefits. These wages are unambiguously part of LEVE’s direct costs, and their inclusion as benefits raises methodological concerns. 43 Timeline LEVE’s analysis focused on the timeline through 2025 and did not distinguish between the observed life of the project and the projected benefits through the end of the HOPE and HELP Acts in 2025. The present study instead distinguishes between observed and expected impacts and models the evolution of the ROI from project inception to 2025. Parameters LEVE’s analysis excluded parameters estimating previous earnings and job loss or turnover. The present model includes variations of these parameters to assess the impact of LEVE on beneficiaries in terms of net earnings and expected job duration. METHODOLOGY 44 Return on investment is a simple calculation of a ratio of benefits to costs. In any given quarter, the calculation of total benefit of job creation is: ������������=������������������������������������������������∗(������������������������������������������������∗������������ ������������������������−������������������������������������������������)∗������������������������������������������������������������∗������������������������������������������������ ������������������������������������������������ represents the jobs created in the quarter, and ������������������������������������������������������������ is the estimate of the prevailing daily wage rate. ������������������������������������= (1− ������������) is an adjustment to wages to account for taxes paid, where ������������ is the tax rate. ������������������������������������������������ rep resents pre-LEVE earnings that are deducted from the benefit calculation to assess LEVE’s net impact. ������������ ������������������������������������������������= (1− ������������) is a parameter to estimate job loss, where ������������ is the quarterly rate of job loss. ������������������������������������������������ is set at 65 workdays per quarter, based on a standard of 260 workdays per year. To generate the total PDV of benefits, the analysis sums quarterly benefits with the compounded discount rate: ������������������������������������(������������)= ������������� ∗(1− ������������������������������������������������) ������������ ������������ ������������=0 42 Economic theory is also ambiguous regarding the impact of investment on jobs, since firm investments could be utilized to add productive capital that reduces the need for labor. In the absence of evidence, the ET excluded investment as a benefit category. 43 As an analogy: A firm invests in a machine to help increase output, and the IRS treats as income the expenditure on the machine because it helped create revenue. Treating costs as benefits is both incorrect and unfair. 44 The ET conducted all data manipulation and analyses using the Intercooled Stata 7.0 statistical analysis software. The ET provides the core data files and Stata code file detailing each command as part of the submission of this ROI analysis. 25 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV The parameter ������������ represents each time period, ������������ represents the total number of periods, and ������������������������������������������������ is the relevant time discount rate. Note that t his analysis ������������ repeatedly adjusts to calculate total benefits in each quarter from project inception to 2025. The ET similarly adjusted costs to PDV, as above, and the calculation of return on investment is: ������������������������������������= ������������������������������������(������������) ������������������������������������(������������) Ideally, ROI will yield a value greater than one, indicating that total benefits outweigh costs. Results Figure 3 summarizes the results of the ROI analysis using PL1 as the basis for benefit calculation. The baseline values yield an ROI of 0.424 for the direct benefits of LEVE. With the application of a 1.28 baseline multiplier, direct and indirect benefits increase to 0.543. The remainder of the table shows ROI values for a series of different scenarios. Progressively increasing the multiplier from 1.28 to 1.6, while holding constant all other baseline parameters, raises the ROI to 0.678. Adjusting the previous wage to zero, implying that all beneficiaries had no income before their employment via LEVE, yields a direct ROI of 0.506, and a total ROI of 0.648. Increasing the effective tax rate from the zero baseline to five percent and ten percent results in a corresponding reduction in both direct and total ROI. The ET also calculated the ROI to allow a 20 percent discount rate to be applied to project costs, boosting direct and total ROI to 0.463 and 0.593, respectively. Overall, during the LOP period, no reasonable variation of the parameters yields a total (direct plus indirect) ROI value greater than 0.700. FIGURE 3: BASELINE AND ALTERNATIVE SCENARIOS FOR LEVE ROI However, as Figure 3 indicates, extending the time horizon for project benefits to 2025, while keeping constant all other baseline parameters, generates a direct ROI of 1.533, and a total ROI of 1.947. The assumption of a relevant timeline to 2025 consistently yields direct ROI values greater than 1.3, and total ROI values greater than 1.7 for any reasonable parameter values. 0.424 0.543 0.593 0.636 0.678 0.648 0.429 0.51 0.477 1.947 0 0.5 1 1.5 2 BaselineMult 1.4Mult 1.5Mult 1.6Prev 0Prev 2Tax 5%Tax 10%Time 2025ROI Values Direct BenefitsDirect + Indirect Benefits BASELINE VALUES Previous wage $1 Discount 12 percent Time LOP Tax 0 percent Job Decay 4 percent Multiplier 1.28 TABLE 2: BASELINE VALUES FOR ROI ANALYSIS USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 26 Figure 4 shows the evolution of LEVE’s ROI each quarter and indicates that LEVE achieves the “break even” ROI value of one in quarter 24, six quarters after project sunset. As a robustness check, Figure 4 also shows the ROI results using indicator PL14 and highlights the consistency in the trend and overall results between the two sets of data. FIGURE 4: LEVE ROI BY QUARTER CONCLUSIONS ROI analysis may not accurately represent LEVE’s impact or success. As a method based on discounting, ROI ascribes greater weight to early events and reduces the contribution of later events. Figure 5 shows that while LEVE incurred 50 percent of costs in the first three years of operation, LEVE created over 50 percent of jobs in the final two years. Normal project startup costs and the lag to realize benefits thus create a downward bias in the ROI, particularly in the implementation period, that does not necessarily reflect LEVE’s achievement or success. Further, it is not clear that a monetary measure like ROI is the appropriate tool against which to assess LEVE. ROI results may be more useful as a type of factor analysis, highlighting which elements contribute most strongly to benefit growth, in combination with other evaluation criteria. FIGURE 5: ACCUMULATION OF LEVE COSTS AND JOBS The continuity of benefits beyond the Activity implementation period is the main driver of LEVE’s long- term ROI. Elements not measured through the ROI analysis, like sustainability, network linkages, and technical assistance, promote continued benefits from LEVE's interventions that accumulate in the long run to outweigh significantly the front-loaded costs that drag down ROI during the implementation period. LEVE needed more robust monitoring and evaluation systems for the ET to conduct an accurate and complete ROI. The limited available data does not provide a complete analysis of LEVE’s direct or indirect impact. Instead of being based on firsthand evidence, the results of the ROI are instead highly 0.0 0.5 1.0 1.5 2.0 2.5 135791113151719212325272931333537394143 ROI Quarter ROI (PL1) ROI (PL14) 0% 10% 20% 30% 40% 2014 2015 2016 2017 2018 2019 Jobs Costs 27 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV dependent on the estimated parameters. A more robust M&E system would have regularly collected data on wages, job duration, job loss, actual taxes paid, previous wages, and proxy measures to estimate discount rates, thus allowing total benefit estimates that accurately reflect LEVE’s efforts. EVALUATION QUESTION 3: TO WHAT EXTENT AND IN WHAT WAYS HAS LEVE’S FACILITATION APPROACH BEEN EFFECTIVE IN EMPOWERING SUSTAINABLE LOCAL FIRMS/ENTITIES? The ET found that LEVE was successful at empowering sustainable local firms/entities for three reasons: 1) LEVE- facilitated business linkages have empowered Activity beneficiaries; 2) LEVE’s assistance to TVETs linked students to industry, creating a tailored supply of trained workers to meet local demand; and, 3) LEVE facilitated capacity and systems development for Haitian businesses. On the other hand, the ET found that political instability, a lack of access to credit and resource gaps resulting from LEVE’s closing may combine to slow local businesses’ ability to grow. The ET presents more evidence below to suppo rt these findings. FINDINGS Business linkages and business empowerment As noted above, half of KII participants said that LEVE fostered commercial links between companies and helped develop synergies between Haitian businesses. "The fact that some companies supported by LEVE have agreed to develop partnership relationships to work together is already a very important sign of long-term viability of interventions,” said a local business owner. While the se business linkages helped individual firms identify new avenues for supplying their businesses – firms have realized that they did not have to work alone and could collaborate with other in their value chains – they also improved the level of trust and/or willingness of Haitians businesses to collaborate. Put differently, LEVE’s business linkages worked to fundamentally change business culture as it pertains to trust between actors in Haiti. Beneficiaries who spoke to the ET identified transparent trading links, and, above all, open communication, as critical elements of developing a culture of trust among Haitian entrepreneurs. According to one IP interviewee, “initially, companies didn’t even know each other, companies in Haiti worked in a vacuum, not knowing all the players in a given sector, and business owners believed that companies in their [respective value chains] were unrelated. LEVE provided the network [connections], that let businesses meet and go through business coaching through which companies meet and work together.” Interventions that focused just on facilitating these types of regular and ongoing interactions (instead of injecting cash into businesses) fundamentally changed the ways in which Haitian businesses operated. It pushed businesses to collaborate because they realized they could mutually benefit from such collaboration. “Textiles is a good example. Companies are coming here, and we introduced companies to ADIH and TASC for them to take advantage of training services. Allowing companies and trainers to work together.” These companies no longer see themselves as competitors, but as partners Empowering sustainable local business Business linkages have empowered local actors Capacity and systems development for local firms TVET assistance linked students and industry USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 28 in export; they work for the common interests of the sector, as each entrepreneur “seeks to improve one part of the value chain.” A GoH official said, “I would say that the business networking will last [as will the] training to guide the search for financing… the training received and credit history with LEVE will allow [Activity beneficiaries] to develop effective contacts with financial institutions.” The former head of a recently closed USAID-funded activity echoed the sentiment of this GoH official, saying, “I do not think companies will break the commercial links created, especially if the companies have received the technical assistance showing them the importance of trade and have seen the benefits, I think they understand the importance of maintaining these exchanges and transactions. It is for sustainability that technical assistance was given to companies. ” Relatedly, almost half of KII and FGD respondents said that LEVE empowered beneficiaries to be business leaders. One entrepreneur in the A&T sector said, “I have experience with other organizations and interventions in this sector but once the project closes everything stops. Because they did not empower the beneficiaries. It's different with LEVE. This project directly helps entrepreneurs. It is very important for job creation in the country.” Representatives from LEVE pointed to illustrative measures of empowerment that resulted from their interventions. For example, the women- owned SMEs that benefited from LEVE back -office support gained confidence in their own brands and business systems. They opened online retail websites that allowed them to sell their goods internationally. Furthermore, LEVE reported helping MSME owners to begin to consider the position of their businesses relative to the entire value chain. LEVE support helped business owners think differently about their businesses, and ultimately, to think bigger. In place of estimating the sales they needed to survive, for example, LEVE’s support encouraged firms to think about expanding through exports, setting larger long-term goals and identifying the tools they would need to meet these larger objectives. MSMEs learned, through LEVE, about money, investment opportunities , as well as the legal codes, systems of business synergies that can help their businesses thrive. As one LEVE respondent put it, “w hen you have knowledge and confidence you are more bullish in your area.” A critical component of empowering MSMEs was helping them diversify their businesses so that they could better survive the fluctuations in Haiti’s unpredictable socio-economic context. Five respondents (three from LEVE, one from USAID, and a beneficiary) noted that LEVE was instrumental in helping agri- transformation businesses like Delice Tropical and SISA diversify. In the case of the latter, during the July 2018 crisis, SISA relied on Délimart for 50-60 percent of its sales. LEVE-sponsored business coaching allowed SISA to increase its sales and diversify its customer base by working in the inform al market. SISA also received equipment to help reach the informal market, including waterproof bags that allow them to make and sell their goods in the absence of electricity . These bags do not break down, enabling SISA to sell products in the street . Nevertheless, a minority of people interviewed felt that LEVE could do more in terms of linking businesses together. For example, there are connections between small businesses creating economies of scale; yet, small businesses do not sell their products to large companies, and there are still inchoate linkages between small businesses and large companies. Additionally, some respondents believed that LEVE could have worked with more MSMEs. The HOPE-HELP law attracts large companies to work in Haiti and bigger international firms can hire large numbers of employees. It is, however, likely that these companies will eventually leave the country after HOPE -HELP ends, while local MSMEs will not leave. Thus, some respondents noted that focusing more attention on Haitian firms will en sure the long -term health of the Haitian businesses sector even without favorable trade legislation. 29 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV Capacity and Systems Development for Local Firms LEVE directly and positively impacted the capacities of local firms by training them to develop better internal systems and standard operating procedures (SOPs). One-quarter of KII interviewees said they believe that LEVE has facilitated the development of business capacity. LEVE accomplished this by designing tailored support programs to meet the specific needs of every beneficiary; LEVE provided personalized assistance in areas such as accounting/bookkeeping, quality control, systems and cost management issues, among many others. For example, as a result of LEVE -supported business coaching, beneficiaries testified that they learned how to keep detail ed records, maintain relationships with suppliers and, ultimately, be more adept entrepreneurs. Said one beneficiary in the A&T sector, “The different results with the LEVE project include: improvements in our productivity, better company structures, improved positioning in the market; better overall organization of the company’s work.” LEVE also hired some Haitian entrepreneurs, depending on their capacity, as STTA to help build capacity of other companies. The positive capacity building results of the training sessions these STTA led are due to the follow-up conducted after the sessions. According to LEVE beneficiaries , company leaders participated in these sessions and made arrangements for sessions to be repeated from time to time with the goal of refreshing their current employee’s knowledge and onboarding new employees. In many cases, these STTA-led training sessions changed the way LEVE beneficiaries operate. Capacity building was not only focused on knowledge transfer and systems building, but also instrumental at the production level for a number of firms. The “machines that LEVE gave us were really adapted to grow," said a business owner in agribusiness. In the parboiled rice sector in Saint -Marc, for example, LEVE helped introduce a new machine for sorting rice as well as training for how to maintain and fix the new machine. “This machine allows uniformity in packages (parboiled rice) and increases in the value of the product internationally. Parboiled rice is twice as expensive as normal rice.” Moreover, this new machine helped create links between the company and farmers, business linkages that could have long-term benefits not just for the company that received the new machine. According to some beneficiaries, this is a relatively new approach in that previous USAID Activities were more likely to impact beneficiaries in a general way—the same way for all beneficiaries. As a result, previous interventions often lost time lost tailoring non-targeted interventions to each beneficiary, making it more difficult to achieve and demonstrate success. Previous Activities that focused on capacity building simply did not have LEVE’s adaptive approach. Unfortunately, not all firms were able to demonstrate consistently positive results from their work with LEVE. While some firms that worked with LEVE from Activity inception have demonstrated positive and durable growth, others, particularly those that only began to benefit as LEVE neared closure did not have enough time to take advantage of project interventions. TVET Assistance Was Impactful by Linking Students and Industry The assistance provided to TVETs produced compelling results over the life of the Activity . Through this intervention, TVETs received support in various forms, including teacher training, the provision of raw materials on which students could practice skills, the establishment of a job placement office s, and the installation of power inverters to ensure some level of energ y autonomy for the schools (to operate even when the central power grid is not functioning. LEVE also paid one year's salary to the placement officer recruited for this program. As noted, fifteen KII respondents said that LEVE’s support to TVETs was somewhat impactful-to-very impactful. According to over one-quarter of KII interviewees, TVETs have historically lacked qualified teachers and the materials necessary to train their students. When a student is trained to work on solar panels, they USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 30 must be able to assemble and disassemble solar panels, as well as fix them when they break. They must also know the range of av ailable types of solar panels among many other facts. Similarly, welders need to be able to practice their skills. Prior to LEVE, many schools struggled to find qualified teachers and supply their students with training materials. The director of a TVET that benefited from LEVE’s assistance said LEVE-led training was impactful for the school. “LEVE bought materials and suppliers made the delivery. LEVE bought the materials in Haiti. The idea was to work with state-of-the-art equipment available on the territory and obtained from serious suppliers. LEVE accompanied us on what could be the suppliers in Haiti. To train in solar energy, you need equipment, the latter are very expensive. We are here to teach, we need the right equipment. LEVE has allowed us to offer quality training that can be used in the long term.” With regards to training staff and teachers in welding, a respondent from a TVET in Port -au-Prince said, “We wanted to set up an international welding workshop. LEVE and USAID helped with infrastructure, equipment and teacher training. They financed the arrival of the experts for the trainings. The goal of the project was to certify students and teachers.” U nfortunately, due to a lack of funding, welding students did not achieve their final certification. The TVET students who spoke to the ET noted that they are happy with the skills they learned, but at this point their welding skills are more sophisticated that necessary in the Haitian job market. Two FGD participants said, “Few companies need our type of welding. In the company where I work, there is no metal construction; We are not trained to live in Haiti because the market is not ready for us.” Overall, TVET students reported satisfaction with the training and education TVETs offered. Indeed, respondents from all four FGDs with TVET students reported being satisfied with the training they received . Said two female FGD participants at an agriculture-focused TVET, “We are not only trained in theory, there is also practice. At home, I was running from the land, I never thought to hold a hoe and work the land. The training allows me to recognize the importance of the rotation of crops in agriculture. It allows me to know nutrition, to know what I should eat or not. In addition, the school allows me to meet professionals like me, requires a lot of effort, allows me to do things on my own with my own. The school teaches me how to do business.” Almost every TVET respondent (including KIIs and FGDs) mentioned job placement services at TVETs. LEVE was instrumental in this, especially at two TVETs the ET met in Cap-Haitian, where the Activity funded the creation of a placement officer position. The activities of a placement officer consisted mainly of researching internships and jobs for graduating students. To do this, she established a database of “In terms of results some students were hired after their internship periods. The students were very satisfied because they could not by themselves integrate into companies, like DFS [in the construction sector]. We collaborated with DFS Construction, they recruited 30 plumbers of Saint-Esprit school after the internship (10 first and then 20 others). The internship lasted one month, the job lasted about three months. The placement program was set up by LEVE for technical schools. It was financial support through the purchase of goods made through the program, i.e. student- made office furniture. It was at the vocational school of Cap- Haitian that LEVE had given training for cabinetmaking trainers.” Placement Officer 31 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV potential companies that could receive students, visited companies in the area (public and private), sent letters, sent follow-up letters and phone calls, and organized guided visits with graduating students. According to the placement officer respondent, LEVE enabled young people to have more opportunities to enter the job market. Prior to LEVE, schools only train students, and rarely sent students back to business. In this sense, LEVE's help was critical . LEVE initially funded the placement officer for one year. Until March 2019, LEVE paid the placement officer. One of the TVETs in the north will ideally continue to pay the placement officer in the absence of LEVE. However, with LEVE’s closure, the payment officer’s exact funding source remains unclear . A further complication is that one person in the north currently functions as a placement officer for two separate TVETs, Saint-Esprit vocational school and the Cap-Haitian vocational school. At the northern TVETs, in order to allow a larger group of students to benefit from internships, TVETs created a system of rotation with the acceptance of businesses; TVET would continually replace the students who acquired internships with another group of students. In general, businesses retained some student interns as employees. The experience was particularly impactful, according to the job placement officer. In particular, DFS construction offered young people practical work experience , but also additional training during the internship period. DFS was also open to providing assessments and evaluations of each student's skills and behavior, which help ed TVETs make improvements to their programming. Unfortunately, DFS Construction has closed it project in Haiti and returned to the US. Due to the prevailing socio-political situation in Haiti, there is an overall lack of activity on the job market, depressing the number of available job opportunities. Of concern is the fact that internships and jobs TVETs identified for students are exclusively in the private sector; GoH entities have been unwilling to accept students as interns despite efforts to change this. According to a job placement officer, GoH entities continue to offer student appointments to TVETs, but these do not generate internship or job opportunities. Despite LEVE’s support, improvements to the TVET system remain necessary, especially to make them sustainable. CRAAN Construction and the Collège National des Ingénieurs et Architectes Haïtiens (CNIAH), with LEVE support organized a proficiency test for TVET- trained plumbers, the results of which were not satisfactory—save one high- scoring female student. According to seven respondents, TVETs need to review their programs and training curricula, become self- sustaining businesses, require ongoing training of trainers, and provide links to industry. One TVET official emphasized that TVET programming must be geared towards the market , toward industry, and must be self-sustaining. “I would suggest that school officials work with companies, expand the framework, and create a lot more links to the market so that I can place students. To reinvigorate the schools, we need to review the workshops. From my experiences… with the training received in cabinetmaking, now the workshop can support some TVET expenses. The workshops are assets for the schools even if one does not find help to finance this or that program. I think we need to help them to create and to offer their services to many more companies because if the school has two or three workshops that work this would allow them to bear some costs.” TVET Official USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 32 Inadequate GoH assistance, even for semi-private TVETs where the state pays teachers ’ salaries, undermines the training available to students, as teachers will strike for higher wages or simply find other employment opportunities, thereby limiting the time students can spend with teachers when they are in school. Relatedly, most vocational schools lack workshops for practicing their skills . Available facilities are often not actually workshops because they lack necessary equipment. Challenges to Empowering Sustainable Local Business Lack of A ccess to Credit Several interviewees said that a lack of access to capital and credit limits the ability of Haitian businesses to grow and thrive. With access to credit, firms can anticipate their revenues and use available funds as purchasing power to acquiring new technology to increase the quality and quantity of production. Haitian commercial banks do not offer businesses access to capital at interest rates that many businesses can afford, according to interviewees. MSMEs can simply not afford the 22 percent loans that exist today, especially for higher risk sectors and value chains. LEVE often provided in-kind grants beneficiaries, but the Activity was not directly involved in facilitating access to credit. Therefore , several beneficiaries, implementing partners, and GoH officials interviewed identified the lack of access to credit as one of the major problems that firms will face after the end of LEVE. One GoH official noted one bank’s current effort to expand access to credit, particularly in the agriculture sector, offering a six percent credit rate. Political Instability Political instability may jeopardize the sustainability of LEVE’s interventions in terms of capacity building for firms, according to almost one-quarter of interviewees. As a result of the enduring threats of labor strikes and protests, it is hard to grow businesses and attract investors; four KII respondents reported the business environment is difficult and unfriendly. The textile sector alone attracts over a billion dollars in investment, but without stability, companies will not be able to continue operating. 45 Political instability can affect companies’ ability to meet production deadlines, and can affect the quality of work, as workers may stay home for fear of their safety. When companies miss deadlines, the disruption causes problems in the broader production chain, particularly to consumers. Further, business leaders have no control over this type of challenge. "The sustainability of the commercial links 45 Haiti Open Data, “Textiles and Apparel Industry,” accessed May 8, 2019, http://opendata.investhaiti.ht/pulbare/textiles-and- apparel-industry. Challenges to empowering sustainable local business Lack of access to credit Political instability Financial and technical gaps after LEVE may slow business growth 33 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV does not depend on the LEVE; it depends on the country. With this political crisis, we cannot talk about sustainability, " according to a corporate training manager. Political instability leads to high turnover rates of key personnel within GoH Ministries . With each staffing change, businesses and must begin working (again) with newcomers to educate them on the need of their business and their sector, because, according to several interviewees, the GoH lacks awareness of business needs and business culture. This necessarily creates a discontinuity in agenda, and limits the ability of Haiti-based companies to take advantage of potential business opportunities. For example, Nike has visited Haiti twice in pursuit of i nitiating production. On its most recent visit to the north, while considering the potential for working with MAS Holding, customs withheld MAS’ necessary imported fabrics. A LEVE staff member then visited the relevant Ministry and moved MAS’ file from one office to another to facilitate the fabric’s release to MAS. This simple inconvenience impacted (to some extent) Nike’s willingness to work in Haiti. Businesses expressed major concerned with local political instability, and general invest ments in sectors with few transparent and clear systems and protocols. Additionally, political instability makes it difficult for industry to project profits and losses because of the unpredictability of the effects of political instability. "We are always in a state of instability, it is difficult to program with the socio-political situation,” according to a professional school manager. Financial and Technical Gaps After LEVE May Slow Business Growth Nearly half of the beneficiaries, implementing partners, or government members the ET interviewed stated that LEVE contributed to strengthening the capacity of its beneficiaries. However, they expressed skepticism toward LEVE sustainability because of political instability, limited access to capital , and enduring gaps in the quality of labor available for businesses operating in Haiti. According to interviewees, it is likely that the individual funding needs of MSMEs will not be met if the same socioeconomic and political conditions persist, because existing financial products are not useful. A good number of business managers are also perplexed about the availability of technical support for industry in the future, especially in the A&T sector, since TASC has not yet found financial support for its operations, despite the continuing need for training labor in the sector. The lack of access to training across all three sectors on interest, lack of access to credit, and the unpredictable political situation , are all likely to stymie MSME growth in the absence of an Activity like LEVE, which helped smooth the rougher edges of the local business environment, according to eight respondents. Said one A&T sector beneficiary, “There should be something that provides customer care that is invested in Haiti and knows how to get things done. Need one actor that is great at this. LEVE is a really great to have, but everything will go on because it has to. LEVE made great decisions, they hit root causes and improved supply chains, and mitigated the problems in the Haitian business environment.” CONCLUSIONS LEVE successfully empowered local businesses through its interventions. LEVE pushed companies to develop new systems and SOPs, to work more efficiently , and think bigger. Empowerment has been part and parcel of capacity development and the diversification of businesses interests, which will ultimately help MSMEs and lead firms to more effectively navigate the uncertainty of the Haitian business environment. LEVE’s interventions focused on empowering businesses to work independently of the A ctivity, but also cooperatively in their respective value chains. LEVE’s focus on developing systems and practices that businesses can replicate and expand upon without LEVE are therefore likely to be sustainable. The business linkages are also likely to be sustainable because they do not require LEVE’s assistance , since USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 34 they have already been created; it is incumbent upon local businesses to maintain these connections, and to hopefully expand them. Local businesses have incentive to expand these linkages because they can help grow firm’s revenues. Similarly, if local businesses, MSME or l ead firms, can afford to pay for STTA, then these training and capacity building relationship are durable without LEVE. Finally, LEVE’s work with TVETs was a beneficial first step toward reinvigorating the Haitian labor market. TVETs are now more focused on training students for industry and student job placement , especially since job placement leads to more students attending TVETs, which helps schools generate vital school-sustaining tuition payments. However, linkages to industry remain limited , and TVETs must continually fine-tune the skills they teach to meet the demands of the local labor market. LEVE has been instrumental in helping some TVETs identify ways to generate income outside of tuition payments to help sustain the school. Unfortunately , LEVE’s effort s have not yet been fruitful in the case of TASC, a key A&T training center, which has yet to identify funding for its activities in the absence of LEVE. Ultimately, TVETs that can fund themselves and place students in income -generating jobs are sustainable businesses. RECOMMENDATIONS • Any follow-on USAID-funded Activities should continue building on LEVE’s training and back- office support, such as the booking keeping training provided to SHAISA tomatoes . This light touch approach to business empowerment has been beneficial in aiding the ongoing development of independent and forward-thinking Haitian entrepreneurs. • To ensure that the GoH is aware and involved in future USAID -funded business empowerment Activities, it will be necessary for USAID to nurture personal relationships within the GoH at the technical-office level. Instead of cultivating relationships with Ministers, IPs and USAID program office staff must work together to identify technical staff in relevant Ministries when Activities start. USAID must invite and/or include these individuals in Activity inception phase— even design phase—meetings to ensure that the GoH can provide input and has buy-in. In this way, new Activities can nurture champions inside the GoH, who may be able to help with streamlined Activity implementation, navigating the GoH bureaucracy, and identifying other supporters in the GoH. • While LEVE’s work in the construction sector had notable gaps, many were not the result of LEVE’s approach or work. A future Activity should capitalize on niche needs in Haitian construction firms by pairing local firms with foreign contractors to facilitate business coaching and systems development, much like what LEVE accomplished with Papyrus. • Follow-on USAID A ctivities must identify ways to help construction sector firms with back- office systems, like opening back accounts and maintaining systematic M&E systems to track construction sector firms’ progress against sector-specific indicators, rather than of asking companies to open their books to USAID. • Any ongoing or future construction sector-focused Activities should look toward niche needs in the sector that local firms can address, such as sourci ng qualified local labor or locally producing as many necessary inputs for a construction job, especially as they related to supplementing imported materials. • Any follow-on Activity requires more robust M&E systems, as well as a commitment from USAID to deliver baseline data, especially if USAID will require data -intensive analysis (like ROI). 35 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV Specifically, follow-on Activities must collect more data on employee skills and income before any new interventions begin, as well as how newly trained laborers assess their future earnings, rates at which they have been taxed, and employment history. USAID should collect this from all newly trained laborers (at TASC and/ or TVETs) and MSMEs. Ultimately, any follow-on Activity must collect more targeted i ndicator data across the full duration of interventions. • USAID should commission panel or cross-sectional surveys of LEVE beneficiaries, particularly individuals who acquired new or improved jobs, over the next five years to evaluate the long- term impact of the Activity. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 36 ANNEXES 37 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX A: EVALUATION SCOPE OF WORK STATEMENT OF WORK Performance Evaluation OF Local Enterprise Value Chain Enhancement (LEVE) Project PURPOSE OF THE EVALUATION The purpose of this evaluation is to inform USAID/Haiti’s future programming in terms of support to Micro-, Small, and Medium Enterprises (MSMEs). The findings of the evaluation should shed light on how successful the LEVE project was in achieving the expected results relative to each of the components of the project. Lessons learned and recommendations provided through the final report should orient the new generation of support to MSMEs. The primary stakeholders for this evaluation include USAID/Haiti, RTI, the Haitian MSMEs, Other donors providing support to MSMEs in Haiti, Haitian private sector entities, and the Government of Haiti (GoH). SUMMARY INFORMATION Option 1: For strategies, projects, or activities with one implementing partner Strategy/Project/Activity Name Local Enterprise and Value Chain Enhancement (LEVE) Project Implementer Research Triangle Institute (RTI) International Cooperative Agreement/Contract # AID-521-C-14-00001 Total Estimated Ceiling of the Evaluated Project/Activity(TEC) $34,265,488 Life of Strategy, Project, or Activity December 2013 - June 2019 Active Geographic Regions Port-au-Prince, Saint-Marc and Cap-Haitien Development Objective(s) (DOs) Food and Economic Security Increased USAID Office USAID/Haiti/ Economic Growth and Agricultural Development (EGAD) Office USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 38 BACKGROUND The USAID-funded Local Enterprise and Value Chain Enhancement (LEVE) project strives to increase economic growth and employment opportunities in Haiti. LEVE expands opportunities for micro, small and medium enterprises (MSMEs) to generate employment for Haitian men, women, and youth in the three development corridors: Port-au-Prince, Saint-Marc and Cap-Haitian and in three key sectors – construction, apparel and textile, and agribusiness. LEVE works lead firms in value chains that hold the most potential for growth. LEVE has four components: 1. Enable MSMEs to engage with other value chain actors to mutually create value. 2. Increase MSMEs access to a productive labor pool with relevant skills and competencies. 3. Improve the sustainability of Haitian organizations serving target sectors and corridors. 4. Identify and improve synergies among existing programs and activities. The LEVE contract was signed in December 2013 for an initial three years, and has a two year option, which USAID can exercise at their discretion. Funding for the initial period is approximately $22 million, with an overall funding of $33 million over 5 years. Included is a substantial amount ranging from $3 to $5 million for funding activities to achieve the results, including grants. LEVE’s major goal is to stimulate the creation of sustainable employment. LEVE is managed by RTI International along with three subcontractors: ● RTI International is responsible for the overall project management and all activities in workforce development ● Tetra Tech is responsible for providing technical expertise in the agribusiness and constructions sectors ● J.E. Austin Associates is responsible for providing technical expertise in the apparel sector, and ● Papyrus S.A., a local Haitian firm, is responsible for capacity building activities. LEVE’s activities started in early 2014, with assessments for each sector, in the respective economic corridors: ● Construction: Cap-Haitian and Port-au-Prince ● Agribusiness: Port-au-Prince and Saint -Marc ● Apparel: Cap-Haitian and Port-au-Prince. At the beginning of the project LEVE conducted an in- depth assessment of the value chains in all three sectors. The objectives of the assessments were to identify value chains with high potential, within each sector, develop upgrading strategies, and then with USAID approval, begin to implement the activities. In particular the assessments were trying to identify lead firms/entities that could take ownership of the upgrading activities, building in sustainability from the initial stages. In this way, LEVE would be acting more as a facilitator than a direct implementer. In addition, when identifying the initial value chains and upgrading interventions LEVE attempted to strike a balance between achieving short-term results and longer-term sustainable change within the value chains and subsequently sectors. The assessments were approved by USAID in mid-2014, and the findings were then vetted with the stakeholders to build ownership; with the activities beginning in earnest by the last quarter of 2014. As of the end of the FY15 fiscal year (September 2014), LEVE progressively constructed a critical mass of integrated activities, focused around selected value chains in three sectors. As of the end of FY15, LEVE was implementing over 80 activities with more than 60 client beneficiaries, with activities throughout the value chains, depending upon the needs and the presence and engagement of lead firms. Activities range from working with firms to improve production and marketing related challenges; to assisting to build 39 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV advocacy and service capacity of key industry associations; to build internal capacity of Haitian firms; to link vocational training institutions to labor demand; to underwrite investment risk in new technologies and production processes. LEVE has found that entities often require assistance across several areas; and as such is often facilitating the delivery of services in an integrated fashion, including technical assistance, background studies, training, capacity building and grants. Description of the Problem, Development Hypothesis(es), and Theory of Change Limited economic development has been a long-standing hindrance, barring Haiti from addressing social and economic issues. Prior to the January 2010 earthquake, Haiti faced high levels of unemployment, extreme income inequality, and a limited tax base. The earthquake itself destroyed productive infrastructure, human capital, and inventory, further increasing economic instability. In response, USAID is supporting Haiti’s economic development by rebuilding damaged infrastructure, promoting policies that lead to a friendly business environment, expanding access to financial products, and providing assistance to micro-, small- , and medium-sized enterprises (MSMEs). USAID launched the Local Enterprise and Value Chain Enhancement (LEVE) project to increase economic security. LEVE is designed to create jobs in target industries/sectors in the Port-au-Prince, Saint-Marc, and Cap Haitien development corridors by creating more inclusive and productive value chains. The “value chain approach” is an innovative economic tool that views a market system from input suppliers to end market buyers in its totality, improving competitiveness and growth potential. LEVE Development Hypothesis: If Haitian MSMEs in high potential sectors are engaged with other value chain actors to mutually create value in response to market demand, and they are supported by a productive labor pool with skills and competencies relevant to the target sectors, then those value chains will be more inclusive and productive, thereby leading to increased job creation in Haiti. ( see LEVE Results Framework in annex- MEP 2018 attached). Summary Strategy/Project/Activity/Intervention to be evaluated Consistent with the objectives of the LEVE contract this evaluation will focus on activities related to the components for the project: Objective 1. Enable MSMEs to engage with other value chain actors to mutually create value Under this this objective the evaluation with focus on activities related to the three target sectors of construction, agri-business, and apparel and textile. For each the sectors the evaluation will analyze the different values chains that were identified through the assessments conducted at the beginning of the project (see the three value chain assessment documents conducted for the LEVE project). Objective 2. Increase MSME access to a productive labor pool with relevant skills and competencies For this objective the evaluation will focus on the support provided by the LEVE project to the Technical and Vocational Education and training (TVET) institutions. It will also analyze the coordination with the Institut National de la Formation Professionelle (INFP) and the USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 40 partnership developed between the vocational schools and the private sector entities to facilitate the integration of the graduates into job opportunities. Objective 3. Improve sustainability of Haitian organizations serving target sectors and corridors Under this objective the project was supposed to provide support to the local institutions to help them developed their capacity and to put them on the path of long-term sustainability. The evaluation will look at all the activities that were developed to implement this objective. In particular, the evaluation will look at the Human Institutional Capacity Development(HICD) approach that was developed to assist institutions in their capacity development. It will also analyze the effectiveness of the Organizational Capacity Assessment Tool (OCAT). Objective 4. Identify and improve synergies among existing projects and activities The intent under this objective was for the LEVE project to coordinate with other activities that have been implemented in the same target areas. These activities would include those that were funded by USAID as well as those funded by other donors. Summary of the Project/Activity Monitoring, Evaluation, and Learning (MEL) Plan The LEVE Results Framework (in annex) is interpreted from the bottom up. Four project objectives (POs) are defined, each of which will be achieved through the implementation of Project IRs (PIR), which in effect are USAID SUB-IRs (IIR). In the initial RFP, USAID suggested a group of 18 indicators to measure the progress of those IIRs. In FY14, LEVE reviewed these indicators, and given the information gathered during the assessment and the first year of program implementation, revised them to align with the realities of each of the sectors and in regard to what is meaningful and measurable. Through discussions with USAID, LEVE added six additional indicators to better track program results brining the final total to 24 indicators. Nine F-indicators (USAID’s standard indicators), were included in the original 24. Over the course of the 3-year base period it became apparent that several of the 24 indicators were either unsuitable to track program activities of were rendered impractical due to prevailing socio- political realities. In discussion with USAID it has been agreed that two indicators would be eliminated. PL4 ( percent increase in productivity) does not provide a standard definition for productivity nor prescribe an appropriate means for data collection/aggregation and data on increases in sales and increased investments are already being collected -- adequate proxies for improved productivity. PL11 (# of firms registered as a result of USG assistance) assumes an operating environment conducive to formalization. Substantial structural deficiencies in Haitian central and municipal authorities require significant restructuring, development and investment; rendering this objective beyond the manageable interest of the LEVE project. An additional modification to the results framework is to move the USAID DO indicators to which LEVE is contributing, to down within the LEVE project indicators under Project Objective 1. (See final Monitoring and Evaluation Plan for FY 2018) EVALUATION QUESTIONS 41 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV EQ 1. To what extent and in what ways was LEVE’s approach successful in achieving its four objectives? The evaluation should specifically examine what factors supported and/or hindered its approach identifying the most promising sectors, the best approach to working with the Government of Haiti, and the private sector. EQ 2. To what extent and in what ways have LEVE activities related to the main value chains generated multiplier effects? To answer this question the evaluation should conduct a return on investment analysis using LEVE’s existing data. Then the evaluation should examine the multiplier effects, if any, that reflect the return on investment. EQ 3. To what extent and in what ways has LEVE’s facilitation approach been effective in empowering sustainable local firms/entities? Among other empowerment criteria, the evaluation should consider the level of leadership taken by the firms in the undertaken activities and success in terms of sales, revenue and job creation in the analysis. EVALUATION DESIGN AND METHODOLOGY It is expected that the evaluation team proposes an overall research design to address the evaluation questions and a plan for collecting and analyzing the data. Nevertheless, given the nature of the evaluation questions and limited time available to plan and implement the evaluation, the evaluation design should be based solely on a combination of qualitative techniques to address the evaluation questions. Key informant interviews focus group interviews, field observation, and in depth review of projects reports are among the techniques that the evaluation team should consider when addressing the evaluation questions. Questions Suggested Data Sources (*) Suggested Data Collection Methods Data Analysis Methods 1.To what extent and in what ways was LEVE’s approach successful in achieving its four objectives? The evaluation should specifically examine what factors supported and/or hindered its approach identifying the most promising sectors, the best approach to working with the Government of Haiti, and the private sector. Documents including performance monitoring data, expert knowledge, beneficiaries, Project staff… Key informant interviews, desk review, focus group discussions…… To be determined by evaluation team 2.To what extent and in what ways have LEVE activities related to the main value chains generated multiplier effects? To answer this question the evaluation should conduct a return on investment analysis using LEVE’s existing data. Then the evaluation should examine the multiplier effects, if any, that reflect the return on investment. Documents including performance monitoring data, expert knowledge, beneficiaries, LEVE CBA raw data, Project staff… Secondary quantitative data, , desk review…… To be determined by evaluation team 3. To what extent and in what ways has LEVE’s facilitation approach been effective in empowering sustainable local firms/entities? Among other empowerment criteria, the evaluation should consider the level of leadership taken by the firms in the undertaken activities and success in terms of sales, revenue and job creation in the analysis. Documents including performance monitoring data, expert knowledge, beneficiaries, Project staff… Key informant interviews, focus group discussions, …… To be determined by evaluation team USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 42 DELIVERABLES AND REPORTING REQUIREMENTS 1. Evaluation Work plan: Upon receipt of this Activity Request, Social Impact (SI) shall submit within two weeks a draft work plan to the Contracting Officer’s Representative (COR). The work plan will include: (1) the anticipated schedule and logistical arrangements; (2) a list of the members of the evaluation team, delineated by roles and responsibilities with their level of effort; (3) the identification of other required personnel and relevant local subcontractors, their LOE, roles and responsibilities and qualifications; and (4) the deliverable schedule. 2. Evaluation Design: Within two weeks of approval of the work plan, SI must submit to Contracting Officer’s Representative (COR) an evaluation design (which will become an annex to the Evaluation report). The evaluation design will include: (1) a detailed evaluation design matrix that links the Evaluation Questions in the SOW to data sources, methods, and the data analysis plan; (2) draft questionnaires and other data collection instruments or their main features; (3) the list of potential interviewees and sites to be visited; (4) known limitations to the evaluation design; and (5) a dissemination plan. USAID/Haiti will take up to 5 business days to review and consolidate comments through the COR. Once the evaluation team receives the consolidated comments on the initial evaluation design and work plan, they are expected to return with a revised evaluation design and work plan within 5 business days. 3. In-briefing / inception report: Prior undertaking field work, the evaluation team will have an in- briefing with the USAID/Governance Team and the Evaluation and Survey Services (ESS) COR to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and to clarify any questions or logistic needs. 4. Evaluation Briefing/Presentation: The evaluation team is expected to hold a final presentation in person to discuss the summary of findings and recommendations to USAID within 20 business days after the conclusion of fieldwork. 5. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided in Section IX: Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, the Governance Office will have 10 working business days in which to review and comment on the initial draft, after which point the ESS COR will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report 5 business days hence, and again the Governance Office will review and send comments on this final draft report within 5 business days of its submission. 6. Final Evaluation Report: The evaluation team will be asked to take no more than 15 business days to respond/incorporate the final comments from the Governance Office. The evaluation team leader will then submit the final report to the COR. All project data and records (FGD and KII summary reports) shall be submitted in full and should be in electronic form in easily readable format, organized, and documented for use by those not fully familiar with the intervention or evaluation, and owned by USAID. 43 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV EVALUATION TEAM COMPOSITION The Evaluation Team shall be comprised of three Key Personnel positions: (i) a Team Leader, (ii) an Assistant Team Leader and, (iii) a Cost Benefit Analysis Expert. The Team Leader (TL) is ultimately responsible for the overall management of the evaluation team, coordinating the implementation of the evaluation, assigning evaluation responsibilities and tasks, and authoring the final evaluation report in conformity with this Statement of Work. The TL must be an experienced evaluation expert, with a documented track record of 10 years of experience in the field of evaluation. S/he should have a strong background in the governance and/or public administration field. S/he should be fluent in French and English. S/he should have at least a Master’s Degree in Economics, Statistics, Political Science, or Public Administration. The Assistant Team Leader (ATL) helps the TL in the overall management of the evaluation team and the final products, in conformity with this Statement of Work. The ATL should be familiar with the Haitian administration. S/he must possess excellent writing and interpersonal skills and must be familiar with USAID programs, objectives, and reporting requirements. S/he should have experience in designing and implementing surveys and conducting FGDs. Fluency in French is required. English and Haitian Creole are highly desirable, as is significant prior work experience in Haiti. A Bachelor's degree in Political Science, Public Administration, or a related field is required to ensure that all areas of technical expertise required for the evaluation are effectively covered. The CBA Expert should have extensive experience (at least 10 years) on using cost benefit analysis type to support decision making. S/he should have at least a Master’s Degree in Economics, Statistics, Political Science, or Public Administration and have received continuous training in Cost Benefit Analysis. All team members will be required to provide a signed statement attesting to a lack of conflict of interest or describing any existing conflict of interest. The evaluation team shall demonstrate familiarity with USAID’s evaluation policies and guidance included in the USAID Automated Directive System (ADS) in Chapter 200. EVALUATION SCHEDULE Timing (Anticipated Weeks or Duration) Scheduled Activities 4 Weeks Preparation of the work plan and evaluation design 1 Week USAID review of the work plan and evaluation design and in-briefing 3 Weeks Data collection 3 Weeks Data analysis and Evaluation Briefing 2 Weeks Draft Report writing 3 Weeks USAID review of Draft Report 1 Week Incorporate USAID comments and prepare Final Report b) Estimated LOE in days by activity for a team of two USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 44 Task LOE for Team Lead (TL) LOE for Assistant Team Lead (ATL) LOE for CBA Expert Total LOE in days Document review/desk review/work planning and evaluation design drafting 20 20 10 50 In-brief, evaluation design finalization (including meetings with USAID) 1 1 1 3 Data collection days 15 15 0 30 Data analysis 10 10 10 30 Out-Brief 1 1 1 3 Draft report 10 10 5 25 Final report 5 5 5 15 Totals 62 62 32 156 FINAL REPORT FORMAT The evaluation final report should include an abstract; executive summary; background of the local context and the strategies/projects/activities being evaluated; the evaluation purpose and main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations. For more detail, see “How- To Note: Preparing Evaluation Reports” and ADS 201mah, USAID Evaluation Report Requirements. An optional evaluation report template is available in the Evaluation Toolkit. The executive summary should be 2–5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) The annexes to the report shall include: • The Evaluation SOW; • Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; • All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; • All sources of information, properly identified and listed; and • Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of. • Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team. • Summary information about evaluation team members, including qualifications, experience, and role on the team. In accordance with ADS 201, the contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within three months of the evaluation’s conclusion. 45 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report. 46 • Evaluation reports should represent a thoughtful, well-researched, and well- organized effort to objectively evaluate the strategy, project, or activity. • Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. • The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. • Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. • Evaluation methodology should be explained in detail and sources of information properly identified. • Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. • Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. • If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. 46 See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 46 ANNEX B: DATA COLLECTION TOOLS KEY INFORMANT INTERVIEW GUIDE FOR LEVE BENEFICIARIES Key Informant Interview (KII) Guide       Researcher Name:   Note taker Name: Interview Date:   Respondent Name:   Respondent Position:   Years in Current Position: Sector/Value Chain: Respondent Gender: Respondent Region: Start Time:   End Time:   Hello, I am _______________ and I work with Social Impact, an American company conducting research for the United States Agency for International Development (USAID) Haiti Office. We are conducting a performance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. We want to understand how successful LEVE has been in meeting its objectives and how its work over the past five years can inform future USAID/Haiti work . We intend to interview approximately 40 individuals, and you have been selected to participate in this research because we believe you can provide a unique perspective on LEVE and the Haitian business environment. Indeed, your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. You will not be provided with any payment for your participation in this interview. Your honest responses will help USAID inform and improve its future programming in Haiti . Your name will not be connected to any information you provide in this interview. The data from this interview (excluding your personally identifiable information) will be used by the Social Impact team for analysis purposes only and your answers will only be reported to USAID as part of an aggregated data set. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Do you understand that your participation in this interview is voluntary, unpaid, and that you can end the interview at any point? If you have any follow up questions, please contact Jennifer Mandel at jmandel@socialimpact.com . Are you willing to participate? Yes___ (check) No___ (check) Do you allow voice recording of the interview? Yes___ (check) No___ (check) EVALUATION QUESTION 1: WAS LEVE’S APPROACH SUCCESSFUL IN ACHIEVING ITS FOUR OBJECTIVES? 1. Is your business a micro, small or medium sized enterprise or a large business supported by LEVE? 47 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV 2. Based on your familiarity with LEVE, how would you assess its work, based how it has aided your business? Very positive impact, positive impact, no impact, negative impact, very negative impact? 3. How have you/your business worked with the LEVE Activity? [PROBE: Did LEVE help you with: expanding your production capacity through new equipment, expanding of your distribution capacity, finding new consumers of your goods? Increased revenue, sales, or better trained employees?] 4. Have you or your business received support from any other USAID or international donor projects, and if so, which ones? Did LEVE play any role is connecting your business to other donor interventions? Objective 1 5. How has LEVE impacted your business’s ability to work with other businesses in your value chain or others to increase the value of the good your company produces? [PROBE: has LEVE helped you find sources of raw goods for your business and/or add value to these raw goods?] 6. How do you think the business linkages you have been able to build as a result of LEVE will change or be sustained after LEVE ends? 7. In what ways, if any, has LEVE contributed to expanding your customer base? [PROBE: Is this something you had hoped for?] Objective 2 8. What (if any) type of work-related training or capability building support has your company received through LEVE? 9. How would you rate the training and capacity building support you have received through LEVE, very positively impactful, positively impactful, no impact, negative impact, very negative impact? 10. How has work-related-training and/or capacity building facilitated by LEVE impacted your business? [PROBE: better worker capacity, better book keeping, larger customer base etc.?] 11. How many new employees have you been able to hire as a result of your work with LEVE? 12. What is the distribution of male female and youth among your business’s employees? [PROBE: Has LEVE affected the types of employees you hire in any way?] Objective 3 13. What role did you and others in your organization play in implementing or reinforcing LEVE’s training? 14. In the future, if you need similar technical assistance related to your work, which organizations will you turn to? [PROBE: Has LEVE provided any guidance on future organizations/resources after the Activity ends?] 15. How will the end of LEVE and its support for your business affect your operations, in terms of sales and revenue? 16. Do you think that LEVE’s interventions, and the benefits your business has accrued through it, will be sustainable in the long -term without another Activity like LEVE? 17. How and in what ways do you work with the GoH to facilitate your business interests and facilitate growth? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 48 EVALUATION QUESTION 3: HAS LEVE’S FACILITATION APPROACH BEEN EFFECTIVE IN EMPOWERING SUSTAINABLE LOCAL FIRMS/ENTITIES? To what extent do you think or feel empowered as a business leader as a result of LEVE’s work? 18. How do you think you and the other LEVE beneficiaries, or other business you have worked with as a result of LEVE will function without LEVE? 19. If no activity like LEVE replaces it, what potential challenges do you foresee affecting your business? 49 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV KEY INFORMANT INTERVIEW GUIDE FOR ACTIVITY IMPLEMENTORS (RTI) Key Informant Interview (KII) Guide       Researcher Name:   Note taker Name: Interview Date:   Respondent Name:   Respondent Position:   Years in Current Position: Respondent Gender: Respondent Region: Start Time:   End Time:   Hello, I am _______________ and I work with Social Impact, an American company conducting research for the United States Agency for International Development (USAID) Haiti Office. We are conducting a performance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. We want to understand how successful LEVE has been in meeting its objectives and how its work over the past five years can inform future USAID/Haiti work . We intend to interview approximately 40 individuals, and you have been selected to participate in this research because we believe you can provide a unique perspective on LEVE and the Haitian business environment. Indeed, your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. You will not be provided with any payment for your participation in this interview. Your honest responses will help USAID inform and improve its future programming in Haiti . Your name will not be connected to any information you provide in this interview. The data from this interview (excluding your personally identifiable information) will be used by the Social Impact team for analysis purposes only and your answers will only be reported to USAID as part of an aggregated data set. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Do you understand that your participation in this interview is voluntary, unpaid, and that you can end the interview at any point? If you have any follow up questions, please contact Jennifer Mandel at jmandel@socialimpact.com . Are you willing to participate? Yes___ (check) No___ (check) Do you allow voice recording of the interview? Yes___ (check) No___ (check) EVALUATION QUESTION 1: WAS LEVE’S APPROACH SUCCESSFUL IN ACHIEVING ITS FOUR OBJECTIVES? 1. How would you assess LEVE’s success in achieving its four objectives? Very successful, successful, neutral, unsuccessful, very unsuccessful? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 50 2. How has LEVE programming impacted Port-au-Prince, Cap-Haitian and St. Marc trade areas? 3. What have been the differential impacts of LEVE in the three development corridors? 4. How has LEVE had an identifiable impact on incorporating more women and/or youth into trade - related work? 5. Are some areas of trade more open to women or youth than others? Which and why? 6. What are the most promising value chains in the Haitian three sectors of interest should a follow-on activity to LEVE be designed? Objective 1 7. How has LEVE affected beneficiaries’ ability to work with other businesses within or across value chain to increase the value of the good produced? [PROBE: has LEVE helped beneficiaries source raw goods and/or add value to raw goods?] 8. To what extent will the business linkages built as a result of LEVE change or be sustained after LEVE ends? [PROBE: how would we measure the sustainability of business linkages created by LEVE?] 9. In what ways, if any, has LEVE contributed to expanding beneficiaries’ customer base? [PROBE: Is this data tracked?] Objective 2 10. How would you rate the training and capacity building support for beneficiaries that LEVE has provided, very impactful, somewhat impactful, no impact, somewhat negatively impact, very negatively impactful? Why? 11. How would you rate LEVE’s work with TVETs in terms of impact on the productivity of the Haitian labor pool, very impactful, somewhat impactful, no impact, somewhat negatively impactful, very negatively impactful? Why? 12. What are the primary ways in which work-related-training and/or capacity building facilitated by LEVE has impacted beneficiaries? [PROBE: better worker capacity, better book keeping, larger customer base etc.?]. 13. How are indirect jobs measured by LEVE and how do they contribute to a more productive labor pool with relevant skills and competencies? FOR LEVE ONLY: How did LEVE respond to the OIG report in which its data collection and data quality standards were contested? Objective 3 14. How will the end of LEVE and its support for businesses and job creation affect the operations of beneficiaries, in terms of sales and revenue? 15. Do you think that LEVE’s interventions, and the benefits to beneficiary business that have accrued through it, will be sustainable in the long -term without another activity like LEVE? Objective 4 16. In what ways did LEVE improve synergies between itself and other Donor-funded/USAID projects and activities? a. How did this affect LEVE beneficiaries? 51 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV 17. How can future activities like LEVE (and how does LEVE) best work with the GoH to enable a positive business and job creation environment? EVALUATION QUESTION 3: HAS LEVE’S FACILITATION APPROACH BEEN EFFECTIVE IN EMPOWERING SUSTAINABLE LOCAL FIRMS/ENTITIES? 18. To what extent has LEVE empowered new business leaders through its interventions? 19. How will LEVE beneficiaries will function without LEVE? 20. If no activity like LEVE replaces it, what potential challenges do you foresee affecting beneficiaries? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 52 FOCUS GROUP DISCUSSION GUIDE FOR USAID AND OTHER IPS Focus Group Discussion (FGD) Guide       Researcher Name:   Note taker Name: Interview Date:   Respondent Numbers:   Respondent Ages:   Sector/Value Chain: Respondent Gender: Respondent Region: Start Time:   End Time:   Hello, I am _______________ and I work with Social Impact, an American company conducting research for the United States Agency for International Development (USAID) Haiti Office. We are conducting a performance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. We want to understand how successful LEVE has been in meeting its objectives and how its work over the past five years can inform future USAID/Haiti work . We intend to interview approximately 40 individuals, and you have been selected to participate in this research because we believe you can provide a unique perspective on LEVE and the Haitian business environment. Indeed, your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. You will not be provided with any payment for your participation in this interview. Your honest responses will help USAID inform and improve its future programming in Haiti. Your name will not be connected to any information you provide in this interview. The data from this interview (excluding your personally identifiable information) will be used by the Social Impact team for analysis purposes only and your answers will only be reported to USAID as part of an aggregated data set. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Do you understand that your participation in this interview is voluntary, unpaid, and that you can end the interview at any point? If you have any follow up questions, please contact Jennifer Mandel at jmandel@socialimpact.com . Are you willing to participate? Yes___ (check) No___ (check) Do you allow voice recording of the interview? Yes___ (check) No___ (check) EVALUATION QUESTION 1: WAS LEVE’S APPROACH SUCCESSFUL IN ACHIEVING ITS FOUR OBJECTIVES? 1. How would you assess LEVE’s work over its five-year period of performance? Very successful, successful, neutral, unsuccessful, very unsuccessful? 53 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV 2. What have been the differential impacts of LEVE in the three development corridors? 3. How has LEVE had an identifiable impact on incorporating more women and/or youth into trade - related work? 4. If a follow-on activity to LEVE is designed, what are the most promising value chains within the three LEVE sectors that you think it should focus on? Should it focus on additional sectors in addition to or in lieu of these three? Objective 1 5. How has LEVE most impacted the ability of Haitian businesses to work with other businesses within the targeted value chains? In what ways did LEVE fall short of expectations? Why? 6. Did LEVE adapt its programming to changing conditions on the ground in Haiti? If yes, how? 7. To what extent will the business linkages built as a result of LEVE change or be sustained after LEVE ends? [PROBE: how would we measure the sustainability of business linkages created by LEVE?] Objective 2 8. How would you rate the training and capacity building support for beneficiaries that LEVE has provided, very impactful, somewhat impactful, no impact, somewhat negatively impact, very negatively impactful? Why? 9. How would you rate LEVE’s work with TVETs in terms of impact on the productivity of the Haitian labor pool, very impactful, somewhat impactful, no impact, somewhat negatively impact, very negatively impactful? Why? 10. How are indirect jobs measured by LEVE and how do they contribute to a more productive labor pool with relevant skills and competencies? FOR LEVE ONLY: How did LEVE respond to the OIG report in which its data collection and data quality standards were contested? Objective 3 11. How will the end of LEVE and its support for businesses and job creation affect the operations of beneficiaries, in terms of sales and revenue? 12. Do you think the benefits from LEVE’s capacity-building interventions with Haitian business will be sustainable in the long-term without another activity like LEVE? Objective 4 13. In what ways did LEVE improve coordination and collaboration between itself and other stakeholders, including: a. GoH (e.g., MCI), b. USAID-funded activities (SMASH, HOME, CHABUMA); and c. Non-USAID donor-funded projects (e.g., IDB, EU)? 14. How did this affect LEVE beneficiaries? EVALUATION QUESTION 3: HAS LEVE’S FACILITATION APPROACH BEEN EFFECTIVE IN EMPOWERING SUSTAINABLE LOCAL FIRMS/ENTITIES? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 54 15. To what extent has LEVE empowered new business leaders through its interventions? 16. Did supported firms demonstrate leadership (i.e., take a proactive role) in implementing or reinforcing LEVE’s training? 17. How will LEVE beneficiaries function without LEVE? 55 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV KEY INFORMANT INTERVIEW GUIDE FOR GoH, Chambers of Commerce and Trade ASSOCIATIONS Key Informant Interview (KII) Guide       Researcher Name:   Note taker Name: Interview Date:   Respondent Name:   Respondent Position:   Years in Current Position: Respondent Gender: Respondent Region: Start Time:   End Time:   Hello, I am _______________ and I work with Social Impact, an American company conducting research for the United States Agency for International Development (USAID) Haiti Office. We are conducting a performance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. We want to understand how successful LEVE has been in meeting its objectives and how its work over the past five years can inform future USAID/Haiti work . We intend to interview approximately 40 individuals, and you have been selected to participate in this research because we believe you can provide a unique perspective on LEVE and the Haitian business environment. Indeed, your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. You will not be provided with any payment for your participation in this interview. Your honest responses will help USAID inform and improve its future programming in Haiti . Your name will not be connected to any information you provide in this interview. The data from this interview (excluding your personally identifiable information) will be used by the Social Impact team for analysis purposes only and your answers will only be reported to USAID as part of an aggregated data set. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Do you understand that your participation in this interview is voluntary, unpaid, and that you can end the interview at any point? If you have any follow up questions, please contact Jennifer Mandel at jmandel@socialimpact.com . Are you willing to participate? Yes___ (check) No___ (check) Do you allow voice recording of the interview? Yes___ (check) No___ (check) EVALUATION QUESTION 1: WAS LEVE’S APPROACH SUCCESSFUL IN ACHIEVING ITS FOUR OBJECTIVES? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 56 1. Are you familiar with the USAID-funded LEVE Activity? 2. How would you assess LEVE’s work over the life of the Activity? Very successful, successful, neutral, unsuccessful, very unsuccessful? 3. How has LEVE had an identifiable impact on incorporating more women and/or youth into trade- related work? 4. Has LEVE had any indirect (not as a result of direct program interventions) positive or negative impacts on Haitian businesses and the business environment? [PROBE: job creation, business facilitation etc.] 5. Are some areas of trade more open to women or youth than others? Which and why? 6. What are the most promising value chains in the Haitian three sectors of interest, should a follow- on activity to LEVE be designed? Objective 1 7. How has LEVE affected beneficiaries’ ability to work with other businesses within or across value chain to increase the value of the good produced? [PROBE: has LEVE helped beneficiaries’ source raw goods and/or add value to raw goods?] 8. To what extent will the business linkages built as a result of LEVE change or be sustained after LEVE ends? [PROBE: how would we measure the sustainability of business linkages created by LEVE?] 9. In what ways, if any, has LEVE contributed to expanding its beneficiaries’ customer base? Objective 2 10. How would you rate the training and capacity building support for beneficiaries that LEVE has provided, very impactful, somewhat impactful, no impact, somewhat negatively impactful, very negatively impactful? Why? 11. How would you rate LEVE’s work with TVETs in terms of impact on the productivity of the Haitian labor pool, very impactful, somewhat impactful, no impact, somewhat negatively impact, very negatively impactful? Why? 12. What are the primary ways in which work-related-training and/or capacity building facilitated by LEVE has impacted beneficiaries? [PROBE: better worker capacity, better book keeping, larger customer base etc.?]. Objective 3 13. How will the end of LEVE and its support for businesses and job creation affect the operations of beneficiaries, in terms of sales and revenue? 14. Do you think that LEVE’s interventions, and the benefits to beneficiary business that have accrued through it, will be sustainable in the long -term without another activity like LEVE? Objective 4 15. In what ways did LEVE improve synergies between itself and other Donor -funded/USAID projects and activities? 57 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV 16. How can future activities like LEVE (and how does LEVE) best work with the GoH to enable a positive business and job creation environment? 17. In what ways can business best work with the GoH to grow and create more jobs? EVALUATION QUESTION 3: HAS LEVE’S FACILITATION APPROACH BEEN EFFECTIVE IN EMPOWERING SUSTAINABLE LOCAL FIRMS/ENTITIES? 18. To what extent has LEVE empowered new business leaders through its interventions? 19. How will LEVE beneficiaries will function without LEVE? 20. If no activity like LEVE replaces it, what potential challenges do you foresee affecting beneficiaries? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 58 KEY INFORMANT INTERVIEW GUIDE FOR TECHNICAL AND VOCATION TRAINING CENTERS Key Informant Interview (KII) Guide       Researcher Name:   Note taker Name: Interview Date:   Respondent Name:   Respondent Position:   Years in Current Position: Respondent Gender: Respondent Region: Start Time:   End Time:   Hello, I am _______________ and I work with Social Impact, an American company conducting research for the United States Agency for International Development (USAID) Haiti Office. We are conducting a performance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. We want to understand how successful LEVE has been in meeting its objectives and how its work over the past five years can inform future USAID/Haiti work . We intend to interview approximately 40 individuals, and you have been selected to participate in this research because we believe you can provide a unique perspective on LEVE and the Haitian business environment. Indeed, your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. You will not be provided with any payment for your participation in this interview. Your honest responses will help USAID inform and improve its future programming in Haiti . Your name will not be connected to any information you provide in this interview. The data from this interview (excluding your personally identifiable information) will be used by the Social Impact team for analysis purposes only and your answers will only be reported to USAID as part of an aggregated data set. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Do you understand that your participation in this interview is voluntary, unpaid, and that you can end the interview at any point? If you have any follow up questions, please contact Jennifer Mandel at jmandel@socialimpact.com . Are you willing to participate? Yes___ (check) No___ (check) Do you allow voice recording of the interview? Yes___ (check) No___ (check) 1. What types of skills and training have you provide at this school? 2. What types of jobs (in terms of specific sectors and value chains) are your students most interested in findings after they graduate? 59 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV 3. How and in what ways has this school benefited from support from the USAID-funded LEVE Activity? 4. How satisfied are you with the assistance this school has received from LEVE? [Very satisfied, satisfied, neutral, unsatisfied, very unsatisfied] 5. What other support does this school require to help its students in terms of training and job placement? 6. What factors in the Haitian labor market and business environment in general might prevent graduates of this school from findings jobs and/or using their training? 7. What is the ratio of female to male students at this school? 8. What is the average age of students at this school? 9. In what ways do technical and vocational schools meet the demands of youth trying to enter the Haitian labor market? 10. What can the GoH do to help create a more business enabling and job creating environments? 11. In what ways do TVETs work with the GoH to help their students get jobs? 12. What are some suggestions for improving professional training schools for the future? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 60 FOCUS GROUP DISCUSSION GUIDE FOR TVET STUDENTS Focus Group Discussion (FGD) Guide       Researcher Name:   Note taker Name: Interview Date:   Respondent Numbers:   Respondent Ages:   Sector/Value Chain: Respondent Gender: Respondent Region: Start Time:   End Time:   Hello, I am _______________ and I work with Social Impact, an American company conducting research for the United States Agency for International Development (USAID) Haiti Office. We are conducting a performance evaluation of the Local Enterprise Value Chain Enhancement (LEVE) Activity. We want to understand how successful LEVE has been in meeting its objectives and how its work over the past five years can inform future USAID/Haiti work . We intend to interview approximately 40 individuals, and you have been selected to participate in this research because we believe you can provide a unique perspective on LEVE and the Haitian business environment. Indeed, your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. You will not be provided with any payment for your participation in this interview. Your honest responses will help USAID inform and improve its future programming in Haiti . Your name will not be connected to any information you provide in this interview. The data from this interview (excluding your personally identifiable information) will be used by the Social Impact team for analysis purposes only and your answers will only be reported to USAID as part of an aggregated data set. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Do you understand that your participation in this interview is voluntary, unpaid, and that you can end the interview at any point? If you have any follow up questions, please contact Jennifer Mandel at jmandel@socialimpact.com . Are you willing to participate? Yes___ (check) No___ (check) Do you allow voice recording of the interview? Yes___ (check) No___ (check) 1. What types of skills and training have you received at this school? 2. What types of jobs (in terms of specific sectors and value chains) are you interested in findings after you graduate? 61 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV 3. Do think that the training you have received at this school will help you find the types of jobs you are interested in? 4. How satisfied are you with the training activity? [Very satisfied, satisfied, neutral, unsatisfied, very unsatisfied] 5. What types of job placement services are available for you through this school? 6. Have you or any of your colleagues met or worked with any people affiliated with a program called LEVE? If yes, tell us about these meetings. 7. Were there any factors that prevented you or other graduates from applying this new training? 8. What, if any, are the barriers in Haiti that affect your ability to find work in the sectors and value chains you are interested in? 9. What are some suggestions for improving professional training schools for the future? USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 62 ANNEX C: SOURCES OF INFORMATION Agribusiness Sector Assessment Apparel and Textile Value Chain Assessment Center for Facilitation of Investments, Textiles and Apparel Industry , via OpenData.InvestinHaiti.ht, http://opendata.investhaiti.ht/pulbare/textiles-and-apparel-industry?lang=en#. Cicowiez, Martín and Agustín Filippo (2018). A Simple Stylized Long-Run Growth Model for Haiti. IDB - TN-1485. Inter-American Development Bank Technical Note, 2018. Congressional Research Service, “Haiti’s Political and Economic Conditions,” July 1, 2019, https://fas.org/sgp/crs/row/R45034.pdf, p. 1. Construction Sector Assessment Daniel Runde, “Haiti’s Governance Problems Require Touch Love From Donors,” Forbes, April 17, 2015, accessed May 7, 2019, https://www.forbes.com/sites/danielrunde/2015/04/17/haitis-governance- problems-require-tough-love-from-donors/#13657df82917. Haitian Institute of Statistics and Informatics, “Economic Characteristics: Main Occupation,” Ministry of Economics and Finances, 2016, accessed May 8, 2019, http://www.ihsi.ht/rgph_resultat_ensemble_CE.htm. LEVE Annual Reports: FY14 - FY18 LEVE Contract, section C LEVE M&E Plans: FY 14 -18 LEVE Quarterly Reports: FY14 – Q2 and Q3; FY15 – Q1, Q2, and Q3; FY16 – Q1, Q2, and Q3; FY 17 – Q1, Q2, and Q3; FY 18 – Q1, Q2, and Q3; FY19 – Q1 LEVE Work Plans: FY14 - FY19 LEVE Workforce Assessment Report Local Enterprise and Value Chain Enhancement (LEVE) Project: Measuring Job Impacts in the Haitian Apparel Sector Technical Report. February 2017. USAID. Haitian Ministry of Economy and Finances, Income Tax Filing Form 2018, https://dgi.gouv.ht/wp- content/uploads/2018/01/DECLARATION-DEFINITIVE-DIMPOT-SUR-LE-REVENU-Version-Simplifiee- Categorie-Traitement-et-Salaires.pdf. Maureen Taft-Morales, “Haiti’s Political and Economic Conditions: In Brief, “ Congressional Research Service,” December 1, 2017, accessed May 6, 2019, https://fas.org/sgp/crs/row/R45034.pdf. See also, Franco Ordeñez and Jacqueline Charles, “Trump Administration Extends TPS for Haitian and Three other Groups,” McClatchy Dc Bureau, March 1, 2019, accessed May 7, 2019, https://www.mcclatchydc.com/news/politics-government/white-house/article226943724.html . Pan American Development Foundation, “Empowering Haitian Rice Farmers,” via PADF.org, August 15, 2016. https://www.padf.org/news/2016/3/3/empowering-haitian-rice-farmers . 63 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV The World Bank, “The World Bank in Haiti: Overview,” April 5, 2019, accessed May 7, 2019, https://www.worldbank.org/en/country/haiti/overview . The World Bank, “Bringing HOPE to Haiti’s Apparel Industry,” November 2009, p.36, http://documents.worldbank.org/curated/en/715301468256506127/pdf/533160ESW0P11410Report10Jan uary2010.pdf. United Nation Develop Programs, “Human Development Report: Haiti,” accessed May 19, 2019, http://hdr.undp.org/en/countries/profiles/HTI. USAID/Haiti, “Value Chain Assessment Annex 1. Apparel and Textiles Sector Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. iii. USAID/Haiti, “Value Chain Assessment Annex 1. Apparel and Textiles Sector Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. 2. USAID/Haiti, ““Value Chain Assessment Annex 2. Construction Industry Assessment: Local Enterprise and Value Chain Enhancement (LEVE) Project,” April 2014, p. 5. USAID/Haiti, “Value Chain Assessment Annex 3. Agribusiness Sector Assessment: Local Enterprise and value Chain Enhancement (LEVE) Project,” April 2014, p. 5. USAID/Haiti, “Local Enterprise and Value Chain Enhancement (LEVE) Project: FY 15 Annual Report (Including Q 4 Summary),” October 2015. USAID/Haiti, “Local Enterprise and Value Chain Enhancement (LEVE) Project: FY 17 Annual Report (Including Q 4 Summary),” October 2017. USAID/Haiti, “Local Enterprise and Value Chain Enhancement (LEVE) Project: FY 18 Annual Report (Including Q 4 Summary),” October 2018. Varshney, Nitish. “Can Haiti mark solid presence as ‘Promising’ apparel manufacturing hub.” Apparel Resources, January 19, 2019. http://apparelresources.com/business-news/sourcing/can-haiti-mark-solid- presence-as-promising-apparel-manufacturing-hub/. USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 64 ANNEX D: DISCLOSURES OF CONFLICT OF INTEREST Team Leader, Zuri Linetsky 65 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV Assistant Team Leader/Sector Specialist, Phanol Philippe USAID.GOV USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT | 66 Return on Investment Analyst, Paul Rivera 67 | USAID/HAITI LEVE PERFORMANCE EVALUATION REPORT USAID.GOV ANNEX E: EVALUATION TEAM MEMBERS Evaluation Team Leader, Dr. Zuri Linetsky, is a practiced M&E specialist with 11 years’ work experience in methodological and technical training, qualitative and quantitative data collection and analysis, as well as team leadership, project and budget management. He has significant regional work experience in the Middle East, Africa and Eastern Europe, particularly in conflict zones such as Egypt, Lebanon, Palestine, South Sudan, Nigeria, Syria, and Somalia. At Social Impact Zuri has led data collection for a water sector performance evaluation in Lebanon, Jordan and Morocco in addition to serving as TL on a performance evaluation of USAID/Kosovo programming on property rights and gender. Prior to joining Social Impact, he was an M&E Expert where he led field teams in collecting quantitative and qualitative data in Somalia and Palestine. He also led the research design and survey methodology implementation for an impact evaluation in northeast Nigeria. In Egypt, Dr. Linetsky lead a team of 12 in the production of impact evaluations, with primary source data collection in South Sudan, Lebanon and Syria. He has used a variety of methodologies and statistical software including Stata, SPSS, R Statistical software, MAXQDA and Dedoose. He is also familiar with USAID reporting regulations and requirements. Dr. Linetsky holds a Ph.D. in Foreign Affairs from the University of Virginia, an M.A. in International Relations from the University of Chicago, and a B.A. in Political Science from the University of Florida. He is fluent in English, Russian and Hebrew, and has intermediate skills in Spanish, Arabic and French. Assistant Team Leader, Mr. Phillippe Phanol, has over 18 years in design, M&E of programs in various areas, including trade and agriculture. Since 2010, Mr. Phanol has worked as an evaluation consultant in Haiti, gaining in-depth knowledge of Haiti’s emergency response challenges as well as its agricultural supply chain challenges. As recently as 2016, Mr. Phanol was a TL for the AVANSE mid-term evaluation. He also has led teams to evaluate natural resource, and transparency and accountability projects in Haiti. He holds a Master of Science from Universidad de los Andes and the University of Maryland. Return on Investment Expert, Paul Rivera, is an international development professional with over 20 years’ experience in cost-benefit/effectiveness analysis, strategic planning, technical analysis, monitoring and evaluation with technical expertise in economic growth and agriculture. Project Director, Adam Reisman, has more than 15 years of experience monitoring, evaluating, designing, and implementing democracy & governance, conflict-mitigation, and capacity- building programs in fragile and post-conflict states. Mr. Reisman has held senior-level M&E and research positions with USAID, the U.S. Department of State (DoS), and a range of non-governmental organizations (NGOs) and donors. His in-depth M&E experience is supported by extensive knowledge of qualitative and quantitative tools and methodologies, and of USAID and DoS M&E practices and requirements. Currently, Mr. Reisman serves as a Senior M&E Technical Specialist with SI’s Performance Evaluation Team, where he provides technical leadership and expertise in the development of performance management tools and performance evaluations of USAID projects both overseas and in Washington, DC. Mr. Reisman holds an MA in International Affairs from the George Washington University.