(2018) Le projet de croissance économique de l'USAID n'était pas en voie de faciliter des chaînes de valeur plus productives et inclusives en Haïti

Office of Inspector General, U.S. Agency for International Development USAID’s Economic Growth Project Was Not On Track To Facilitate More Productive and Inclusive Value Chains in Haiti OFFICE OF INSPECTOR GENERAL U.S. Agency for International Development 1300 Pennsylvania Avenue NW • Washington, DC 20523 https://oig.usaid.gov  202-712-1150 AUDIT REPORT 1-521-18-001-P JUNE 14, 2018 Office of Inspector General, U.S. Agency for International Development The Office of Inspector General provides independent oversight that promotes the efficiency, effectiveness, and integrity of foreign assistance delivered through the entities under OIG’s jurisdiction: the U.S. Agency for International Development, U.S. African Development Foundation, Inter-American Foundation, Millennium Challenge Corporation, and Overseas Private Investment Corporation. Report waste, fraud, and abuse USAID OIG Hotline Email: ighotline@usaid.gov Complaint form: https://oig.usaid.gov/content/oig-hotline Phone: 202-712-1023 or 800-230-6539 Mail: USAID OIG Hotline, P.O. Box 657, Washington, DC 20044- 0657 Office of Inspector General, U.S. Agency for International Development San Salvador, El Salvador https://oig.usaid.gov MEMORANDUM DATE: June 14, 2018 TO: USAID/Haiti Mission Director, Jene Thomas FROM: Acting Regional Inspector General/San Salvador, Van Nguyen /s/ SUBJECT: USAID’s Economic Growth Project Was Not O n Track To Facilitate More Productive and Inclusive Value Chains in Haiti (1-521-18-001-P) This memorandum transmits the final report on our audit of USAID/Haiti’s Local Enterprise and Value-Chain Enhancement (LEVE) Project. Our audit objectives were to (1) determine if USAID/Haiti's LEVE Project was achieving its goal to facilitate more productive and inclusive value chains that will contribute to broad -based economic growth and (2) review the mission’s monitoring of project performance and impact. In finalizing the report, we considered your comments on the draft and included them in their entirety, excluding attachments, in a ppendix C. The report contains two recommendations to improve USAID/Haiti’s management of the project. After reviewing information you provided in response to the draft report, we consider both recommendations closed. We appreciate the assistance you and your staff extended to us during this audit. Office of Inspector General, U.S. Agency for International Development CONTENTS INTRODUCTION .............................................................................................................................. 1 SUMMARY ............................................................................................................................................ 1 BACKGROUND ................................................................................................................................. 2 THE LOCAL ENTERPRISE AND VALUE- CHAIN ENHANCEMENT PROJECT WAS NOT ACHIEVING ITS GOAL ......................................................................................................... 3 Businesses Required Much More Extensive Training and Capacity Building Than Anticipated ....................................................................................................................................... 4 Project’s Focus on Supporting Smaller Enterprises Did Not Align With Job Creation Targets .............................................................................................................................................. 4 Constraints in the Value Chains Limited Progress ................................................................. 4 USAID/HAITI DID NOT MONITOR PROGRESS OR IMPACT EFFECTIVELY ................. 5 USAID/Haiti Did Not Establish Baselines for Key Indicators ............................................... 6 USAID/Haiti Did Not Ensure Performance Data Met Quality Standards ......................... 6 CONCLUSION ................................................................................................................................... 8 RECOMMENDATIONS .................................................................................................................... 8 OIG RESPONSE TO AGENCY COMMENTS ............................................................................. 8 APPENDIX A. SCOPE AND METHODOLOGY ........................................................................ 9 APPENDIX B. LIST OF REQUIRED INDICATORS ................................................................ 11 APPENDIX C. AGENCY COMMENTS ..................................................................................... 12 APPENDIX D. MAJOR CONTRIBUTORS TO THIS REPOR T ............................................ 15 Office of Inspector General, U.S. Agency for International Development 1 INTRODUCTION A devastating earthquake in 2010 destroyed much of Haiti’s productive infrastructure and disrupted the entire Haitian economy. As the country struggled to recover, the Haitian Government recognized that micro-, small, and medium-size enterprises (MSMEs) could play an important role in creating new jobs and improving working conditions in the country. 1 The Department of State notes in its 2011 post-earthquake strategy for Haiti 2 that MSMEs accounted for about 90 percent of new jobs in the country, but most operated informally—often run by families or individuals without formal accounting systems or Haitian Government registration—and it was hard for them to access finance or technical support through official channels, like banks. They also lacked connections to value chains—all the activities such as production, marketing, and distribution required to produce a product or service and deliver it to the final customer—beyond local communities. These problems have limited the abilities of MSMEs to grow and stimulate the Haitian economy. USAID/Haiti developed the Local Enterprise and Value-Chain Enhancement (LEVE) Project to support MSMEs and create jobs by making the value chains more productive and inclusive. In December 2013, the mission awarded a contract, which has grown to over $32 million, to implement the project. According to the contract, progress toward this goal would be measured by (1) the number of jobs created, (2) increased sales in the value chain, (3) increased investment, and (4) increased productivity. The Office of Inspector General (OIG) conducted this audit to (1) determine if USAID/Haiti's LEVE Project was achieving its goal to facilitate more productive and inclusive value chains that will contribute to broad- based economic growth and (2) review the mission’s monitoring of project performance and impact. To conduct our work, we reviewed project documentation; interviewed officials from USAID, contractor staff, and Haitian businesses that the project worked with; tested performance data; and conducted site visits to project-supported businesses, where we observed activities and interviewed project beneficiaries. A ppendix A contains a detailed description of the scope and methodology. SUMMARY We found that the project was not on track to achieve its goal to facilitate more productive and inclusive value chains. Project activities had done little to produce inclusive value chains, generate broad-based economic growth, or increase employment. 1 USAID/Haiti adopted the Professional Bank Association’s definition of MSMEs as businesses with between 10 and100 employees — a generally accepted threshold used in the country. 2 Department of State, “Post-Earthquake Haiti Strategy: Toward Renewal and Economic Opportunity,” January 3, 2011. Office of Inspector General, U.S. Agency for International Development 2 Multiple factors stalled progress. For example, the project’s first year focused on planning and updating value-chain assessments that would inform future activities, leaving just 2 years to achieve the ambitious 3- year targets. The contractor’s initial plans underestimated training and capacity-building needs , and more work was required up front to prepare and engage businesses, putting off targeted increases in jobs, investments, and sales. Furthermore, the project’s focus on assistance to MSMEs, which tend to hire few employees, made it difficult for the project to create a large number of new jobs. In addition, constraints in the value chains, like the slow growth in a new industrial park and a lack of regulations and structure in the construction sector, proved difficult to overcome. As a result, the project was behind in its targets related to increased jobs, sales, and assistance to MSMEs. In addition, the mission did not monitor the project’s progress or overall impact effectively. Because USAID/Haiti did not establish baselines for key indicators measuring increases in sales, investments, and productivity, the mission could not clearly identify the project’s contribution to the increased productivity of value chains and economic growth. Furthermore, the mission did not ensure performance data met quality standards, which reduced the data’s value. We made two recommendations to improve USAID/Haiti’s implementation and monitoring of the LEVE Project . BACKGROUND USAID/Haiti developed LEVE to target three designated development corridors and three value chains: construction, agribusiness, and textiles and apparel. 3 The contract lists four objectives: 1. Help MSMEs connect with other entities in the value chains, including established businesses. 2. Connect MSMEs to useful and qualified labor pools. 3. Strengthen the capacity of Haitian organizations to operate sustainably. 4. Identify and unite project efforts with other donor activities. In December 2013, the mission awarded a 3- year, $22.7 million contract to RTI International. The contract included a 2- year option period worth an additional $10 million, and the mission modified the contract in July 2016 to exercise this option— extending the project to December 2018. 3 The post-earthquake strategy for Haiti identified Port-au-Prince, Saint-Marc, and Cap-Haitien as the development corridors on which USAID would focus most of its programming. Office of Inspector General, U.S. Agency for International Development 3 THE LOCAL ENTERPRISE AND VALUE -CHAIN ENHANCEMENT PROJECT WAS NOT ACHIEVING ITS GOAL Through its contract with RTI International, USAID/Haiti set targets for creating 6,000 full-time-equivalent jobs in 3 years. 4 The project also planned to increase the investments of 155 businesses, increase the sales of 300 businesses, and provide 800 microenterprises (businesses operating on a very small scale), with U.S. Government assistance within this time . However, at the time of the audit, the project was not on track to achieve these ambitious targets. As shown in the table below, the project had: • Created 628 new jobs, about 10 percent of the target. Most were created by one large lead business —not an MSME—that received project assistance. • Increased investments for six businesses, only 4 percent of its target. Five of these six businesses received funding from the project enabling them to make the reported investments. • Increased sales for 23 businesse s, only 8 percent of its target. Some of these businesses reported increased sales in one quarter followed by reduced sales in later quarters, raising questions about the project’s data quality and sustained impact on sales. • Assisted 223 microenterprises only 28 percent of its target. The assistance, mostly training, usually did not result in new jobs, investment, or sales for the participating businesses. 4 In the contract signed December 2013, the mission expected the contractor to create between 8,000 and 14,000 direct jobs; the mission later reduced the target to 6,000 to achieve consistency among the solicitation, the contractor’s original proposal, and the contract. The contractor defined a full-time- equivalent job as a job worked 260 person-days per year. They also noted that a direct job comes from the project; an indirect job comes as a result of the direct job. Office of Inspector General, U.S. Agency for International Development 4 Reported Progress on Selected Performance Indicators , December 31, 2015 Performance Indicator Target December 2016 Reported Result, December 2015 Percent of Target Achieved Number of jobs created in target sectors in designated corridors 6,000 628 10 Number of businesses increasing their investments 155 6 4 Number of businesses in target value chains with increased sales 300 23 8 Number of microenterprises supported by U.S. Government enterprise assistance 800 223 28 Source: LEVE Project FY 16 Quarterly Report. USAID and contractor staff explained that the project’s first year focused on planning and performing an assessment of each targeted value chain to update old information from the project’s design. The contractor relied on these assessments to inform the project’s future activities, leaving just 2 years to achieve the ambitious 3- year targets. Beyond that, we noted the following reasons the project was behind in meeting targets . BUSINESSES REQUIRED MUCH MORE EXTENSIVE TRAINING AND CAPACITY BUILDIN G THAN ANTICIPATED In its early work plans, the contractor acknowledged that activities were fluid and would depend on “ the state of preparedness and level of engagement of value-chain stakeholders.” It turned out that businesses were not prepared and needed more training and capacity building than anticipated. For example, the project provided 7,050 hours of training to employees of microenterprises in FY 2015, surpassing the 350 hours it had budgeted by more than 2,000 percent. And, at the time of the audit, the project was identifying, assessing, and supporting lead business es to create opportunities for MSME engagement. PROJECT’S FOCUS ON S UPPORTING SMALLER ENTERPRISES DID NOT ALIGN WITH JOB CREATION TARGETS According to the U.S. Government’s post-earthquake strategy for Haiti, stability and economic growth required support for MSMEs given their potential as an engine for broad-based economic growth. However, according to contractor personnel, only large companies that employed 100 people or more could generate the number of jobs specified in the contract. In their view, each MSME could add only a few jobs, so the project would need to work with an unrealistic number of MSMEs to reach the employment target. After over 2 years, USAID/Haiti and the contractor decided to count indirect jobs traceable to the project in the job creation total, diluting the ability to measure progress toward the target. CONSTRAINT S IN THE VALUE CHAIN S LIMITED PROGRESS The following constraints proved difficult to overcome during implementation: Office of Inspector General, U.S. Agency for International Development 5 • To create jobs, the project targeted the textiles and apparel value chain in the Port- au-Prince and Cap- Haitien development corridors. Yet, the contractor said a lack of available facilities for business expansion in Port -au-Prince and the slow growth of the new industrial park in the Cap- Haitien corridor limited the value chain’s ability to grow. • Haiti’s construction sector lacked regulations and was dominated by foreign companies and personal relationships. Because this limited how the project could engage with the construction sector, the contractor said the focus shifted to enhancing the skills of those who were already employed, rather than creating new jobs. • The greatest opportunity for success in the agribusiness value chain is in the Cap-Haitien corridor; however, to avoid overlap with another USAID project, the mission limited LEVE’s work in this sector to the Port-au-Prince and Saint-Marc corridors. USAID and contractor officials acknowledged performance delays but were optimistic that results would increase over time. Thus, i n July 2016, the mission exercised the contract’s option years, extending the implementation period to December 2018 and increasing total estimated costs to $32.6 million. When the contract was signed in December 2013, USAID/Haiti hoped the additional 2 years would allow the project to double the contract’s original job creation target. However, until it focuses more on high-impact areas, USAID may miss an opportunity to effect lasting changes to Haiti’s significant unemployment and income disparity. The project could benefit from an assessment to determine which areas to focus on to spur economic impact in the time remaining on the contract. USAID/HAITI DID NOT MONITOR PROGRESS OR IMPACT EFFECTIVELY USAID guidance requires missions to assess projects and learn from them through the continuous collection of performance data. 5 The data should reveal whether the project is on track to achieve intended goals or if it needs adjustments. However, the mission did not monitor progress or impact effectively. Specifically, the mission did not establish baselines for key indicators or ensure that performance data met quality standards. 5 Effective September 7, 2016, USAID replaced its guidance on the program cycle (Automated Directives System [ADS] chapter 203) with revised chapters 200 and 201. However, ADS 203.1, on continuous data collection, was in effect during this audit. Office of Inspector General, U.S. Agency for International Development 6 USAID/HAITI DID NOT ESTABLISH BASELINES FOR KEY INDICATORS According to USAID guidance, baselines must be established for each indicator at the beginning of the project to accurately track project progress. The guidance goes on to state that baselines are “required in order to learn from and be accountable for the change that occurred during the [project] with the resources allocated to that [project].” 6 However, at the time of our audit and two years into the contract, USAID/Haiti had not established baselines for three of the four key indicators intended to measure overall success and impact; the fourth indicator, the number of jobs created by the project, started at zero and did not require additional baseline information. The key indicators were: • Increase in sales (volume and U.S. dollar value) in target value chains • Percent or U.S. dollar value increase in investment in target value chains • Percent increase in productivity (modified to fit each targeted value chain) • Number of new jobs (full-time equivalents) created in target sectors in designated corridors The contractor relied on USAID/Haiti to set baselines for these indicators, but the mission could not find a qualified, independent company to perform the study of what realistic baselines would be. The lack of baselines skewed USAID’s ability to measure progress on these indicators. USAID/HAITI DID NOT ENSURE PERFORMANCE DATA MET QUALITY STANDARDS USAID guidance says that performance data should reasonably meet outlined quality standards. 7 It also says that managers should know the strengths and weaknesses of all indicators they use for project monitoring. 8 We found that perfor mance data did not meet these standards and that USAID/Haiti was not sufficiently aware of the weaknesses, limiting the data’s value for monitoring and decision making. USAID/Haiti and the contractor said challenges staffing the project’s monitoring and ev aluation (M&E) team contributed to these shortcomings, as outlined below: • Insufficient data quality a ssessments. At the time of the audit, the mission had performed data quality assessments (DQAs) of the following indicators: Percent change in USD value of micro entrepreneurs input purchases , and Number of persons receiving new or better employment as a result of U.S. Government-funded workforce development programs. However, the mission had not assessed the remaining 15 of the 17 indicators USAID required the contractor to collect data on ( listed in 6 ADS 203.3.9, “Setting Performance Baselines and Targets.” 7 ADS 203.3.11.1, “Data Quality Standards.” 8 ADS 203.3.11.2, “Purpose of Data Quality Assessments.” Office of Inspector General, U.S. Agency for International Development 7 appendix B). USAID/Haiti thus had not assessed the quality of key indicators measuring project progress, success, and impact. USAID/Haiti said that it prioritized DQAs for standard State Department indicators , some of which were reported annually to Congress. While the mission saw this level of review as sufficient, we believe it needed deeper review of indicators measuring project success in light of the project’s ambitious targets and USAID’s multimillion-dollar investment. • Weak supporting documentation of reported data. The contractor’s M&E team acknowledged discrepancies between some data reported to USAID and supporting documentation. For two indicators measuring job placement, the contractor did not collect any data, because the beneficiary institutions did not have a system in place to maintain vital documentation . For 11 of the 21 indicators that were being tracked, the contractor received data from MSMEs and lead businesse s on worksheets with no additional supporting documentation, like financial reports showing increases in sales, to validate it. The contractor said asking for more documentation would have overburdened the small businesses and that its level of reporting aligned with the project’s approved M&E plan. Yet we found some of the data sources and collection methods that USAID approved—like “reports generated by the beneficiary firms”—were too general to ensure quality and relied too heavily on self-reporting. The contractor said that it checked data randomly during site visits, but it could not provide us with documentation of its checks. The mission also identified documentation weaknesses with its 2016 DQA that were not corrected at the time of audit fieldwork. • Unclear reporting. We found that USAID/Haiti and the contractor interpreted several indicators differently, leading to unclear reporting. For an indicator measuring the value of activities coordinat ed with other USAID projects and other donors (linked to the project’s fourth objective), at least $250,000 of the $1.3 million the contractor reported for an activity took place years before LEVE started and was thus not directly attributable to the project. Mission officials did not agree with this methodology; however, an RTI representative said USAID/Haiti had not provided sufficient guidance on reporting the value of activities. In the absence of guidance, the contractor opted to report the activities’ full value—to avoid subjectivity. The mission and contractor also disagreed whether the job creation target should include indirect jobs and whether the mission should revise targets to account for conditions that had changed since the project’s design. An RTI representative said discussions with the mission about indicator definitions, data collection methods, and calculations still were not resolved more than 2 years into the project. The mission staff said it was reluctant to change targets or indicators in a way that would allow the contractor to produce less than it was contractually obligated to produce. Without baselines or quality performance data, the mission is hindered in its ability to properly assess the project’s progress and impact, or make prompt and informed decisions to keep the project on track. Office of Inspector General, U.S. Agency for International Development 8 CONCLUSION Economic growth is vital to Haiti’s recovery from the devastating 2010 earthquake. By supporting MSMEs and strengthening value chains through the LEVE Project, USAID/Haiti has the opportunity to increase employment, improve productivity , and aid in the country’s economic recovery. However, a slow start and implementation challenges put the project off track. USAID/Haiti added time and money to the project and reduced its number of jobs targets to overcome setbacks, but it did not address the underlying issues that hindered the project’s ability to attain intended results , including a lack of focus on those areas with the highest potential for economic impact. Further, ineffective monitoring left the mission without quality information on project progress and impact. Until USAID/Haiti re focuses its activities and improves monitoring, it will be challenged to maximize the project’s impact on Haitian livelihoods and Haiti’s economy. RECOMMENDATIONS To improve project outcomes and monitoring, we recommend that USAID/Haiti take the following actions: 1. Conduct an assessment and implement a plan to align activities in each value chain to areas with the greatest potential for economic impact in the time remaining. 2. Revise the monitoring and evaluation plan and implement procedures to more effectively measure and evaluate project success and impact. OIG RESPONSE TO AGEN CY COMMENTS We provided our draft report to USAID on August 18, 2017, and on October 10, 2017, received its response, which is included as appendix C . The report included two recommendations. We consider both of them closed . Office of Inspector General, U.S. Agency for International Development 9 APPENDIX A. SCOPE AND METHODOLOGY We conducted our work from October 2015 through August 2017 in accordance with generally accepted g overnment auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We conducted this audit to determine whether USAID/Haiti’s Local Enterprise and Value-Chain Enhancement Project was achieving its goal to facilitate more productive and inclusive value chains, that will contribute to broad-based economic growth and - review the mission’s monitoring of project performance and impact. On December 23, 2013, USAID/Haiti signed a 3- year, $22.7 million cost-plus-fixed-fee contract with RTI International, with a 2- year option period. We covered program activities from inception through December 31, 2015, representing about two-thirds of the base period. As of December 31, 2015, USAID/Haiti reported obligations of $16.7 million of the contract amount and disbursed $9.2 million. This represents the dollar value of activities subject to this audit. We conducted fieldwork from January 13 through May 17, 2016. We visited sites in the Port-au-Prince corridor, where the contractor carried out the majority of the project’s activities. Civil unrest prevented site visits in the Saint-Marc and Cap-Haitien corridors. However, we were able to review relevant project data and perform review steps to audit the activities in these areas. In planning and performing the audit, the audit team assessed the management controls the mission used to monitor the program. We assessed whether the mission (1) reviewed and tested indicator targets and results, (2) reviewed and approved required deliverables, (3) performed data quality assessments and a portfolio review, and (4) conducted and documented site visits to evaluate progress and monitor quality. To gain an understanding of the program, the audit team reviewed the mission’s contract with RTI International, its modifications, the activity approval document 9 , and the U.S. Government post -earthquake strategy for Haiti. We also reviewed work plans, quarterly and annual reports, data quality assessments, monitoring and evaluation plans, and the project’s financial data as of December 31, 2015. To validate reported results, we judgmentally selected local consortium members, subgrant recipients, and other technical assistance recipients in the Port-au-Prince corridor. We based our selection on (1) location—security and travel restrictions 9 The activity approval document is a comprehensive plan that includes intended results, the roles of partners and plans for management oversight, procurement and monitoring and evaluation. Office of Inspector General, U.S. Agency for International Development 10 limited our visits to the Port- au-Prince development corridor; (2) subaward amount— LEVE provided no grants larger than $250,000, so we looked at subgrants close to that awarding limit; and (3) highest expected results—if the subawardee contributed to multiple indicators, the percentage of cumulative targets the subawardee contributed to and similar numbers. We assessed the reliability of reported data by verifying all 10 indicators the program used as of September 30, 2015. We selected at least one reported value for each indicator, tested the reported results by tracing them to supporting documentation and interviewing project beneficiaries during site visits. To answer the audit objectives , we evaluated the project’s reported progress, supporting documentation, and testimonial evidence. We also considered the mission’s assessment and implementation of internal controls, its management and oversight of RTI International’s performance, and the effectiveness and sustainability of activities. We met with various USAID/Haiti officials, including the c ontracting officer’s representative (COR), alternate COR, and M&E specialist. We held numerous meetings with RTI International representatives, including the chief of party, value-chain leads, and the M&E team. We also visited and confirmed information with 8 of the 48 beneficiary institutions. In addition, to determine the progress made toward the project goals, we relied in part on computer-processed data contained in quarterly and annual progress reports prepared by the contractor using Microsoft Excel. We assessed the quality of this data by verifying how all indicators were tracked and comparing at least one reported value against supporting documentation as of September 30, 2015. These tests identified many weaknesses in supporting documentation and the mission’s assessments of data quality. However, when viewed alongside interviews, project documentation, and site visits, we determined that the data are sufficiently reliable to use in answering the audit objective. We met with staff from 6 of the 13 subgrantees that received assistance through the project. Our selection of subgrantees covered about 67 percent of the amount disbursed for subgrants. Since we based the selection on judgmental samples, our results and conclusions are limited to the items and areas t ested and do not apply to all program activities. Nonetheless, we believe the testing supports the audit’s findings. 10 The contractor used a total of 24 indicators to measure the project’s results, of which 17 were required by USAID/Haiti per the contract. Office of Inspector General, U.S. Agency for International Development 11 APPENDIX B. LIST OF REQUIRED INDICATORS Indicator Title Key Indicator of Success Per December 2013 Contract Underwent USAID/Haiti DQA 1 Number of jobs created in target sectors in designated corridors (as measured by full-time equivalents) X 2 Increase in sales (volume and USD value) at sector level or value chain level X 3 Percentage increase in investment at sector level or value chain level X 4 Percentage increase in productivity X 5 Increase in sales (volume and USD value) among MSMEs in target value chains 6 Number of MSMEs in target value chains with increased sales 7 Percent change in USD value of micro entrepreneurs input purchases X 8 Number of private sector firms that have improved management practices 9 Number of MSMEs registered [with the Haitian Government] 10 Number of MSMEs increasing their investments 11 Number of persons receiving new or better employment as a result of U.S. Government-funded workforce development programs X 12 Percent increase in job placement rate among workforce service providers 13 Percent increase in employer satisfaction survey scores with TVET graduates 14 Number of workforce service providers with improved service delivery 15 Increased organizational capacity assessment tool (OCAT) scores of target organizations 16 Value in USD of activities leveraged/complemented with other projects/donors 17 Number of project activities coordinated with other projects/donors Office of Inspector General, U.S. Agency for International Development 12 APPENDIX C . AGENCY COMMENTS TO: Jon Chasson, Regional Inspector General (RIG)/San Salvador FROM: Jene C. Thomas, Mission Director /s/ DATE: October 10, 2017 REF: JChasson/JThomas Memo dated 08/17/2017 SUBJECT: Mission response to the audit of USAID/Haiti’s Local Enterprise Value Chain Enhancement (RIG draft report I- 521-17-00X-P, dated August 17, 2017) The Mission would like to thank the RIG for the opportunity to provide comments on the subject draft report. The Mission agrees with both recommendations and herein provides plans and timelines for incorporating the guidance. Background As acknowledged in the draft audit report, the LEVE contract got off to a slow start due to unforeseen factors, such as the weaker than expected capacity of a number of micro, small, and medium-sized enterprises (MSMEs) in the targeted value chains that limited their immediate participation in the program. Another factor was the necessity for the program to carry out extensive assessments of the targeted sectors in order to plan the allocation of support to each one. The program then adapted its plan of action to provide additional business development support to strengthen the businesses’ operational and financial capacity in order for them to benefit from assistance. As a result, the pace of activities and performance picked up starting in the second year. Recommendation No. 1: Conduct an assessment and implement a plan to align activities in each value chain to areas with the greatest potential for economic impact in the time remaining. Mission Response Office of Inspector General, U.S. Agency for International Development 13 The Mission agrees with this recommendation to conduct an assessment and implement a plan to align activities in each value chain to areas with the greatest potential for economic impact. While the remaining period of performance of the contract does not allow enough time to conduct and implement the findings of an assessment, under USAID’s oversight, a mid-term evaluation (MTE) of the LEVE program was conducted during the second quarter of FY 2016. The findings of this evaluation revealed that, among the three targeted sectors, garment/textile and agri- business are the key sectors for job creation, while the construction sector offers mainly opportunities for labor market linkages, training, and workforce development(Attachment #1). The assessment also revealed that most of the jobs were created by lead firms rather than by MSMEs. Based on these findings, LEVE adjusted its annual work plan to focus on the garment and agri-business sectors, with an increased effort to work more with the lead firms, since they have the highest potential for job creation (Attachment # 2). Plan of Action LEVE will continue to adjust its program work plan to show the new focus on job creation, taking advantage of opportunities in the sectors that offer the highest potential for growth as recommended by the MTE. Target completion date: The updated work plan will be reviewed and approved by USAID by December 15, 2017. Recommendation No. 2: Revise the monitoring and evaluation plan and implement procedures to more effectively measure and evaluate project success and impact. Mission Response The Mission agrees with this recommendation, and would like to underline the fact that project reporting has improved since the audit work. In particular, Mission Management believes that the monitoring and evaluation plan (M&E), which is a living document adjusted on an annual basis, provides sufficient data to allow for proper monitoring of the project. To this end, the M&E plan has been reviewed and approved during the first quarter of FY 2017(Attachment # 3). Furthermore, a new version is being updated for FY 2018. In November 2016, the Mission conducted Data Quality Assessments (DQAs) for the main indicators of the LEVE project. The DQAs generated recommendations which addressed the weaknesses related to the validity of the data and the lack of supporting documentation. These were shared with the contractor for follow-up actions (Attachment #4). Plan of Action The Mission will continue to make additional improvements to project reporting. In particular, the following actions will be undertaken: (1) The M&E plan will continue to be adjusted on an annual basis to be in line with the annual work plan. Office of Inspector General, U.S. Agency for International Development 14 (2) USAID will conduct new DQAs for all of the indicators of the M&E plan to ensure that the recommendations made in the DQAs reports last year were addressed. (3) Recognizing that baseline data are not available for three of the key performance indicators, the final performance evaluation of the contract will use a methodology which will allow the measurement of project progress and impact in the absence of baseline data. Target completion date: USAID will undertake the following actions, all to be completed by October 30, 2018: (1) Review and approve the M&E plan. (2) Complete a second round of DQAs for all project indicators. (3) Conduct the final performance evaluation of LEVE. Attachments: 1- Local Value Chain Enterprise Value Chain Enhancement (LEVE), Mid- term Evaluation Report, March 2016. 2- FY 2017 Work Plan- Local Value Chain Enterprise Value Chain Enhancement (LEVE) - October 1, 2016 – September 30, 2017. 3- Monitoring and Evaluation Plan, Local Value Chain Enterprise Value Chain 4- Enhancement (LEVE)- October 1, 2016 – December 22, 2018 Local Value Chain Enterprise Value Chain Enhancement (LEVE) DQA Reports, March 2017. Office of Inspector General, U.S. Agency for International Development 15 APPENDIX D . MAJOR CONTRIBUTORS TO THIS REPORT The following made major contributions to this report: Van Nguyen, acting regional inspector general; Jon Chasson, regional inspector general; Brad Moore, audit manager; David Clark, audit manager; Robyn Blount, lead auditor; Guilloux Cayo, auditor; Ming Liu, analyst; Hannah Maloney, auditor; Hugo Solano, auditor; and John Umana, attorney advisor.