Rapport final d'évaluation d'AVANSE

CACAO DRYING TABLES, ACUL DU NORD, HAITI HAITI FEED THE FUTURE NORTH (FTFN)/APPUI A LA VALORISATION DU POTENTIEL AGRICOLE DU NORD POUR LA SECURITE ECONOMIQUE ET ENVIRONNEMENTALE (AVANSE) ACTIVITY FINAL EVALUATION REPORT Haiti Evaluation and Survey Services (ESS) JANUARY 2020 This publication was prepared independently by Social Impact, Inc. at the request of the United States Agency for International Development. USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | ii HAITI FEED THE FUTURE NORTH (FTFN)/ APPUI A LA VALORISATION DU POTENTIEL AGRICOLE DU NORD POUR LA SECURITE ECONOMIQUE ET ENVIRONNEMENTALE (AVANSE) ACTIVITY FINAL EVALUATION REPORT Date: January 2020 Submitted To: USAID/Haiti Evaluation Mechanism Number: AID-521-C-17-00002 Haiti Evaluation and Survey Services for USAID/Haiti This publication was prepared independently by Social Impact, Inc. at the request of the United States Agency for International Development. Contact: Jennifer Mandel, Chief of Party 2300 Clarendon Blvd, Suite 1000 Arlington, VA 22201, USA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iii | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV ABSTRACT The United States Agency for International Development (USAID)/Haiti requested an independent final evaluation of the Feed the Future North (FTFN) activity, also known as Appui à la Valorisation du potentiel Agricole du Nord pour la Sécurité Economique et Environnementale (AVANSE). DAI Global, LLC (DAI) implemented AVANSE, valued at USD 87.8 million, from 2013 to 2019. To increase agricultural incomes, AVANSE promoted approaches and technologies to increase farmers’ capacity and performance, build input systems through agricultural input shops and private agro-businesses, and strengthen relations in three value chains (cacao, rice, banana/plantain). Farmers appreciated AVANSE’s impact on increased income and yields. Many will continue applying these technologies, while some would appreciate ongoing training and support. Most farmers noted water shortage as a barrier to continued use of these approaches and technologies. Based on increased yields, and the existence of underserved and new areas, scaling up potential exists for AVANSE-promoted approaches and technologies. Private enterprises are potential partners for continuing to promote these approaches/technologies. However, access to water and finance are necessary to both continued use and sustainably scaling up AVANSE impact. Storeowners felt that AVANSE capacity building support helped them function more professionally and agricultural input subsidies helped to increase their incomes. They will continue selling inputs and farmers will both buy inputs and produce their own seeds/seedlings. Value chain actors described the market partnerships AVANSE promoted as beneficial. Private enterprises benefited from access to increased quantity and quality products. Farmers benefited from access to secure markets and embedded support services. USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | iv CONTENTS ABSTRACT III ACRONYMS V EXECUTIVE SUMMARY VII INTRODUCTION 1 EVALUATION SCOPE OF WORK 4 EVALUATION METHODOLOGY 5 PHASE III: DATA ANALYSIS AND REPORTING 7 FINDINGS AND CONCLUSIONS 9 RECOMMENDATIONS FOR USAID 30 ANNEX A: EVALUATION STATEMENT OF WORK 32 ANNEX B: LIST OF DOCUMENTS CONSULTED 40 ANNEX C: PROFILES OF THE EVALUATION TEAM MEMBERS 42 ANNEX D. EVALUATION DESIGN MATRIX SUMMARY 46 ANNEX E: LIST OF PERSONS INTERVIEWED 50 ANNEX F: GEOGRAPHIC LOCATION OF INTERVIEWS 52 ANNEX G: DATA COLLECTION PROTOCOLS 53 ANNEX H: MAIN TECHNOLOGIES PROMOTED BY AVANSE 82 ANNEX I: LIST OF MAIN APPROACHES PROMOTED BY AVANSE 83 ANNEX J: MAIN INPUTS AND EQUIPMENT USED BY CROP 85 ANNEX K: NON-DISCLOSURE AGREEMENTS 86 TABLES TABLE 1: RESPONDENT GROUPS AND VALUE CHAINS 7 v | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV ACRONYMS AVANSE Appui à la Valorisation du potentiel Agricole du Nord pour la Sécurité Economique et Environnementale BAC Communal Agricultural Offices ( Bureau Agricole de la Commune ) BAP BIA Best Agricultural Practices Boutiques d’Intrants Agricoles (Input stores) CBO Community Based Organization CLES Collective Against Social Exclusion (Collectif de Lutte contre l’Exclusion Sociale) CNSA Conseil National de Sécurité Alimentaire COMAG Commerce Agricole S.A. COP Chief of Party COR Contracting Officer’s Representative DAI Development Alternatives Inc. DEC Development Experience Clearinghouse DEED Développement Economique pour un Environnement Durable ESS Evaluation and Survey Services ET Evaluation Team EQ Evaluation Question EQUI ® Evaluation Quality, Use, and Impact ® FCR Findings, Conclusions, and Recommendations FECANO Federation of Cacao Cooperatives of the North ( Fédération des Coopératives Cacaoyères du Nord FFS Farmer Field School FGD Focus Group Discussion FTF Feed the Future FTFN Feed the Future North GOH Government of Haiti GUC Grant Under Contracts HAP Hillside Agriculture Program IDB Inter-American Development Bank IFAD International Funds for Agricultural Development IFPRI International Food Policy Research Institute IR Intermediate Result KII Key Informant Interview LOE Level of Effort LOP Life of Project M&E Monitoring and Evaluation MARNDR Ministry of Agriculture, Natural Resources, and Rural Development (Ministère de l'Agriculture des Ressources Naturelles et du Développement Rural) MFI Micro-Finance Institution MSU Michigan State University MT Metric Tons NRM Natural Resources Management OIG Office of Inspector General PE Performance Evaluation USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | vi PIF Intensive Production via Fragmentation ( Production int ensive par fragmentation ) PISA Produit des Iles S.A. PO Producer Organization PTTA Program for Technology Transfer to Small Farmers QA Quality Assurance SI Social Impact, Inc. SIBA SIMA System of Incentives through Purchase Vouchers ( Système d’incit ation via les bons d’acha t) Agriculture Market Information System ( Systeme d'Information sur les Marches Agricoles ) SRA S ystème de Riziculture Am é lioré SRI Intensive Rice - growing System (Sy stème rizicole intensif ) SOW Scope of Work TA Technica l Assis tance U - FE Utilization - Focused Evaluation USAID United States Agency for International Development USD United States Dollar USG United States Government vii | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY The United States Agency for International Development (USAID)/Haiti requested that Social Impact’s (SI) Haiti Evaluation and Survey Services (ESS) project design and conduct an independent final performance evaluation of the Feed the Future North (FTFN) activity, also known in French as Appui à la Valorisation du potentiel Agricole du Nord pour la Sécurité Economique et Environnementale (AVANSE). USAID/Haiti developed the FTFN/AVANSE activity after the January 2010 Haiti earthquake to improve incomes in Haiti’s northern corridor. DAI Global, LLC (DAI) implemented this activity, valued at USD 87.8 million, from 2013 to 2019. EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this evaluation report is to inform USAID’s possible future agriculture programming in Haiti. The primary audience includes USAID/Haiti, DAI Global, and Government of Haiti (GOH) institutions, including the Ministry of Agriculture, Natural Resources, and Rural Development ( Ministère de l'Agriculture des Ressources Naturelles et du Développement Rural [MARNDR]) and the Communal Agricultural Offices ( Bureau Agricole de la Commune [BAC]). This final evaluation seeks to answer the following evaluation questions (EQs): • To what extent and in what ways will AVANSE’s approaches and technologies used in the rice, plantain, and cacao value chains continue to be used at the end of the activity, and why? • To what extent and in what ways has AVANSE strengthened inputs systems in the activity area? • To what extent and in what ways could approaches and technologies promoted by AVANSE be scaled up? • To what extent have AVANSE activities created partnerships that could, over the long term, guarantee farmers’ access to markets and private sector led provision of services? ACTIVITY BACKGROUND AVANSE began on April 1, 2013, and continued through December 31, 2019. 1 In line with GOH and United States Government post-earthquake strategies, AVANSE’s goal was to increase agricultural incomes in the northern corridor. 2 The activity incorporated priorities set under Feed the Future to combat hunger and food insecurity by boosting agricultural productivity and increasing market opportunities for smallholder farmers. The activity’s design specifically targeted the banana/plantain, cacao, and rice value chains. Subsequent to an audit by USAID’s Office of the Inspector General, USAID/Haiti significantly revised the activity’s focus in 2015. As part of these revisions, USAID/Haiti established the following goals and objectives for the activity: to increase agricultural incomes for at least 28,000 rural households, and double the export volume that AVANSE-supported cacao farmers produce. AVANSE expected to achieve these goals through three Intermediate Results (IRs): • IR 1: Agricultural Productivity Increased • IR 2: Watershed Stability Above Selected Plain Improved • IR 3: Agricultural Markets Strengthened 1 AVANSE has requested a cost-free extension to March 31, 2020. It plans to use extension to complete remaining activities interrupted by political unrest that occurred in 2019 in Haiti. 2 The northern corridor consists of the following departments: North and Northeast. USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | viii EVALUATION DESIGN, METHODS, AND LIMITATIONS The evaluation team (ET) conducted this qualitative evaluation across three departments (West, North, and Northeast) from September 2019 to January 2020 in four phases: desk review; fieldwork; data analysis; and, reporting. The first phase generated preliminary findings and highlighted information gaps to inform the evaluation methodology design and subsequent fieldwork. The fieldwork phase objective was to collect primary data to answer the evaluation questions. During the data analysis phase, the ET aggregated interview data for content analysis triangulation and cross-comparison to provide evidence for the findings and subsequent conclusions. The ET conducted fieldwork in Northern Haiti in November 2019. They employed two data collection methods: Key Informant Interviews (KII) and Focus Group Discussions (FGDs). The team completed a total of 80 interviews, of which 51 were KIIs and 29 were FGDs. The sample included 392 respondents, of which 144 were women (37 percent) and 248 (63 percent) were men. The ET conducted all interviews in-person. The ET developed mitigation strategies to overcome potential response, selection, and gender bias risks in data collection and analysis. To avoid potential bias and strengthen cross-cutting analysis of evaluation findings, the team asked similar questions to all informant groups and triangulated responses. FINDINGS AND CONCLUSIONS EQ 1 - TO WHAT EXTENT AND IN WHAT WAYS WILL AVANSE’S APPROACHES AND TECHNOLOGIES USED IN THE RICE, PLANTAIN, AND CACAO VALUE CHAINS CONTINUE TO BE USED AT THE END OF THE ACTIVITY, AND WHY? The ET found that farmers acquired capacity to continue using approaches and technologies AVANSE promoted through: (i) increased technical skills (mainly through Farmer Field Schools); (ii) subsidized inputs use; and, (iii) ease of applying and adapting technologies to their situation. AVANSE promoted a total of 32 technologies to increase cacao, rice, and banana value chain production. These included Système de Riziculture Intensive (SRI), Production Intensive par Fragmentation (PIF), nurseries and demonstration plots to promote Best Agricultural Practices (BAP). The activity also promoted a total of ten approaches through which to disseminate knowledge, including Farmer Field Schools (FFS), Lead Farmers, farmer-to-farmer exchange visits, and increasing input access through a system of declining subsidies. Many farmers will continue to use AVANSE promoted technologies, particularly those relatively easy to use. However, farmers using more complicated rice value chain technologies may require additional support. According to farmers in all three value chains, the primary motivation for continuing AVANSE promoted technology use is observed increases income and yields, as well as cost efficiency. These positive results also had a demonstration effect for non- beneficiaries, some of whom adopted AVANSE techniques without support from the activity. The evaluation found four main obstacles to continued use of AVANSE’s promoted approaches and technologies: drought; political unrest; the quality, quantity and timeliness of input availability; and, the lack of GOH capacity to continue to support activities begun under the activity. ix | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV EQ 2 - TO WHAT EXTENT AND IN WHAT WAYS HAS AVANSE STRENGTHENED INPUTS SYSTEMS IN THE ACTIVITY AREA? A key component of the activity’s approach to increasing input access was building input systems’ actors capacity. The activity used a market-based approach to strengthen both the supply and demand sides of the input value chain. To increase agricultural input access, AVANSE implemented a system of subsidized vouchers that could be redeemed at local agricultural input stores. Farmers used subsidies to purchase inputs such as fertilizer, equipment such as motor pumps, and services such as plowing. Shop owners indicated they will continue to sell inputs. Farmers reported their intent to continue buying them, though in smaller quantities and less frequently in the absence of subsidies. The activity also provided eight agricultural input stores technical and financial assistance to make physical improvements in their shops, and improve their management and accounting. AVANSE worked directly with farmers to increase their capacity to produce and sell inputs through seedling nurseries. Farmers, particularly women, appreciated that these nurseries provided them ongoing access to healthier inputs. AVANSE promoted partnerships among input stores to form a buyers’ network to purchase from wholesalers in bulk, reducing their costs and improving input access. By distributing vouchers for subsidized inputs through local Microfinance Institutions (MFIs), AVANSE also encouraged farmers to build relationships with financial services providers. Building relationships within value chains encouraged private enterprises to provide farmers with embedded support services such as inputs, credit, and technical assistance. According to input shop owners, the voucher system increased the number of clients and improved their business turnover. MFIs also noted an increase in the number of clients visiting their branches due to the voucher program. While farmers may not purchase the same quantities of inputs without subsidies, their increased input use during the activity produced a positive demonstration effect. Farmers stated that lack of access to water was a major constraint to their input use. Farmers of all three crops also mentioned lack of access to finance as a constraint to using inputs. Finally, the GOH’s distribution of free tools, seeds, and fertilizer created potential competition with AVANSE’s market-based approach to supporting input systems. EQ 3 - TO WHAT EXTENT AND IN WHAT WAYS COULD APPROACHES AND TECHNOLOGIES PROMOTED BY AVANSE BE SCALED UP? Most stakeholders believed that farm product demand is greater than supply, and others indicated the existence of underserved and new areas that increase potential for scaling production. Beneficiaries and non-beneficiaries both noted a demonstration effect of improved yields and higher income that encouraged their adoption and use of technologies AVANSE promoted. However, farmers described important constraints a future activity must address to increase their use of AVANSE’s approaches and technologies, primarily access to water and training. In addition, ease of use (and potential for scaling up use) of technologies varies by value chain. Some technologies (particularly the rice value chain) are more complicated and demanding. Farmers expressed the need for additional support to continue their use. They also cited other technologies, such as shade control for cacao trees, as much simpler to use and, thus, easier to scale. Many of the activity promoted approaches and technologies were popular with farmers, namely pruning, shade control, seedling production, PIF, and SRI. Some private enterprises are offering embedded support services to farmers using these same technologies. While it is likely that they USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | x will continue to do so without any outside support, private enterprises are focused on farmers with whom they already have buying relationships, and are not positioned to scale up these services on a sector wide basis. Farmers described the primary obstacles to scale as the lack of access to water, lack of labor for weeding and mechanized plowing services, and lack of access to finance. Farmers, GOH officials, and private enterprises all mentioned the ongoing socio-political unrest as a barrier to scaling up their activities. According to interviews with multiple respondents, although the GOH would like to support scale-up of AVANSE-promoted approaches and technologies, it lacks the resources. Farmer of all three crops described the positive demonstration effects of applying AVANSE- supported approaches and technologies, as well as unmet demand for agricultural products. These conditions establish a foundation for scale-up among both beneficiaries and non- beneficiaries. Farmers indicated several important preconditions that future activities will need to address to build the foundations for growth in the sector, access to water and finance. EQ 4 - TO WHAT EXTENT HAVE AVANSE ACTIVITIES CREATED PARTNERSHIPS THAT COULD, OVER THE LONG TERM, GUARANTEE FARMERS’ ACCESS TO MARKETS AND PRIVATE SECTOR LED PROVISION OF SERVICES? According to interviews with farmers, private enterprises and AVANSE staff, the activity successfully promoted relationships among value chain actors in the cacao and rice value chains. However, AVANSE staff stated that efforts to promote partnerships in the banana value chain were less successful for various internal and external reasons. Through partnerships established with private enterprises, producers said that they benefited from more diversified markets and increased prices, as well as embedded support private businesses provided, such as inputs, equipment, training and technical assistance, credit, and transport to market. Private enterprises stated that they benefited from the partnerships in terms of increased quantity, quality, and consistency of products sold to the business. Both farmers and private enterprises agreed that these relationships helped increase their profits, and that they are interested in continuing them. Some farmers said they formed marketing groups to further increase their bargaining power through bulk sales. Because of these direct relationships, farmers and private enterprises said they were also able to work together to upgrade to higher end production (particularly organic and Fair-Trade cacao). Most actors up and down the value chain intend to continue the relationships that AVANSE promoted. Continuation of these relationships implies that in selected value chains, farmers will have long-term access to markets and embedded support services. RECOMMENDATIONS FOR USAID USAID should continue to support synergistic approaches to value chain strengthening. AVANSE’s successes were due, in large part, to its use of synergistic strategy to market building and value chain strengthening. Future activities should include the following components of this synergistic strategy: capacity building; input system strengthening; reinforcing value chain relations; and, private sector engagement. USAID should continue supporting capacity building activities for farmers. Farmers expressed their appreciation for the activity’s capacity building approaches and main technology packages (SRI, PIF, and BAP). USAID/Haiti should adapt these approaches and xi | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV technologies to new conditions, continue in current intervention communes, and expand to additional communes and farmers. USAID should continue supporting capacity building for private value chain actors to foster private sector engagement. Future activities should use capacity building support to private value chain actors to incentivize expansion to new and underserved areas, and partnering with more farmers, including providing extension services aiming at increasing agricultural production. USAID should support capacity building for GOH offices and CBOs. To ensure the long-term sustainability of agriculture sector building activities, it is critical that USAID provide capacity building for GOH and CBOs, and support the coordination and ownership of government and civil society organizations in activity implementation. USAID should further strengthen the input systems using integrated, market-based approaches. To sustainably increase access to inputs, USAID should further strengthen input systems, focusing on improving market capacity to supply the right quality and quantity of inputs at the right time. This should include strengthening both the capacity and relationships among market actors, prioritizing improving market system efficiency and reaching new markets and farmers. Improving access to water should be a priority of future USAID agricultural programming. It is very likely that drought and flooding will be the new normal in Haiti. Future programming should include irrigation and watershed management components, and work with a range of government, private sector and community actors to increase access to water and improve water resource management. USAID should continue to reinforce value chain relations. Future activities should build relations among value chain actors through intra-value chain partnerships, embedded support services, and improved market access. Upgrading to new products and higher end markets (such as Fair-Trade and organic) will require increased capacity and ongoing engagement of farmers, private agro-enterprises, and investors. Future USAID projects should implement more responsive and flexible approaches. USAID/Haiti’s activity planning should include periodic assessment of the context, conditions, and lessons learned to adaptively manage new barriers and take advantage of new opportunities. In addition, given the instability of the current situation in Haiti, future programming requires flexibility in planning and implementation to overcome new constraints that arise during activity implementation. Access to finance is integral to building sustainable agricultural value chains and attract new investors. To reduce subsidy dependency, promote modern farming, and increase long-term financial autonomy, USAID/Haiti should consider access to finance as an integral component of agricultural sector building. USAID should promote broad market-based approaches, such as loan guarantees, equity investment, public-private partnerships, foreign direct investment, leasing, joint-ventures, intra-value chain financial services, and village savings and loan associations. 1 | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV INTRODUCTION GENERAL OVERVIEW The United States Agency for International Development (USAID)/Haiti requested that Social Impact’s (SI) Haiti Evaluation and Survey Services (ESS) project design and conduct an independent final Performance Evaluation (PE) of the Feed the Future North (FTFN) activity, also known in French as Appui à la Valorisation du potentiel Agricole du Nord pour la Sécurité Economique et Environnementale (AVANSE). DAI Global, LLC (DAI) implemented this activity, valued at USD 87.8 million, in Haiti from 2013 to 2019 (see Annex A for the evaluation Statement of Work). 3 Prior to conducting the fieldwork for this evaluation, the Evaluation Team (ET) conducted a document review to generate initial findings (see Annex B for list of documents the ET consulted, and Annex C for the ET members’ profiles). The ET presented these findings in an inception and design report that USAID/Haiti approve, and based on which they prepared an evaluation design. USAID/Haiti approved the design prior to the fieldwork start (see summary of evaluation design in the Evaluation Design Matrix found in Annex D ). Upon completing the fieldwork, the ET triangulated data from the document review with the primary data, crosschecked results, and applied qualitative analytic methods to develop evidence-based findings. The ET presented the findings from this analysis, along with their conclusions and recommendations, to USAID/Haiti in an out-briefing. This final evaluation report represents the culmination of the ET’s research and analysis process, and incorporates feedback the Mission provided during the out-briefing. ACTIVITY BACKGROUND 3 Throughout the report, “AVANSE” is referred to as “the activity.” ACTIVITY OVERVIEW Name : Feed the Future North (FTFN) Project Program Objective : Increase agricultural incomes for 28,000 households in Northern Corridor and double export volume of cacao produced by supported farmers Period of Performance : April 2013 – December 2019 Total Funding Amount : USD 87.8 million Main Implementing Partner : DAI Global Key Partner : Government of Haiti (namely, Ministry of Agriculture, Natural Resources, and Rural Development) Geographic Presence : North and Northeast departments, Haiti USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | 2 Although less than half of the country's agricultural supplies are produced locally, Haiti’s agricultural sector has very high production potential. 4 In addition, Haiti has potential to upgrade to higher-end crops, such as coffee, cacao, mango, organic rice, and essential oils (vetiver). 5 In 2017, banana, plantain, and rice were among the top ten Haitian agricultural products, with production of 259 million Metric Ton (MT), 243 million MT, and 179 million MT, respectively. 6 For the same year, rice was Haiti’s 11 th most exported product, with USD 6.4 million in sales. However, Haiti’s agriculture sector has faced many challenges, including environmental degradation, limited access to capital, weak public or private agricultural extension services, and poor access to markets. According to the International Fund for Agricultural Development (IFAD), “agricultural productivity is severely constrained by a number of factors. Small farmers generally lack access to appropriate technology and key production factors, especially irrigation water. Post-harvest losses are considerable, often the result of a lack of storage and processing facilities. In addition, the condition of road infrastructure is poor, and small farmers and poor rural households have extremely limited access to credit for productive activities.” 7 These impediments have limited farmers’ potential to increase crop productivity and sales of food and cash crops. 8 As noted in the AVANSE activity description, “following the January 2010 earthquake, the importance of jumpstarting agricultural productivity has been echoed strongly by Government of Haiti (GOH), the United States Government (USG), and throughout the international community.” 9 As a result, USAID/Haiti developed the AVANSE activity to improve farmers’ incomes in Haiti’s northern corridor, one of three regions targeted for concentrated U.S. Government support. 10 The USD 87.8 million AVANSE activity began on April 1, 2013, and continued through December 31, 2019. 11 In line with GOH and USG post-earthquake strategies, the AVANSE activity goal was to increase agricultural incomes in Haiti’s northern corridor. Reflecting lessons learned from similar past initiatives, USAID/Haiti designed AVANSE to work in entire watersheds and their associated plains in the northern region. 12 According to the original activity description, AVANSE aimed to increase agricultural incomes for at least 63,500 rural households, with the goal of doubling agricultural income for 43,500 households. 13 The activity’s design targeted the banana/plantain, beans, cacao, corn, and rice value chains with four Intermediate Results (IRs): IR 1: Agricultural Productivity Increased; IR 2: Watershed Stability above Selected Plains Improved; IR 3: Agricultural Markets Strengthened; and IR 4: Capacity of Local Organizations Strengthened (Cross-Cutting). 4 Center for the Facilitation of Investments, Agricultural Sector Fact Sheet, http://opendata.investhaiti.ht/fxmmuxd/agricultural-sector . 5 Ibid. 6 Most recent data available from http://opendata.investhaiti.ht/fxmmuxd/secteur-agricole . 7 IFAD, Country Assistance Strategy 2019, https://www.ifad.org/en/web/operations/country/id/haiti . 8 USAID/Haiti, AVANSE Scope of Work, September 2013. 9 Ibid. 10 USAID, “Post-Earthquake USG Haiti Strategy - Toward Renewal and Economic Opportunity,” January 3, 2011. 11 AVANSE has requested a cost-free extension to March 31, 2020. It has planned to take advantage of this extension to complete remaining activities paused by political unrest occurred in 2019 in Haiti. 12 USAID/Haiti, AVANSE Scope of Work, September 2013. 13 Ibid. 3 | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV Initially, AVANSE integrated two U.S. Government programs: Feed the Future (FTF), which the Obama Administration launched in 2009 as the President’s Global Hunger and Food Security Initiative following riots related to escalating food prices in several countries, including Haiti. This initiative addresses global hunger and food security challenges around the world. By supporting country-driven approaches, FTF seeks to address the root causes of hunger and poverty and find long-term solutions to under-nutrition and chronic food shortages by helping countries transform their own agricultural sectors to sustainably grow enough food to feed their population. USAID Forward, also known as “Local Solutions,” was the Agency’s ambitious reform initiative to build local organizations’ capacity to directly implement USAID-funded activities. 14 In line with this initiative, USAID initially anticipated that AVANSE would create a pool of local organizations that would meet USAID’s eligibility criteria for direct funding by the activity’s third year. Subsequently, these organizations would become the primary implementers of future USAID agriculture activities in the northern corridor. Originally, activities under IRs 1, 2, and 3 included research and technical assistance partners, such as Michigan State University (MSU), Tufts University, the University of Nebraska, Zamorano University, and the International Food Policy Research Institute (IFPRI). Along with Haitian institutions, these partners sought to identify and test new approaches and technologies in selected watersheds to improve productivity and natural resource management (NRM). The activity also included private sector companies in the marketing of selected value chain products. The primary implementation tools for IR4 and USAID Forward were sub-contracts and grants under contracts (GUCs) to Haitian organizations. Sub-awards served the dual purpose of activity implementation and preparing local firms to become eligible for direct USAID funding, while DAI/AVANSE served as a management unit overseeing sub-awardees. This implementation strategy proved more challenging than anticipated, and elicited insufficient participation from local organizations. USAID’s Office of Inspector General (OIG) performance audit noted this lack of progress in strengthening local organizational capacity, and USAID ultimately dropped this component of the activity. 15 Significant problems related to contract management undercut the first years of activity implementation. As a result, USAID reduced the scope of the initial activity contract in 2015. For example, the activity originally aimed to increase average agricultural income for beneficiary households by 88 percent. USAID reduced this target to 65 percent. 16 USAID dropped bean and corn value chains from the list of target crops, and also dropped most agro-forestry activities, some market strengthening activities, all local organization strengthening activities, and all road and hillside stabilization activities (access to irrigation activities continued). 17 In September of 2015, USAID modified the activity results framework to the following IRs: 18 • IR 1 : Agricultural Productivity Increased o 1.1: Knowledge and Availability of Improved Technologies and Systems Increased o 1.2: Strengthened Extension of Agricultural Technologies and Nutrition Information o 1.3: Access to Inputs Increased  1.4: Irrigation Systems Rehabilitated/Constructed 14 USAID, “USAID FORWARD”, 2017. 15 USAID/Office of the Inspector General, Audit Of USAID/Haiti’s Feed The Future North Project, October 2015. 16 “AVANSE Indicator Performance Tracking Table FY19”, DAI for USAID, July, 2019. 17 USAID/Haiti, AVANSE Modified Scope of Work, September 2015. 18 Ibid. USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | 4 o 1.4.1: Management Capacity of User Associations Increased • IR 2 : Watershed Stability Above Selected Plain Improved o 2.1: Critical Slopes Stabilized Through Farmer-Level Investment • IR 3 : Agricultural Markets Strengthened o 3.1: Improved Access to Storage and Processing Facilities o 3.2: Improved Market Information Systems o 3.3: Relationships in Targeted Value Chains Strengthened. DEVELOPMENT HYPOTHESIS The activity’s current overall approach is based on three development hypotheses: 19 IR 1 Hypothesis: Increased agricultural production and productivity will increase the quantity and diversity of available foods, contributing directly to higher nutrition outcomes. It will also boost incomes through sales and farm jobs, enabling households to increase consumption of nutritious foods and reducing income poverty. IR 2 Hypothesis: Stabilizing hillsides will both protect crops grown on the plains and boost production of mango, cashew, avocado, cacao, and other crops. This will increase incomes and, therefore, enable households to increase consumption of nutritious foods and reduce income poverty. Mango production spillovers to local consumption will also increase nutrition outcomes directly. IR 3 Hypothesis: Strengthening agricultural markets will create additional sales channels for farmers and agribusinesses and create new off-farm jobs along the agricultural value chain. This will increase incomes and, therefore, enable households to increase consumption of nutritious foods and reduce income poverty. EVALUATION SCOPE OF WORK PURPOSE AND AUDIENCE The purpose of this final evaluation of the AVANSE activity is to inform possible future USAID agriculture programming in Haiti. Primary stakeholders include USAID/Haiti, DAI Global, and GOH’s institutions, especially the Ministry of Agriculture, Natural Resources and Rural Development ( Ministère de l’Agriculture, des Ressources Naturelles et du Développement Rural [MARNDR]), including the central office and Communal Agricultural Offices ( Bureau Agricole de la Commune [ BAC ]) located in the north and northeast departments. EVALUATION QUESTIONS This final evaluation sought to answer the following Evaluation Questions (EQs): 20 To what extent and in what ways will AVANSE’s approaches and technologies used in the rice, plantain, and cacao value chains continue to be used at the end of the activity, and why? The evaluation should highlight constraints and facilitators to continued use of new approaches and technologies. 19 DAI, AVANSE Monitoring and Evaluation Plan, January 9, 2016. 20 Comments in italics added by USAID for clarification of EQs 5 | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV To what extent and in what ways has AVANSE strengthened inputs systems in the project area? The evaluation should consider all types of inputs, including, but not limited to, seeds, fertilizers, and pesticides in its analysis. To what extent and in what ways could approaches and technologies promoted by AVANSE be scaled up? Among other possible constraints and facilitators, the evaluation should consider the enabling environment in its analysis. To what extent have AVANSE activities created partnerships that could, over the long term, guarantee farmers’ access to markets and private sector-led provision? The evaluation should consider partnerships and access to market through a sustainability lens. EVALUATION METHODOLOGY GENERAL APPROACH The ET conducted this qualitative evaluation from September 2019 to January 2020 in four phases: desk review; fieldwork; data analysis; and, reporting. PHASE I: DESK REVIEW The first evaluation phase was reviewing the following documents: the AVANSE contract and modifications, quarterly and annual reports, annual work plans, monitoring and evaluation (M&E) plans, and related background materials. To better understand some performance indicators AVANSE reported, the ET consulted the reports and data other key actors published (See Annex B for a full list of sources). The purpose of this first phase was to generate preliminary findings and highlight gaps in information to inform the evaluation methodology design and subsequent fieldwork. PHASE II: FIELD WORK The ET carried out the fieldwork phase of the evaluation from November 6th to 23rd, 2019, with the objective of collecting primary data to fill the gaps identified during the desk review. DATA COLLECTION METHODS The ET employed two data collection methods: Key Informant Interviews (KII) and Focus Group Discussions (FGDs). The ET completed a total of 80 in-person interviews, including 51 KIIs and 29 FGDs. A total of 392 respondents participated, of which 144 were women (37 percent) and 248 (63 percent) were men. The ET determined the total KII and FGD numbers based on time and resource constraints, as well as the prevailing security situation. KEY INFORMANT INTERVIEWS The ET conducted KIIs to better understand AVANSE stakeholder experiences, including successes and challenges, and their intentions to continue interventions initiated through the activity. The ET used these interviews to gather background information on the adoption of approaches and technologies, strengthening of input systems, potential for scale-up and value chain relationships. Over the course of fieldwork, the ET conducted a total of 51 KIIs with a purposively selected sample of 52 individual stakeholders in 18 communes four departments. Of this total, local research firm, SIKSE, conducted eight KIIs, and the ET conducted the remainder. USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | 6 FOCUS GROUP DISCUSSIONS The ET conducted FGDs to obtain beneficiary insights into their experiences with AVANSE activities, particularly their influence on the farming systems, as well as stakeholders’ intentions to continue activities in the future. The ET used this information to fill data gaps identified during the document review. SIKSE conducted 29 FGDs with a total of 336 beneficiary producers, marketing groups, and non- beneficiaries. The ET determined this number to be proportionally stratified by crop based on total beneficiary numbers, as of the latest AVANSE reporting period. The ET further disaggregated FGD participants across intervention sites in the North and Northeast Departments. FGDs included 17 with cacao farmer groups, six with banana farmer groups, and marketing groups, and six with rice farmer groups. The ET conducted FGDs in the two departments of the Northern Haiti (22 in the North Department, seven in Northeast). FGDs covered a total of 18 communes (see Annex E for full list of interviews by location). RESPONDENT SELECTION The ET identified respondents using a purposeful selection technique (see list of persons consulted in Annex F ). The ET coordinated closely with USAID and AVANSE staff on the final respondent selection, based on the likelihood that selected individuals possessed information or experiences relevant to the EQs and data gaps. The ET supplemented purposeful selection with snowball sampling to identify additional key informants and fill gaps in the initial list of key stakeholders. The ET consulted AVANSE staff to identify individuals and beneficiary groups within the list of stakeholders who had a good understanding of the activity, and belonged to selected respondent groups. In addition, the ET modified the respondent list in response to logistical and security considerations. For example, the ET dropped one commune (Borgne) from the list for security reasons, as the road was blocked by protesters. For FGDs with beneficiary producers, the ET selected farmers by primary crop (cacao, rice, bananas) and type of assistance (FFS, Système d’incitation via les bons d’achats [SIBA] vouchers, Lead Farmers) based on total reported beneficiary numbers as of latest AVANSE reporting. FGDs also included beneficiary marketing groups (cacao and rice) and non-beneficiaries (cacao, rice and bananas). The ET interviewed two women’s farmer groups (cacao and banana) to gather gender specific insights. To determine specific participants in each FGD, the ET selected eight to twelve beneficiary households based on primary crop and commune using beneficiary lists AVANSE provided. The ET sought to include both men and women from farmer specific groups. Although more men than women participated in KIIs (63 percent men and 37 percent women), the proportion was a fair reflection of the roles played by men and women in the activity. 21 The ET identified a total of ten respondent groups based on analysis of key activity stakeholders. They interviewed most respondent groups in Northern Haiti, with the remainder interviewed in Port au Prince. All interviews were in-person. For some respondent groups, the ET conducted multiple interviews, particularly with implementing partners and GOH. For example, the ET 21 According to AVANSE reporting from Q3 2019 (DAI, AVANSE IPTT report, November 2019), women made up 24 percent (1,347 women) of farmers applying improved technologies or management practices, and 33 percent (972 women) of farmers who have received agricultural sector productivity or food security training. 7 | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV interviewed 11 AVANSE staff and six GOH officials (see data collection protocols in Annex G ). The following table details respondent groups and value chains of farmers the ET interviewed. TABLE 1: RESPONDENT GROUPS AND VALUE CHAINS RESPONDENT GROUPS RESPONDENTS NUMBER OF RESPONDENTS Beneficiary Producers Cacao, rice and bananas – including women’s groups 25 Implementing Partners AVANSE, AGRIDEV 13 Input Suppliers BIA, plowing service owners 9 Beneficiary Marketing Groups Cacao and rice – including women’s groups 6 GOH Central level, department level, commune level 6 Private Enterprises AgroTech, CLES, UPBH, NOVELLA, PISA, FECANO 6 Non-Beneficiaries Cacao, rice and bananas 5 International Organizations USAID, World Bank, Interamerican Development Bank 4 Financial Institutions Unibank, Le Levier credit union, Fonkoze 4 Civil Society Organizations Université Henry Christophe de Limonade, Université Chretien du Nord 2 Total Interviews 80 VALUE CHAINS DEPARTMENTS NUMBER OF RESPONDENTS Cacao North, Northeast 25 Bananas North, Northeast, Northwest 16 Rice North, Northeast 12 Farmers in Selected Value Chains 53 PHASE III: DATA ANALYSIS AND REPORTING DATA ANALYSIS The ET’s approach to analysis involved data triangulation to crosscheck results, as well as the following qualitative methods, to provide evidence for the evaluation’s findings and subsequent conclusions. DATA ANALYSIS METHODS For qualitative data collected during fieldwork, the ET’s analytic methods included: Content Analysis: Content analysis entailed the ET’s intensive review and systematic coding of KII and FGD notes to identify and highlight key themes and their frequencies. The ET then summarized these standardized data to better understand the outcomes of specific interventions and experiences. Triangulation: Triangulation enabled the ET to cross-verify and cross-validate findings that emerged from the content analysis, and to identify trends among findings to draw conclusions. The ET also utilized methodological triangulation by asking the same or similar questions across KIIs and FGDs. USAID.GOV USAID/HAITI AVANSE FINAL EVALUATION REPORT | 8 Gender Analysis: The ET’s analysis include gender-based findings and conclusions. The team worked across all EQs to capture and compare the responses of women beneficiaries with those of men. Additionally, the ET organized interviews specifically for women beneficiaries and analyzed for effects on both male and female participants to show any major differences. DATA PROCESSES At the outset of fieldwork, the ET staff trained and supported the local research firm, SIKSE, in the use of the data collection tools. Throughout fieldwork, the ET and local research firm maintained daily contact, both in-person and remotely. The ET used these internal debriefs to discuss progress, make any necessary adjustments to the evaluation schedule, and develop probing questions in response to emerging findings. All team members took detailed notes of KIIs and FGDs, and subsequently cleaned and shared electronic summaries continuously throughout fieldwork. The ET conducted frequent spot checks, and began identifying emerging patterns for analysis during coding. To facilitate data analysis, the ET aggregated and analyzed interview notes in a comprehensive, cohesive, and consistent manner using Excel-based tally sheets. 22 The ET developed a tally sheet matrix, which listed key themes that emerged from each KII and FGD. The ET used the tally sheet to examine trends across themes by stakeholder group, value chain, geographic area, gender, etc. This enabled the ET to identify trends within and across respondent groups and geographies. The ET also triangulated primary interview data with secondary data from AVANSE reports and published literature, to identify trends, and gain insights into lessons learned. Following data analysis, the Team Leader transferred findings, conclusions, and recommendations (FCR) in an Excel-based matrix that categorizes analysis by EQ. This matrix: (i) ensured that the ET prepares a systematic and thorough response to each EQ; (ii) verified that the analysis accounts for gender and social dimensions; (iii) identified any gaps where additional clarification or analysis may be necessary; and, (iv) served as the basis for developing the evaluation report. DISSEMINATION AND UTILIZATION The report offers evidence-based recommendations for USAID’s consideration when designing future agriculture interventions in Haiti and other similar countries. In addition, in keeping with SI’s commitment to Utilization-Focused Evaluations (U-FE), ESS presents recommendations in a manner that identifies the issue, action, responsibility, and timeline for implementation. To promote utilization across USAID and the broader development community, ESS will collaborate with USAID to disseminate findings, present and discuss the results to the Mission, and will post the Final Evaluation Report on the Development Experience Clearinghouse (DEC). POTENTIAL BIASES AND MITIGATION STRATEGIES Potential bias associated with the methodology for this final evaluation include: Response Bias: Response bias is the risk that key informants may be motivated to provide responses they consider socially desirable or influential in obtaining donor support. The ET mitigated this bias by posing questions that sought to uncover causal chains, as opposed to asking participants to simply describe outcomes, as well as by probing for both successes and challenges. 22 King, Gary, Robert Keohane, and Sydney Verba, “Designing Social Inquiry: Scientific Inference in Qualitative Research”, Princeton University Press, Princeton University Press, 2016. 9 | USAID/HAITI AVANSE FINAL EVALUATION REPORT USAID.GOV Selection Bias: Selection bias is an inherent risk when implementing partners facilitate contact with beneficiaries and stakeholders. To mitigate the risk of implementers directing the ET toward only the most active, satisfied, or “successful” stakeholders, the ET identified respondents independently from contact lists provided by AVANSE staff. The ET made