RANFOSE Final Performance Evaluation Report

This report is made possible by the support of the American People through the United States Agency for International Development (USAID). The contents of this report are the sole responsibility of International Business Initiatives, Corp. (IBI) and Social Impact, Inc. (SI) and do not necessarily reflect the views of USAID or the United States Government. PHOTO CREDIT: DAVID ROCHKIND, USAID RANFOSE FINAL PERFORMANCE EVALUATION REPORT HAITI EVALUATION AND SURVEY SERVICES PLUS (ESS+) December 2023 Task Order #72052122F00001 Under EVAL-ME II IDIQ Contract #7200AA20D00026 RANFOSE FINAL PERFORMANCE EVALUATION REPORT Haiti Evaluation and Survey Services Plus (ESS+) December 2023 Submitted to: USAID/Haiti Task Order #72052122F00001 Under EVAL-ME II IDIQ Contract #7200AA20D00026 USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | i TABLE OF CONTENTS Table of Contents ..................................................................................................................... ii Acronyms ................................................................................................................................. iii Executive Summary ................................................................................................................ iv Background ................................................................................................................................................... iv Evaluation Purpose and Evaluation Questions ........................................................................................ iv Evaluation Design, Methods, and Limitations .......................................................................................... iv Findings and Conclusions ............................................................................................................................ v Recommendations ...................................................................................................................................... vii Introduction .............................................................................................................................. 1 Evaluation Purpose and Questions ......................................................................................... 2 Evaluation Purpose ...................................................................................................................................... 2 Evaluation Questions .................................................................................................................................. 2 Evaluation Methods and Limitations ...................................................................................... 4 Findings and Conclusions......................................................................................................... 6 EQ 1. Considering the activity’s theory of change, to what extent has the project been effective in improving the availability of adequately fortified foods that are priorities for improving the health of women of reproductive age and children under five? .......................................................... 6 EQ 2. To what extent has the activity’s stakeholder engagement and involvement been effective in accomplishing the rollout of large-scale food fortification (LSFF)? ............................................... 12 EQ 3. To what extent has the activity’s implementation strategy been effective in strengthening the fortified foods supply chain? .............................................................................................................. 16 EQ 4. What has the project done to support or inhibit the sustainability of food fortification? 18 EQ 5. How has RANFOSE evolved and adapted to the political, security, and economic challenges they encountered? .................................................................................................................. 21 Recommendations ................................................................................................................. 22 Continue LSFF Support in Haiti .............................................................................................................. 22 Maintain the Food Fortification Working Group (FFWG) ................................................................. 22 Advocacy at the Highest Political Level ................................................................................................. 22 Premix Availability and Affordability of Fortification ........................................................................... 22 Strengthen Laboratory Facilities ............................................................................................................. 23 Couple Inspections for Enforcements .................................................................................................... 23 Social Audits: An Alternative for Monitoring Compliance of Fortified Foods ................................ 24 Awareness-Raising as the Key to Consumption .................................................................................. 24 Annexes ................................................................................................................................... 25 Annex I: Evaluation Timeline ................................................................................................................... 25 Annex II: Evaluation Scope of Work ...................................................................................................... 26 Annex III: Evaluation Design & Inception Report ................................................................................. 40 Annex IV: Data Collection Tools ............................................................................................................ 68 Annex V: Sources of Information ......................................................................................................... 119 Annex VI: Findings, Conclusions, Recommendations (FCR) Matrix ............................................... 122 Annex VII: Disclosure of Conflicts of Interest.................................................................................... 128 Annex VIII: Evaluation Team Members ................................................................................................ 134 Annex IX: RANFOSE Product mapping outputs ................................................................................ 136 Annex X: Summary of production data (RANFOSE partners and non-partners) ........................ 139 Annex X1: Technical Specifications Published .................................................................................... 140 Annex XII. Assessment on the laboratory capabilities to analyze fortified foods. ....................... 142 USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | ii LIST OF TABLES Table 1: Planned vs Realized KIIs, FGDs, and RIs ........................................................................................... 5 ACRONYMS BHN Haitian Standards Bureau COA Certificate of Analysis CSO Civil Society Organization CU5 Children Under Five DCQPC Direction for Quality Control and Consumer Protection DEC Development Experience Clearinghouse EQ Evaluation Question ET Evaluation Team FAO United Nations Food and Agricultural Organization FFWG Food Fortification Working Group FGD Focus Group Discussion GAIN Global Alliance for Improved Nutrition GoH Government of Haiti HP Health Practitioner IO International Organization KII Key Informant Interview LSFF Large-Scale Food Fortification MCI Ministry of Commerce and Industry MoA Ministry of Agriculture MoH Ministry of Health MT Metric Ton NGO Non-Governmental Organization PaP Port-au-Prince PNCS Programme National de Cantines Scolaires QA Quality Assurance QC Quality Control RANFOSE Ranfose Abitid Nitrisyon Pou Fè Ogmante Santé RI Rapid Interview SI Social Impact SOW Scope of Work ToT Training of Trainers UNICEF United Nations Children’s Fund USAID United States Agency for International Development USDA United States Department of Agriculture WFP World Food Programme WRA Women of Reproductive Age WTO World Trade Organization Yr Year USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | iii EXECUTIVE SUMMARY BACKGROUND In 2017, the United States Agency for International Development (USAID) launched Ranfose Abitid Nitisyon Pou Fè Ogmante Santé (RANFOSE)—a food fortification project—to reduce some micronutrient deficiencies in Haiti. In collaboration with the Ministry of Health (MoH), Partner of the Americas implemented the project and worked with importers, distributors, and local food producers to fortify wheat flour with iron, folic acid, and other vitamins, as well as vegetable oils with vitamin A and salt with iodine. In 2021, the project received an initial two-year extension until June 2023 and a subsequent extension until January 2024 to allow planned activities to be implemented with the remaining funds. EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of the evaluation is to assess progress to date and provide actionable recommendations to inform activity improvements or new approaches and opportunities. USAID, the primary user of this evaluation, may use the results to adjust and adapt its strategy in the Nutrition and Food Fortification program in Haiti. Secondary users include Implementing Partners, the Government of Haiti (GoH), local stakeholders from the private sector, civil society organizations, universities, and international organizations (IOs) in the country. This evaluation responds to the Evaluation Questions (EQ) drafted by USAID in the Scope of Work (SOW) (Annex II), namely: ● EQ1. Considering the activity’s theory of change, to what extent has the project been effective in improving the availability of adequately fortified foods that are priorities for improving the health of women of reproductive age (WRA) and children under five (CU5)? ● EQ2. To what extent has the activity’s stakeholder engagement and involvement been effective in accomplishing the rollout of large-scale food fortification (LSFF)? ● EQ3. To what extent has the activity’s implementation strategy been effective in strengthening the fortified foods supply chain? ● EQ4. What has the project done to support or inhibit the sustainability of food fortification? ● EQ5. How has RANFOSE evolved and adapted to the political, security, and economic challenges it encountered? EVALUATION DESIGN, METHODS, AND LIMITATIONS Following a desk review of available literature, the evaluation used several qualitative methods to capture different perspectives: key informant interviews (KIIs) with stakeholders, participation in the Food Fortification Working Group (FFWG), focus group discussions (FGDs) or individual interviews with health practitioners (HPs), and rapid interviews (RIs) with shopkeepers, customers, and WRA. Some data collected by the evaluation were triangulated against secondary quantitative data from project monitoring data or other existing sources. Data collection lasted five weeks (May 30, 2023, to June 30, 2023) instead of three as planned. FGDs and RIs were carried out in three key departments that RANFOSE facilitated training of trainers (ToT) with HPs, i.e., Port-au-Prince (PaP) and Petion-Ville in the Ouest department, Cap-Haitian in the Nord department, and Jeremie in the Grande Anse department. All KIIs were conducted remotely, reaching 55 percent of planned KIIs, therefore limiting the number of perspectives obtained. Only reaching this percentage of planned KIIs possibly created some bias in the results if those that refused or were unavailable did so because they held a less favorable view of the project or had nothing to say about the project. It should be noted that a separate USAID market analysis of fortified foods comprised conducting interviews with the same USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | iv stakeholders at the same time as the evaluation; this may have led to respondent fatigue and may explain the low response rate among KIIs. Three FGDs were held instead of five, and six individual interviews were conducted in lieu of an FGD, meaning 78 percent of planned HP were reached. Again, but to a lesser extent, only reaching this percentage of planned HP limited the range of perspectives obtained, especially in Grand Anse Department, where no HPs were reached. The response rate for Ris was 87 percent. The evaluation faced several challenges, including communication problems, the unavailability of key informants, an earthquake in Jeremie, and floods across the country, which delayed data collection. A thematic analysis of the data was carried out using Dedoose software. While RIs provided a snapshot of population awareness of fortified foods, it did not allow any statistically significant inferences to be made about the population, which would have required a population-based survey applied to a representative sample. FINDINGS AND CONCLUSIONS Below, the evaluation presents high-level findings and associated conclusions for each EQ. The details of the evaluation findings can be found in the body of the report under the findings section. It is important to contextualize these findings as food fortification is not yet mandatory in Haiti. While the Food Fortification Law was passed in 2017, no application texts (regulations)—which form the basis for the enforcement and monitoring of food fortification—have been approved yet. EQ1. Availability. While it is beyond the scope of this evaluation to measure the market share of fortified foods or the adequacy of fortification levels, which are best measured through market analysis, this evaluation found that RANFOSE has increased the availability of fortified wheat flour and oil and contributed to improving the quality of nationally iodized salt and fortified wheat flour on the Haitian market. According to KIIs with key actors in fortification in Haiti, this increase has been influenced by the project’s engagement with, as well as raising awareness to and providing technical support to the private sector, together with the latter’s commitment to fortify or improve their fortification processes, if they were fortifying already. However, the evaluation found that fortified foods are not consistently available in all shops and markets, remote areas, or across all departments, which is often where populations need them the most. It is not clear in what way the project has strengthened the availability of fortified foods for women of reproductive ages and children under five specifically. Awareness-raising and training events targeted different stakeholders and population groups through a variety of mass media campaigns, including WRA and CU5, through HPs and other non- governmental organizations (NGOs) working specifically with these target groups. According to RIs conducted, awareness exists of fortified foods and their benefits. However, RIs provided only a snapshot of the level of awareness in specific areas, whereas a national representative survey would more effectively measure population awareness and consumption behaviors, therefore allowing better targeting of awareness-raising activities (format, locations, and population groups). According to FGDs and RIs, the added cost of fortified foods is a barrier to consumption, in addition to limited availability, mainly due to distribution problems and the internal situation in Haiti. EQ2. Stakeholder Engagement. Both private and public sectors have contributed to advancing large-scale food fortification through their engagement with the project. The MoH, the Ministry of Agriculture (MoA) and the Ministry of Commerce and Industry (MCI) were all listed in the Food Fortification Law as key players in its implementation. All have actively advocated for the approval of the application texts. The MoH collaborated with RANFOSE on the advocacy and behavior change strategy; the MCI, through the Haitian Standards Bureau (BHN), was key to reviewing the USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | v fortification standards with the project; and the national laboratory under the MoA was involved from the start but has been out of operation since 2021. The project further engaged stakeholders from industries, civil society organizations, universities, and international and non-governmental organizations. It carried out advocacy and capacity-building initiatives with the public and private sectors, as well as civil society actors. These activities have maintained motivation levels and strengthened collaborations within and between the private and public sectors to improve the legal environment. According to most stakeholders interviewed, the lack of approved regulations is a major barrier to private sector engagement in fortification, which subsequently limits the effective rollout of LSFF. RANFOSE has influenced the legislative environment in the country by collaboratively reviewing and supporting the approval standards provided by WFP for fortified foods, which are voluntary, as well as the development of the application texts and continued advocacy for their approval to make fortification mandatory. The three key ministries identified by the Food Fortification Law (MoH, MCI and MoA) have independently and collaboratively advocated for its approvals, as has USAID. The lack of approval to date is out of RANFOSE’s control, but advocacy must persist if the application texts are to be approved and guarantee mandatory food fortification. Only then will the country have the legal framework to implement monitoring and enforcement. In the absence of approved regulations for fortification, which form the basis for enforcement, alternative solutions have not been found; there is no enforcement of food fortification to date despite the Food Fortification Law being approved in 2017. Since the MoH has been doing food safety inspections with salt producers, the food safety inspections could be extended to food fortification inspections, thereby strengthening enforcement. The inaccessibility of the national laboratory, in addition to the lack of regulations, hinders the GoH’s ability to monitor fortified foods. The equipment donated by the project to the laboratory could not be installed and remains unutilized. Industries received testing kits from RANFOSE and are reportedly using these to monitor fortification levels. Until the national laboratory is operational again, other laboratories could have been strengthened to conduct sample testing to monitor quality control (QC) of fortified foods. EQ3. Supply Chains. Industries identified several factors that negatively impact their production and the supply chain, such as a) the unfair competition resulting from those factories not fortifying, who are able to make higher profits than those fortifying given the added cost of fortification; b) smuggled products flooding the market; c) the exorbitant customs tariffs which apply to premix and fortification equipment; and d) poor quality and limited raw material availability. Those who are fortifying do so voluntarily because they want to contribute to the population’s health and because they can sustain the added cost of fortification. All project partner importers and producers reported implementing Quality Assurance (QA) and QC procedures to control the production or importation of fortified foods, thanks to RANFOSE’s training and technical assistance. Incentives for industries are needed to ensure that the cost of fortification does not render fortified products too costly to produce and, therefore, unaffordable for the population, especially for vulnerable populations in hard-to-reach areas. EQ4. Sustainability. The sustainability of LSFF in Haiti cannot be guaranteed until the application texts are approved, resulting in mandatory fortification and enforcement. Some strong foundations for the implementation of LSFF have been laid by RANFOSE through the creation of multi- stakeholder networks, which are said to be beneficial to all stakeholders involved in the FFWG. However, LSFF takes time and requires persistence, especially in a challenging socio-political context USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | vi such as Haiti. Many key informants report that financial and technical support from IOs to the GoH to consolidate LSFF should continue and thereby build on the foundations that have been laid. EQ5. RANFOSE Adaptability. The operating environment in Haiti entails a great number of significant challenges: the deteriorating national infrastructure, widespread and severe insecurity, national hyperinflation, and the COVID-19 pandemic are major factors that have limited the ability of RANFOSE to deliver a full rollout of LSFF. To overcome the challenges, RANFOSE developed and adopted alternative strategies to continue advocacy, awareness raising, stakeholder engagement, and capacity development. RECOMMENDATIONS Key Priority. USAID should continue supporting LSFF in Haiti to capitalize on the achievements made by RANFOSE and to contribute to improving the nutritional intake of a population that greatly needs it. Food Fortification Working Group. The FFWG should be structured through a steering committee where sensitive matters can be discussed and solutions can be proposed. Include an advisory committee. Advocacy. The FFWG should advocate at the Inter-Ministerial Human Rights Committee, which is presided over by the Prime Minister, for the approval of the application texts (regulations), application of enforcement by GOH, and any other important matters related to fortified food that arise in the future. Premix and Affordability of Fortification. The FFWG should advocate to the GoH to implement measures to facilitate imports of premix, such as tax exemptions for purchases of premix, equipment (feeders and reagents), and supplies for i-checks used in QA/QC, and to provide technical assistance to develop the technical criteria needed to implement these measures. Strengthen Industry Production Process. USAID should strengthen the overall capacities of factories in food safety and food fortification so that they can comply with potential client requirements, such as organizations distributing fortified foods through government or other food aid programs. Food Fortification Cost Analysis. USAID should carry out a cost analysis of fortification in Haiti and use that information as an advocacy tool for the food industry and GoH, showing that food fortification’s impact on price is not as high as previously thought and that the industry can still have revenue. Updating the Application Texts. Based on the standards approved by BHN, the FFWG should review the application texts to ensure that there are no contradictions between them and address any issues identified before they are approved. E.g.: consider the addition of vitamin B12 to wheat flour. QC/QA Activities. While it is still ongoing, RANFOSE should analyze the data from industry reports and present a report periodically to the FFWG to discuss results and implement actions to improve the process. Consider praising the actions of those factories complying with the Food Fortification Law. Laboratory Facilities. USAID should map laboratories around the country and assess their capacity to find an alternative laboratory to provide testing services Enforcement. Food fortification inspections should be coupled with other inspections carried out by the MoH, MCI, and MoA to optimize available resources. While there is improvement in USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | vii laboratory capabilities, inspections should focus on verifying the QA/QC practices of the factories, with an equivalent approach for imported foods. “Social Audits.” While enforcement is being established in the country, USAID could support consumer associations or consumer organizations to carry out “social audits”: purchasing fortified food samples at retail stores around the country and sending them to a laboratory to determine compliance. Awareness-Raising Activities. USAID should continue raising awareness of fortified foods, especially in remote and hard-to-reach areas where the population is most vulnerable to malnutrition. Awareness raising must be coupled with the availability of fortified foods. USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | viii INTRODUCTION In 2017, the United States Agency for International Development (USAID) launched Ranfose Abitid Nitrisyon Pou Fè Ogmante Santé (RANFOSE)—a food fortification project—to reduce some micronutrient deficiencies in Haiti. In collaboration with the Ministry of Health (MoH), Partner of the Americas implemented the project and worked with importers, distributors, and local food producers to fortify wheat flour with iron, folic acid and other vitamins, as well as vegetable oils with vitamin A and salt with iodine. RANFOSE also supported the implementation of food fortification legislation by providing technical assistance to the Government of Haiti (GoH) to create a legislative and regulatory environment supporting the production and importation of fortified foods. In 2021, the project received a two- year extension to January 2024. The project aimed to contribute to long-term health and economic benefits for the Haitian people, including the decreased prevalence of micronutrient deficiencies and related micronutrient deficiency disorders (e.g., anemia), increased nutritional resilience against future food security shocks, decreased morbidity and mortality via improved disease immunity, and increased economic productivity and growth. In addition, the project was to ensure that strong channels of communication and participation among the public sector, private sector, and civil society remain open in support of the nutrition strategy and its operationalization. Lastly, RANFOSE aimed to strengthen the national strategy for fortification and improve quality control (QC) for locally sourced or imported fortified food products. RANFOSE aligned with both a) the Plan Stratégique de Nutrition 2013-2018 of the GoH’s MoH, which focuses on the prevention of malnutrition, and b) USAID’s Multi-Sectoral Nutrition Strategy (2014– 2025), which seeks to decrease chronic malnutrition through the increased availability of and access to high-quality nutrition-specific services and commodities, increased political will and resources for nutrition programs, and strengthened engagement with the private sector to improve nutrition. USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | 1 EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE The main purpose of this evaluation is to assess progress to date and provide actionable recommendations to inform activity improvements or new approaches and opportunities. As the primary user of this evaluation, USAID may use the results to adjust and adapt its implementation strategy in Nutrition and Food Fortification in Haiti. Secondary users include the Implementing Partners, GoH, MoH, and local stakeholders such as producers, importers, sellers in markets, consumers, as well as local universities, and major international organizations (IOs), such as UNICEF (United Nations Children’s Fund), Food and Agriculture Organization (FAO), World Food Programme (WFP), and Iodine Global Network. EVALUATION QUESTIONS This performance evaluation aims to answer the evaluation questions (EQs) drafted by USAID (see Annex II). For each EQ, USAID has requested that the Evaluation Team (ET) consider the project performance in relation to specific aspects. These are as follows: EQ1. Considering the activity’s theory of change, to what extent has the project been effective in improving the availability of adequately fortified foods that are priorities for improving the health of women of reproductive age (WRA) and children under five (CU5)? a. Identify specific areas in which RANFOSE’s input has improved the availability of fortified foods, particularly for WRA and CU5. b. Consider key factors that facilitate or inhibit the availability of fortified foods, particularly among WRA and CU5. c. Highlight the relevancy and effect of the advocacy campaigns with consumers and the factors affecting consumers’ awareness and consumption of fortified foods. EQ2. To what extent has the activity’s stakeholder engagement and involvement been effective in accomplishing the rollout of large-scale food fortification? a. Identify where RANFOSE’s engagement with leadership in different areas (e.g., private sector; GoH) has proven to be instrumental in improving availability. b. Assess the project’s accomplishments and influence on the local Food Fortification Alliance, the national standards, and legislative environment about fortified foods. For instance, consider the performance of the GoH in the areas of: 1) regulation; 2) enforcement (at factories, importation sites, and markets); and 3) the monitoring of fortified samples. c. Determine what—if anything—were the limiting factors or what RANFOSE could have done better to support the development and implementation of the national standards. EQ3. To what extent has the activity’s implementation strategy been effective in strengthening the fortified foods supply chain? a. Consider what factors and activities affected food fortification production and value/supply chain stakeholders (production centers, importers, distributors, markets and retailers, regulatory agencies, etc.). b. Capture business performance and practices as well as price and perceived affordability of fortified wheat flour and edible oil/ghee, how the project influences the premix and micro feeder market, and the technical challenges and lessons learned in strengthening the fortification compliance of the current food fortification programs (wheat flour, oil, and salt—including use of iodized salt in bouillon cubes). 2 | RANFOSE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV EQ4. What has the project done to support or inhibit the sustainability of food fortification? a. Identify nutrition-related barriers and opportunities for sustainability. b. Consider synergy with other USAID projects, factors affecting public and private sector engagement, the regulatory and socio-political environment, and other factors that are likely to support or inhibit the sustainability of food fortification. Consider if there were opportunities and stakeholders that RANFOSE had hoped to use that did not materialize, and document why they were not possible. c. Explore emerging opportunities and alternative options for monitoring compliance of food fortification beyond reliance on the GoH. EQ5. How has RANFOSE evolved and adapted to the political, security, and economic challenges it encountered? a. Explore how the environment has affected the project’s ability to deliver from when it was awarded to what it is doing now. Document the lessons learned to avoid reinventing the wheel in the future. USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | 3 EVALUATION METHODS AND LIMITATIONS The evaluation adopts Social Impact’s (SI) Evaluation, Quality, Understanding and Impact (EQUI) approach, ensuring quality checks are applied at each stage of the evaluation process. Based on the nature of the evaluation objectives and following the desk review, the ET chose a qualitative approach to data collection, to be triangulated against secondary data from project monitoring data or other existing sources, where possible. Please refer to Annex I for the evaluation timeline, Annex III for the Evaluation Design and Inception Report—including full details of the evaluation design and ethical clearances obtained—and Annex IV for the full list of data collection tools used. Methods. The ET used various methods to capture different perspectives: Key Informant Interviews (KIIs) with stakeholders, Food Group Discussions (FGDs) or individual interviews when FGDs were not possible with Health Practitioners (HPs), Rapid Interviews (RIs) with shop owners and managers, customers and WRA, as well as an observation of and participation in a Food Fortification Working Group (FFWG) meeting—instead of holding a planned FGD with FFWG members. The sampling was purposeful, with locations selected based on the departments in which the project facilitated the training of trainers (ToT) to HPs to disseminate the benefits of fortified foods among their colleagues and the wider population. The three selected departments were the Ouest (Port-au- Prince [PaP] and Petion-Ville), Nord (Cap-Haitian) and Grande Anse (Jeremie) departments. Data Collection. , an ESS+ subcontractor, and the ET collectively carried out data collection. organized a five-day training for enumerators and the supervisor (May 24 to 29, 2023), including a day of piloting, and carried out field visits for RIs and FGDs, which started immediately after the training and continued for two weeks. then began outreach and scheduling for KIIs. Data collection ended on June 30, 2023, having lasted five weeks instead of three as planned. teams conducted FGDs and individual interviews and RIs in person in the three departments and carried out both remote and face-to-face KIIs. In Grand Anse, the few HPs that were reachable (six) were not available to attend an FGD; therefore, remote individual interviews were carried out with those who were available. All FGDs and RIs were carried out face-to-face either in health centers (FGD and RIs with WRA) or in shops (RIs with shopkeepers and customers), except one FGD in the Ouest, which took place in a private space as the sampled health center did not authorize the evaluation to conduct activities on the premises. In the Nord, HPs were either not reachable or unwilling to participate, meaning no HP was interviewed in the Nord. The ET carried out remote KIIs, with the assistance of two note-takers from ESS+ and performed data quality checks on the data collected by . The ET checked both the original transcripts (French or Kreyòl) and the English translated versions provided by , in addition to translating their own KII transcripts. Data Analysis. A thematic analysis of the data was done in Dedoose software, using a hybrid approach of both deductive and inductive methods for coding the data. Before analysis, a codebook was prepared based on the EQs. During analysis, new codes were generated as required. The coded data was extracted from Dedoose and analyzed to allow themes and sub-themes pertinent to each EQ to emerge. Interactions between themes and sub-themes were analyzed to produce findings. Several analytical working sessions were held by the ET to triangulate the findings, draw conclusions, and produce recommendations. Data was disaggregated by department and respondent type wherever it was helpful to do so. Limitations. The political insecurity in Haiti limited the team’s mobility and geographic reach during data collection. Moreover, during the initial weeks of data collection, an earthquake in Jeremie and severe floods in Haiti (start of June) delayed teams from collecting RI and FGD data, given health 4 | RANFOSE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV centers were overwhelmed with casualties. identified an alternative health center to reach WRA in Jeremie. However, no alternative was found in Petion-Ville (Ouest) to replace the sampled health center that refused to participate due to a lack of awareness of RANFOSE's activities. Table 1: Planned vs Realized KIIS, FGDs, and Rls Method Number of Planned Number of Realized Percentage Percentage of Realized Number of Missing Interviews Interviews Interviews Interviews Kils 40 22 55% 18 Industries-RANFOSE partners 7 6 86% industries-not RANFOSE 4 | 25% 3 partners GoH 7 5 71% 2 Int. Organizations | || ས 7 64% 4 CSOs 3 3 100% 0 FGDs 6 4 67% 2 HP 50 33 78% || Individual int. 0 6 78% || FFWG I | 100% 0 Rls 90 78 87% 12 Customers 30 30 100% 0 Shopkeepers 10 8 80% 2 WRA with CU5 50 40 80% 10 The evaluation also experienced challenges with reaching key informants, including unresponsive emails, incorrect contact details, and refusals to participate (often because they did not feel knowledgeable enough about the project). Key informants' limited availability created further delays. To address these challenges, the ET sought alternative communication channels or alternative stakeholders and explored strategies to engage informants despite their availability constraints. Because of these challenges, only 55 percent of the sampled key informants were interviewed. See Table I for planned versus realized data collection activities. While Rls provided a snapshot of the population's awareness and consumption of fortified foods, it did not allow any statistically significant inferences to be made at the population level. Furthermore, the evaluation could not capture all perspectives, experiences, and insights related to food fortification, which could lead to missed opportunities for further recommendations. USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | 5 FINDINGS AND CONCLUSIONS EQ 1. CONSIDERING THE ACTIVITY’S THEORY OF CHANGE, TO WHAT EXTENT HAS THE PROJECT BEEN EFFECTIVE IN IMPROVING THE AVAILABILITY OF ADEQUATELY FORTIFIED FOODS THAT ARE PRIORITIES FOR IMPROVING THE HEALTH OF WOMEN OF REPRODUCTIVE AGE AND CHILDREN UNDER FIVE? PROJECT EFFECTIVENESS IN IMPROVING THE AVAILABILITY OF FORTIFIED FOODS The evaluation finds that RANFOSE has been effective in improving the availability of fortified foods in Haiti. One of the first initiatives undertaken by the project was the mapping of fortified products and associated stakeholders in 2018 and again in 2022; this initiative enabled the identification of a) industries already engaged in food fortification; b) the fortified products available on the market; and c) potential industries to approach to join food fortification efforts (see Annex IX for the RANFOSE mapping outputs). The project advocated the importance of food fortification to these industries and obtained the commitment of some industries that were already fortifying to strengthen their fortification practices (wheat flour [two] and salt [one]) as well as recruiting new industries and importers to begin fortifying (e.g., oil [two], importer [one] and wheat flour [one]). Later in 2018, the project supported partner industries with site visits and observations of the factories’ fortification capacities to provide technical recommendations for reaching adequate fortification levels. While providing this technical support to industries, the project collaboratively reviewed the fortification standards (see Annex XI for the final version of the technical specifications) with the Haitian Standards Bureau (BHN) and industries to ensure the adequacy of fortification. Producers also received rapid test kits to verify the adequacy of vitamin A levels in oil and iron levels in wheat flour. AVAILABILITY OF FORTIFIED WHEAT FLOUR, OIL, AND SALT It is beyond the scope of this evaluation to assess the availability of fortified foods on the market. However, according to the data collected by this evaluation, by 2020, the project had partnered with three wheat flour mills, three oil bottlers, and one salt producer to fortify to recommended levels. Wheat Flour One of the three wheat mill partners unfortunately ceased operations following a violent attack in October 2022, consequently reducing the availability of fortified wheat flour on the market. According to secondary data, there is a wheat mill that recently began fortifying and produces approximately 30,000 metric tons per year (MT/Yr). However, the evaluation has no information on the specifications used. Collectively, the two remaining wheat mills that partnered with RANFOSE produce 95,000 MT/Yr of fortified wheat flour (equivalent to 129,960 MT/Yr of wheat).1 The quantity of fortified wheat flour produced by these two factories alone represents nearly one-third of the total wheat flour and wheat products obtained from imported wheat grain into Haiti (405,000 MT/Yr).2,3 Wheat flour and wheat products (including pasta and other wheat products) are imported from the Dominican Republic and other countries (Turkey, Peru, Egypt, Argentina, European Union).4 It is 1 Considering wheat grain imports were around 210,000MT, the flour yield from wheat grain being about 70-75%, therefore, that amount of wheat would yield 152,250 MT of wheat flour. 2 USDA & GAIN “Grain And Feed Annual”, Haiti 2022, https://www.fas.usda.gov/data/haiti-grain-and-feed-annual-4 3 Food and Agriculture Organization (FAO). Food Balance Sheets (2010- ). https://www.fao.org/faostat/en/#data/FBS/report 4 USDA & GAIN “Grain And Feed Annual”, Haiti 2022, https://www.fas.usda.gov/data/haiti-grain-and-feed-annual-4 6 | RANFOSE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV estimated that illegal imports of wheat flour and wheat products comprise between 100,000 to 120,000 MT/Yr.5 Oil Haiti imports 100 percent of the vegetable oil on the market. Palm is the most widely consumed oil (about 80 percent) followed by soybean oil (13 percent). 6 It is mainly imported from Indonesia and Malaysia, with some from the Dominican Republic.7 There was no locally fortified palm oil before RANFOSE began; the project approached and obtained the engagement of two oil bottlers to start fortifying the oil they bottle and one importer to import fortified oil. It is not clear what share of the market is held by illegally imported oil, which reportedly remains the biggest competitor to fortified oil. Thanks to the increased availability of fortified oil on the Haitian market, WFP now reports being able to source fortified oil and wheat flour locally for their school feeding programs, which they couldn’t before RANFOSE. Salt The only salt producer to iodize salt in the country has a market share of 3 to 5 percent, produces 2,500 MT/Yr of salt, and supplies both industries and the retail market. RANFOSE worked with this producer to improve the iodization process, thereby improving the adequacy of fortified salt for the Haitian market. Despite this low reported market share, according to the population and HPs interviewed, salt is the most readily available fortified food, possibly resulting from the technical support brought by UNICEF to salt iodization over the years. Based on documents reviewed, either from RANFOSE or international reports,2,8 as well as data collected by the evaluation, information about the coverage of fortified wheat flour, oil, and salt in the country is sometimes contradictory and is not based on actual data from sampling and chemical analysis of products. However, most respondents report that availability is restricted to select shops in urban areas, with limited or no availability in markets, rural areas, and departments with low accessibility. Even though RANFOSE is a policy project and therefore not directly targeting specific areas or population groups, the ET found that the project has made concerted efforts to raise awareness among WRA and mothers of CU5 (vulnerable groups targeted through the Nutrition Strategic Plan for Haiti)9 on the benefits of fortified foods by training HPs and partnering with civil society organizations (CSO) and nongovernmental organizations (NGOs) that work directly with these target groups. AVAILABILITY OF OTHER FORTIFIED FOODS Although the law on food fortification is limited to specific products, there are other fortified products on the market, such as butter, margarine, mamba, peanut butter, and cereals. However, it is not clear what specifications are being used in these products. Moreover, according to both public and private sector actors, the widespread benefits of fortification could be strengthened by fortifying other products on the market, such as rice, pasta, and maize, which are widely consumed by the Haitian population. 5 USDA & GAIN “Grain And Feed Annual”, Haiti 2023, https://www.fas.usda.gov/data/haiti-grain-and-feed-annual-4 6 Food and Agriculture Organization (FAO). Food Balance Sheets (2010- ). https://www.fao.org/faostat/en/#data/FBS/report 7 Malaysian Palm Oil Council. https://mpoc.org.my/palm-oil-market-chain-in-haiti/ 8 Food and Agriculture Organization (FAO). Food Balance Sheets (2010- ). https://www.fao.org/faostat/en/#data/FBS/report 9 Ministère de la Santé Publique et de la Population.Unité de Coordination de Programme National D’alimentation et de Nutrition UCPNANU. Plan Stratégique de Nutrition 2013-2018. 2013. USAID.GOV RANFOSE FINAL PERFORMANCE EVALUATION REPORT | 7 FACTORS THAT FACILITATE OR INHIBIT THE AVAILABILITY OF FORTIFIED FOODS Among the key factors facilitating the availability of food fortification in Haiti are the motivation of industries to fortify foods and the demand for these products, with the latter increasing as awareness of the benefits of fortified foods grows, according to a variety of respondents. However, these two aspects alone are stalled by difficulties associated with the availability of raw materials, the legislative and operational environment, economic instabilities, and product identification, all of which hamper the availability of fortified foods. KIIs collectively suggested the following key factors inhibit the availability of fortified foods. “The micronutrients have to be cleared at exorbitant prices. They sometimes impose slips with false amounts on us and we cannot rectify it, we just have to give in. These factors push me to sell our products at a price that allows me to stay in the market and be able to order more products.” – KII Industry “Sometimes we spend six months without reaching some areas because of insecurity.” – KII Industry Raw Materials The private sector has the will to fortify foods but is discouraged by the high customs tariffs that sometimes apply to premix imports into Haiti, which are not standardized. On the other hand, the Food Fortification Law states that the GoH will subsidize the premix,10 but this has not been implemented yet. Haiti does not produce wheat or oil but relies on imports to satisfy domestic demand11 and, therefore, suffers the associated customs tariffs. With the additional cost of the premix and equipment related to fortification, which are also imported, production prices go up even further. Legislative Environment The evaluation finds that the lack of approved application texts is a key limitation to increasing the availability of fortified foods, as well as to ensuring the compliance of the ones already being fortified. It inhibits oil producers that are still not engaged from fortifying and discourages wheat flour producers from continuing to do so. Some industries refuse to fortify because of the absence of the application texts, while others refuse or are discouraged from continuing fortifying because there are non-compliant, sometimes illegal wheat flour and oil products entering Haiti that are more competitive. Controls and enforcement are lacking, making fortification unattractive to some industries. Operating Environment The operating environment is highly unstable, and the availability of fortified foods is highly volatile. Insecurity, bad roads, fuel scarcities, corruption, and COVID-19 hinder the production and distribution of fortified foods. At the production level, it creates irregularities in raw material supply, inhibiting the constant availability of fortified foods. At the distribution level, areas like Plateau Central and Bas Artibonite, where gangs operate, have difficulties receiving supplies. Some companies interviewed distribute fortified foods to St. Mark’s via boats. This situation inhibits the availability of fortified foods for the population outside the metropolitan area or in remote areas. 10 Food Fortification Law. 2017. Le Moniteur. Journal Officiel de la Republique D'Haiti. 172 Année. Special No. 6 11 USDA & GAIN “Grain and Feed Annual” Haiti 2022, https://www.fas.usda.gov/data/haiti-grain-and-feed- annual-4 8 | RANFOSE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV "Tariffs reach 40–45 percent of Cost, Insurance, and Freight (CIFs). In the end, it's the consumer who pays. Given living conditions, if this continues, I may stop fortifying." – FFWG Economic Pressures Food fortification incurs a cost that affects a) producers, who can’t always afford the raw materials and equipment for fortification; b) importers, for whom customs tariffs are high; and c) consumers, who can’t afford fortified foods, especially in a context of inflation. Lack of Entry Barriers According to KII respondents, smuggled imports (fortified and unfortified) compete with fortified foods as they are often cheaper but reportedly not adequately fortified or pose food safety issues. Smuggling cases are common in Bas Artibonite, Plateau centra