(2021) Premye Rapò sou Evalyasyon Final Pwogram EFSP WV Ayiti nan Lagonav

Emergency Food Security Program in La Gonâve USAID EFSP #72DFFP19GR00074 Final Evaluation Study Final Report Submitted by: Amisial LEDIX, Economist -Statisticia n/Monitoring & Evaluation Sp ecialist In collaboration with : Amos MONTREUIL JEAN, Agronom ist/Food Security and Nutrition Specialist Obed JULES, IT Technology, Economic Development Document Version Date Author(s) Final Evaluation’s Initial Report 1.0 August 28 th , 2021 Amisial Ledix & Col. World Vision International Haiti USAID EFSP 2 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Table of Contents 1 Acknowledgements 3 List of Tables, Pictures and Charts 4 List of Abbreviations and Acronyms 6 Executive Summary 7 1. Final Evaluation’s Background 10 2. Purpose of the Final Evaluation 11 3. Work Methodology 11 3.1. Quantitative Survey 14 a) Calculation of Sample Size 17 b) Processing and analysis of quantitative data 22 3.2. Qualitative Survey 23 a) Focus Groups 24 b) Key Interviews 24 c) Evaluation of the Program’s Effects on the Markets 25 d) Processing and analysis of qualitative data 25 4. Constraints and Limitations of the Study 26 5. Analysis of the Final Evaluation’s Findings 26 5.1 Broad Characteristics of Beneficiary Households 26 5.2 Analysis of Evaluative Questions 33 5.2.1 Relevance 33 5.2.2 Profitability 37 5.2.3 Effectiveness 38 5.2.4 Impact 49 5.2.5 Sustainability 52 5.2.6 Linkages, Overlaps and Exit Strategies 54 5.3 Analysis of the program’s effects on local markets and certain interest groups (women and men; youth population; boys and girls, etc.) 57 6. Conclusion and Recommendations 60 7. Bibliographical References 65 8. Appendices 66 Appendix I: FCS Indicator 66 Appendix II: HHS Indicator 67 Appendix III: Reduced Coping Strategy Index (rCSI) 68 Appendix IV: Data collection tools (in separate files) 70 USAID EFSP 3 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Acknowledgements We, the team of consultants in charge of this study would like to express, in a very special manner, our gratitude to all those who contributed to the completion of this work. In spite of the various challenges and constraints, we were able to succeed thanks to your active and efficient participation. First, we thank the Monitoring and Evaluation team at the World Vision Haiti’ s National Office for their open and fruitful cooperation. We had very useful interactions with those we discussed with, and that helped achieve a smooth working process. We also wish to thank the EFSP Project Manager with whom we also spoke during the meeting that set this study’s guidelines. In addition, we thank the Monitoring and Evaluation Officer of the La Gonâve Office for his support in making the required information available to facilitate field operations. We also thank the investigators and supervisors who were in the field to collect data from beneficiaries within the project’ s various intervention communities in spite of the fact that this activity started exactly at the same time the country was hit by the devastating August 14 th , 2021 earthquake, in the larger South. The team of consultants still believes that this study’s conclusions and recommendations will adequately help the EFSP project team face the challenges and achieve the expected results for the benefit of beneficiaries living in the organization’ s intervention areas. USAID EFSP 4 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation List of Tables, Pictures and Charts Table 1 : Number of beneficiaries by commune and communal section ........................................ 19 Table 2: Breakdown of the number of beneficiary households to be surveyed in Anse-à-Galets by communal section ................................................................................................................................ 20 Table 3: Breakdown of the number of beneficiary households to be surveyed in Pointe-à- Raquettes by communal section ......................................................................................................... 21 Table 4: Breakdown of the sample of children aged 0-59 months to be surveyed in Anse-à-Galets ............................................................................................................................................................... 22 Table 5: Breakdown of the sample of children aged 0-59 months to be surveyed in Pointe-à- Raquettes .............................................................................................................................................. 22 Table 6: Number of focus groups per commune and category of beneficiaries ............................. 24 Table 7: Number of key interviews by commune and category of stakeholder ............................. 24 Table 8 : Distribution of respondents by commune according to the type of habitat (n = 726) ... 30 Table 9 : Distribution of respondents by commune according to the head of household’s main activity .................................................................................................................................................. 30 Table 10 : Distribution of respondents by commune according to the head of household’s second activity .................................................................................................................................................. 31 Table 11 : Key indicators measured and their progress against target .......................................... 39 Table 12 : Food consumption score of project beneficiaries by commune .................................... 40 Table 13 : Hunger score of project beneficiaries by department .................................................... 41 Table 14 : Dietary Diversity Score by department ........................................................................... 42 Table 15 : Percentage of food use by type of project beneficiaries for the island of La Gonâve and the communes ............................................................................................................................... 42 Table 16 : Percentage of households where adults and children eat at least 2 meals a day (adults and children) ........................................................................................................................................ 43 Table 17 : Reduced Coping Strategies Index (ISSr/rCSI) ............................................................... 43 Table 18 : Percentage of targeted households using and/or benefiting from community assets created/rehabilitated ........................................................................................................................... 44 Table 19 : Percentage of targeted households reporting an increase in the amount saved through the program ......................................................................................................................................... 45 Table 20 : Percentage of households adopting best practices in health and nutrition .................. 46 Table 21 : Prevalence of acute malnutrition by commune .............................................................. 46 Table 22 : Comparison of Baseline and Final Evaluation Results .................................................. 50 Table 23: Level of satisfaction in terms of meeting criteria ............................................................ 56 Chart 1  : Age pyramid of EFSP Program beneficiaries in the commune of Anse- à-Galets ........................ 27 Chart 2 : Age pyramid of EFSP Program beneficiaries in the commune of Pointe-à-Raquettes ................ 27 Chart 3 : Distribution of respondents by commune according to their sex (n=726) .................................... 27 Chart 4 : Distribution of respondents by commune according to household status (n= 726) ..................... 27 Chart 5 : Distribution of heads of households by commune according to their education level (n=726) .. 28 Chart 6 : Distribution of heads of household by commune according to their marital status (n= 726) ..... 28 Chart 7: Distribution of respondents by commune according to whether or not there is at least one orphan in the household ..................................................................................................................................... 28 Chart 8 : Distribution of respondents by commune according to whether or not there is a pregnant woman in the household ..................................................................................................................................... 29 Chart 9 : Distribution of respondents by commune according to whether or not there is at least one disabled person in the household ....................................................................................................................... 29 USAID EFSP 5 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Chart 10 : Distribution of respondents by commune according to the monthly household income bracket .............................................................................................................................................................................. 32 Chart 11  : Distribution of respondents by commune according to the first item of annual household expenditure .......................................................................................................................................................... 32 Chart 12 : Distribution of respondents by commune according to the monthly household income bracket 32 Chart 13 : Hunger score of project beneficiaries by department .................................................................. 39 USAID EFSP 6 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation List of Abbreviations and Acronyms LA Local Aut horities ASCP Agent Sanitaire Communautaire Polyvalent (Community Health Worker) FVFW Food Voucher for Work EFSP Emergency Food Security Program FCS Food Consumption Score FFP Food for Peace GPS Global Positioning System HDDS Household Dietary Diversity Score HHS Household Hunger Scale IPC Integrated Phase Classification OECD Organization for Economic Cooperation and Development rCSI Reduced Coping Strategies Index S4T Savings for Transformation SPSS Statistical Package for the Social Sciences ToR Terms of Reference USAID U.S. Agency for International Development UVT Unconditional Voucher Transfer VFW Voucher for Work USAID EFSP 7 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Executive Summary The Emergency Food Security Program (EFSP), implemented by WV in two of the West department’s communes, specifically on the island of Gonâve (Pointe-à-Raquettes and Anse- à- Galets), aims to provide immediate access to food to communities in crises and emergencies (IPC 3 and 4) and help them access training on good health and nutritional practices. This 15- month Emergency Food Security Program in La Gonâve provided support to 5,770 vulnerable households (28,850 people) to alleviate the crisis-level food insecurity as a result of depleted harvests and eroded livelihoods. This report presents an analysis of the findings of this program’s evaluation by reviewing the achievement level of the goal, objectives and outputs of the EFSP project and the extent to which they were achieved. To carry out this work, we considered both quantitative and qualitative approach and, to collect the data required by this evaluation, a representative sample of 726 beneficiaries (365 in Anse- à-Galets and 361 in Pointe-à-Raquettes) were surveyed. On the other hand, to implement the qualitative approach, 25 key interviews and 14 focus groups were conducted with other project stakeholders (project managers, government institutions, community leaders, l ocal authorities, pregnant women, nursing women, individuals with reduced mobility, members of community savings and credit groups, and young people). The results of this evaluation show a study universe characterized by the following aspects: • The 0-24-year age groups are the most represented in the 2 intervention communes of beneficiary households’ populations. Data also show that in both communes, over 24 years of age, there are slightly more women than men in beneficiary households. • More than 25% of households are in fact led by women with slightly more cases in Anse-à-Galets. • In Anse-à-Galets, just as in Pointe- à-Raquettes, more than 70% of beneficiaries surveyed are in a common- law relationship or are married. • Beneficiaries’ houses are mostly made of tin roofs, which are respectively 77% in Anse- à-Galets and 87% in Pointe-à-Raquettes. This is consistent with the surveyed beneficiaries’ level of vulnerability. • Unfortunately, more people are in charcoal production (6% in Anse-à-Galets and 7% in Pointe-à-Raquettes) than in fishing, even though we are on an island (4% of fishermen in Anse-à-Galets and 6% in Pointe-à-Raquettes). • Most people are living in poverty with a monthly income of less than 5,000 HTG (around 50 USD), which is less than 1.65 USD a day in both communes. By analyzing the quantitative and qualitative data collected to provide answers to evaluation questions, the overall findings show the project fulfilled very well the relevance criterion in the sense that the priorities of targeted vulnerable households were taken into account in the activities. USAID EFSP 8 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation The profitability/efficiency criterion, on the other hand, was not met by the project. Internal systems slowed down the implementation of activities, particularly in the nutritional component. The recruitment process for 2 nurses was unsuccessful. The slow delivery of vendor checks negatively affected their resupplying. In terms of project effectiveness, only 1 indicator (percentage of targeted households with an acceptable food consumption score (FCS) did not increase according to the desired trend despite the intervention of the project among those mentioned in the terms of reference. It is noteworthy that all of them reached at least 50% of their target. So, the project has met this criterion in a way. The project’s impact criterion was met insofar as the 5,770 households were reached in the various distribution cycles. In addition, the difference test shows at least 6 indicators, out of the 10 mentioned in the terms of reference, with significant change in their value in the desired trend, when comparing the situation at the start and at the end of the project. The sustainability criterion, for its part, was moderately met within this project in the sense that certain project actions such as community savings and loan groups can continue past the discontinuation of donor funding. However, it is not sure that ASCP will keep supporting the communities, given the fact that they have been let down by the government. Some are not even included in the government’s budget. Finally, regarding linkages, overlaps and exit strategies, we can say that the project was successful in the sense that it received investments from other programs in the use of SIMAST for targeting but especially with the presence community savings groups created since Kore Lavi that made the overall action sustainable. USAID EFSP 9 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Summary of project indicator results Indicators Island of la Gonâve (%) ± CI Baseline values Endline values Desired trend Current trend Target Explanations Percentage of targeted households with an acceptable food consumption score (FCS) 48.2 ± 6.2% 46%±7.2% + - 75% Throughout the project intervention, the country's macroeconomic situation deteriorated due to socio- political crises and the slowdown in activities with COVID19. There was high inflation, the intervention of the project was not enough in this context to improve the household food consumption score. Prevalence of households with little or no hunger (Household Hunger Scale - HHS) (score 0-1) 15.5 ±4.7% 47.3%±7.3% + + 70% The desired trend is maintained but the target has not been reached due to the shocks mentioned above. Households that were at the bottom of the vulnerability scale despite the intervention. Proportion of households consuming at least 6 food groups in the previous month 73.9±5.6% 89.5±2.2% + + 100% Idem Percentage of food use by type (household consumption) 95±2.8% 98.9% ±1.6% + + 100% Idem Percentage of households where adults and children eat at least 2 meals per day (adults and children) 67.2%± 2.8% 95% ±3.1% + + 70% Reduced Coping Strategies Index (% of households with a less serious or moderate index) 62.5±6.3% 68.8±6.45% + + 70% Idem Percentage of targeted households using and/or benefiting from community assets created/rehabilitated. 0% 71.6±3.3% + + 60% Percentage of targeted households reporting an increase in the amount saved thanks to the program. 0% 87.2±10.5% + + 60% Percentage of households adopting best practices in health and nutrition 0% 31%±6.70% + + 60% Prevalence of Global acute malnutrition (GAM) 6% 5.8±2.2% - - 10% + : increased trend ; - : decreased trend USAID EFSP 10 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 1. Final Evaluation’s Background La Gonâve, one of the most food insecure regions in Haiti, is currently experiencing extreme food insecurity in its rural areas. The poor harvests for spring and fall of the 2017- 2018 season caused 45% of households (around 40,000 people in IPC 3 and 4) to be at risk of a serious food crisis if urgent aid is not provided to this population which barely manages to meet its basic food needs (OCHA Haiti, 2019) 1 . The sorghum and pigeon pea harvests, the two most dominant crops of the fall season in La Gonâve, are estimated at less than 40 percent of what is traditionally considered a normal fall/winter harvest. The last failed harvest had an impact on food security in rural areas and, according to the latest surveys by CNSA and FEWSNET, the crisis will most likely be prolonged until the next spring harvest. Haiti’s precarious economic situation, characterized by currency depreciation and inflation, continues to drive up the prices of staple foods, especially grains (IPC Haiti) 2 . Access to food is even more difficult for most of the inhabitants, as 60% of the food consumed is purchased outside the island. To cope with this situation, people sell their livestock and non- productive assets (FEWSNET, 2019) 3 . Other coping strategies include migration to the Dominican Republic, South America, particularly Chile or Brazil, labor sale, prostitution, fishing, and some households have already consumed their seed reserves (World Vision Haiti) 4 . This is the context in which World Vision Haiti, thanks to funding from USAID, implemented a 15-month Emergency Food Security Program in La Gonâve providing support to 5,770 vulnerable households (28,850 people) to alleviate critical food insecurity resulting from depleted harvests and eroded livelihoods. This project targeted the 2 communes of the island of La Gonâve (in IPC 3 & 4 and a crisis and humanitarian emergency situation) and included the following activities: ➔ Unconditional electronic voucher transfer (UVT) ➔ Conditional voucher Transfer-Food Voucher for Work (VFW) ➔ Essential complementary activities: agriculture, livelihoods, savings for transformation (S4T) and nutrition This document serves as the report for this final evaluation of the EFSP program, analyzing the field findings from the intervention communities according to evaluation questions, in the light of OECD criteria. 1 https://www.humanitarianresponse.info/en/operations/haiti/document/haiti-2019-humanitarian- needs-overview- 2019-summary- january-2019 2 http://www.ipcinfo.org/ipc-country-analysis/details-map/en/c/1068538/?iso3=HTI 3 http://fews.net/node/22705 4 https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/assessments/la-gonave-one-pager.pdf USAID EFSP 11 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 2. Purpose of the Final Evaluation The overall purpose of this final evaluation is to analyze the level of achievement of the goal, objectives and outcomes of the EFSP project and the extent to which they were achieved. This is a performance review. More specifically, the final evaluation aims at the following specific objectives : ● Evaluate program’s achievements with respect to goal, objectives, outcomes and targets. ● Assess program’s effects on local markets and certain interest groups (women and men; youth population; boys and girls, etc.). ● Evaluate the effectiveness and relevance of the modality, transfers and complementary interventions to achieve program’s outcomes. ● Identify best practices, lessons learned, strengths and challenges in the program’s design, including the logical framework and implementation to achieve project’s achievements. ● Recommended strategies for other projects or new interventions. 3. Work Methodolog y To achieve the objectives of this final evaluation, a dual quantitative and qualitative approach was considered. This approach combined the collection of project-related secondary data and primary data among project partners and participants in the intervention communities. The evaluation questions mentioned in the Terms of Reference are based on OECD criteria. The following tables clarify the data sources that helped address the different questions according to the Relevance- Effectiveness- Efficiency-Impact-Sustainability criteria: USAID EFSP 12 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Criteria Evaluative Questions Data Sources/Collection Methods Relevance What are the opinions of stakeholders on the nature and quality of implementation? Were relevant government officials involved? Has the project implementation strategy been adjusted to take into account the realities on the ground? If so, in what way? Were program activities and outputs consistent with the expected impacts and effects? Did they meet the needs and priorities of the most vulnerable and targeted pregnant and nursing women? - Quantitative survey of beneficiaries - Key interviews with project manager and/or project component managers - Key interviews with the project’s key partners (Local Authorities (Mayors / CASEC), MAST, MSPP, Wesleyan Hospital, Mother-leaders, ...) - Focus group with beneficiaries from vulnerable groups (pregnant women, nursing women, people with reduced mobility, etc.), mothers’ clubs and ASCP Profitability Did the project have adequate and appropriate resources (human, financial and capital) for its implementation? If not, how was it addressed? Were quality control and accountability measures in place and consistently applied during the review, approval, funding disbursement, monitoring, and reporting phases? Do recipient comments indicate widespread instances where funds (vouchers) were taxed or stolen, or where receiving a voucher represented a protection risk? - Key interviews with project manager and/or project component managers - Focus group with beneficiaries from vulnerable groups (pregnant women, nursing women, people with reduced mobility, etc.), mothers’ clubs and ASCP - Consultation of project’s financial reports - Key interviews with the project’s key partners (Local Authorities (Mayors / CASEC), MAST, MSPP, Wesleyan Hospital, Mother-leaders, ...) Efficiency To what extent were the objectives achieved? What were the main factors that did or did not influence the achievement of the objectives? Did the M&E system provide appropriate and reliable quality information to measure the planned indicators? - Quantitative survey of beneficiaries - Key interviews with project manager and/or project component managers - Key interviews with the project’s key partners (Local Authorities (Mayors / CASEC), MAST, MSPP, Wesleyan Hospital, Mother-leaders, ...) USAID EFSP 13 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Criteria Evaluative Questions Data Sources/Collection Methods Efficiency (continued) How effective was the project model in terms of design, relevance, management and accountability? How effective was the program in terms of implementation (coordination, cooperation, effectiveness, standardization)? Have humanitarian standards been met and humanitarian principles complied with (SPHERE, HAPs, and Codes of conduct)? What measures were taken to identify and reduce possible negative effects? -Consultation of M&E tools/periodic reports (Post-Distribution Monitoring if available, Monitoring plan, Indicator Tracking Table, etc.) -Focus group with beneficiaries from vulnerable groups (pregnant women, nursing women, people with reduced mobility, etc.), mothers’ club and ASCP Impact Did the project reach the expected number of beneficiaries and territorial coverage? To what extent has the project contributed to reducing beneficiaries’ vulnerability level? What are the unintended positive and negative impacts of project implementation? How satisfied are the communities with the response? Did the program require more time from the women than from men? What gender-specific issues were addressed? Did the voucher project affect the market and context in any way (did the voucher assistance have an impact on inflation?) Did the voucher aid affect food availability on the markets? How did the voucher aid affect local trade? - Quantitative survey of beneficiaries - Consultation of beneficiaries’ monitoring databases - Key interviews with project manager and/or project component managers/M&E manager - Key interviews with the project’s key partners (Local Authorities (Mayors / CASEC), MAST, MSPP, Wesleyan Hospital, Mother-leaders, ...) - Key interviews with some vendors from the voucher component and a few others not involved with the project - Focus group with beneficiaries from vulnerable groups (pregnant women, nursing women, people with reduced mobility, etc.), mothers’ club and ASCP USAID EFSP 14 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Criteria Evaluative Questions Data Sources/Collection Methods Sustainability To what extent will project benefits continue after donor funding ends? Are positive effects sustainable? To what extent did the project take into account factors that, in experience, have a major influence on sustainability such as economic, ecological, social and cultural aspects? What sustainability drivers are obvious (local ownership, partnership, transformed relationships, household and family resilience)? - Key interviews with project manager and/or project component managers - Key interviews with key project partners (Local Authorities (Mayors / CASEC), MAST, MSPP, Wesleyan Hospital, Mother- leaders ...) - Focus group with beneficiaries from vulnerable groups (pregnant women, nursing women, people with reduced mobility, etc.), mothers’ club and ASCP Linkages, overlaps and exit strategies To what extent did the project take advantage of other US Government (USG) and non- USG investments in the same area to facilitate linkages with complementary services, overlaying previous investments and implementing exit strategies to minimize reliance on external support? To what extent has the project aligned and integrated with the host country’s service provision strategy/policy for social protection? - Consultation of government policy document related to the issues addressed by the project - Key interviews with project manager and/or project component managers/M&E manager - Key interviews with the project’s key partners (Local Authorities (Mayors / CASEC), MAST, MSPP, Wesleyan Hospital, Mother-leaders, and NGO directors implementing similar projects...) - Focus group with beneficiaries from vulnerable groups (pregnant women, nursing women, people with reduced mobility, etc.), mothers’ club and ASCP The collection tools that were developed in relation to data sources listed in the tables above have taken into account lessons and major challenges, as well as strengths, weaknesses, opportunities and threats experienced by the project during its implementation and the way they were addressed. The evaluation was then able to examine the effects or potential impacts of the project on participants and their community knowledge, attitudes and practices. 3.1. Quantitative Survey A quantitative sampling survey was conducted among project beneficiaries in the island of La Gonâve’s intervention communes, which are Anse -à-Galets and Pointe-à-Raquettes. As mentioned in the ToRs, the same sampling methodology used for the baseline was applied. Thus, a random stratified with Proportional Probability by Size (PPS) Cluster Sampling was used in each communal section, and a simple random sampling was used to select beneficiaries from the predefined list. One must say that this strategy is made possible by the existence of beneficiary lists serving as the survey basis. USAID EFSP 15 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation The structured questionnaire used in the project baseline for the collection of quantitative data is used in this study with additions taking into account the evaluative questions based on OECD criteria. The questions take into account the calculation method for project’s key indicators as presented: Key Indicators Definition of Indicators Data Collection Method Percentage of targeted households with an acceptable food consumption score (FCS) The frequency-weighted dietary diversity score is calculated from the consumption frequency for the various food groups consumed (Appendix 1) by a household during the 7 days preceding the survey. Quantitative survey of project beneficiary households Prevalence of households with little or no hunger (household hunger scale – HHS) Is essentially a behavioral measure that tends to capture more serious behaviors such as: - Was there never anything to eat in your house? For lack of resources to get food? - Did you or a member of your family fall asleep at night hungry because there was not enough food? - Did you or a member of your family go all day and night without eating anything because there was not enough food? Quantitative survey of project beneficiary households Proportion of households consuming at least 6 food groups in the previous month Household Dietary Diversity Scale (HDDS): Dietary diversity represents the number of different foods or food groups consumed in a given reference period. - Similar to the FCS, but usually with a 24-hour recall period with no information on frequency or weighted categorical thresholds - It is an indirect measure of household access to food. Quantitative survey of project beneficiary households Percentage of food use by type (household consumption, sale, exchange, livestock feed) This indicator takes into account foods generally available in the household, regardless of its origin and quantity. It analyzes the different modes of use, namely: a. Self-consumption b. The sale c. Exchange d. Livestock feed The priority use of food can inform on the economic situation of the household. Quantitative survey of project beneficiary households USAID EFSP 16 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Key Indicators Definition of Indicators Data Collection Methods Reduced Coping Strategies Index (rCSI) rCSI measures behavior: What people do when they don’t have access to enough food by answering the question: What do you do when you don’t have enough to eat and you don’t have enough money to buy food? - Measures the adjustments households make in consumption and livelihoods. These may be changes in consumption, reduced spending, or income growth; - rCSI tends to measure less severe coping behaviors. - rCSI uses the five most common strategies with standardized weights: 1- Focus on less preferred and less expensive foods? 2- Borrow food or rely on the help of a friend or relative? 3- Limit portion size at mealtime? 4- Restrict adult consumption so that young children can eat? 5- Reduce the number of meals consumed per day? (Appendix III) Quantitative household survey Percentage of target households using and/or benefiting from community assets created/rehabilitated. This indicator takes into account households living in areas using and/or benefiting from community infrastructure (stone cord, earthen dike (soil conservation works/watershed protection), etc.) that were created or rehabilitated by the project. Quantitative household surveys Percentage of targeted households reporting an increase in the amount saved because of the program. This indicator refers to the building of cash assets for the families benefiting from the project, whether through cash transfers, IGAs, agricultural production activities, remittances and S4Ts which aim to increase household income, reduce expenditure ite ms (such as food), strengthen savings and thus promote asset growth. Quantitative household surveys Percentage of households adopting best practices in health and nutrition This indicator analyzes the understanding and application of nutritional and family health practices within households as a direct result of the program. The analyzes relate to: -For a pregnant woman, a household adopting good health and nutrition practices if it takes into consideration one or more of these aspects: 1. Iron and folic acid supplementation 2. Advice on mother and/or child nutrition 3. Calcium supplementation 4. Multiple micronutrient supplementation Quantitative household surveys USAID EFSP 17 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 5. Direct food aid from fortified/specialized food products (i.e., CSB+, Super cereal Plus, RUTF, RUTF, etc.) - Children under 5, including one or more of these aspects: • Immediate, exclusive and continuous breastfeeding • Appropriate, adequate and safe complementary foods from 6 to 24 months • Vitamin A supplementation during the last 6 months • Zinc supplementation during episodes of diarrhea • Multiple micronutrient powder supplementation (MNP) • Treatment of severe acute malnutrition • Treatment of moderate acute malnutrition • Direct food aid from fortified/specialty food products (i.e., CSB+, Super cereal Plus, RUTF, ASPE, etc.) (Ref: FFP indicators, page 54) Key Indicators Definition of Indicators Method of Data Collection Prevalence of Global Acute Malnutrition (GAM) To measure GAM, anthropometric measurement is taken (weight-for-height or MUAC) for children aged 0 to 59 months to calculate wasting. All children with a weight-to-weight Z score less than -2 standard deviation and/or edema are classified as acutely malnourished. Anthropometric data from a quantitative survey of a sub- sample of children aged 0 to 59 months from project beneficiaries. The indicator values found in this final study were compared with those established during the baseline study. A comparison test of means or proportions was carried out to see if there are statistically significant differences or not in order to confirm whether the project had the desired effects and impacts. a) Calculation of Sample Size To determine the size of the beneficiary sample, the calculation formula is based on the FFP/USAID protocol in that the required minimum number (339 per commune ) of beneficiaries to be surveyed was met for food security indicators (FCS, HHS, rCSI). One-step random sampling (One-stage Single Random Sample (SRS)) is used just as it was at the baseline, considering each commune as a layer. The values of the indicators were established by commune. The formula for calculating the survey’s sample size is as follows: ������������ ������������������������������������������������������������������������������������=������������ ������������������������������������ ⎣ ⎢ ⎢ ⎡������������ 1−∝ �2�������������1−������������� + ������������ 1−������������������������� 1,�������������������������������������1−������������ 1,�������������������������������������+������������ 2,�������������������������������������1−������������ 2,������������������������������������� ������������ ⎦ ⎥ ⎥ ⎤ 2 Where ������������ ������������������������������������������������������������������������������������ = is the initial sample size required by surveys for each one of the two- stage points ������������ = ������������ 1,������������������������������������- ������������ 2,������������������������������������ = minimum effect size to be achieved over the period specified by both surveys USAID EFSP 18 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation - The calculation of the sample size for Anse -à-Galets ������������ ������������,������������������������������������= 0.27 proportion of the population with a less severe rCSI at baseline. FCS is chosen over the other 2 (namely the FCS and HHS) as the one that yielded the most suitable sample, although insufficient for the measurement of food insecurity indicators. ������������ ������������,������������������������������������= 0.37 proportion of the population with a less serious rCSI in Anse -à-Galets in the 3rd MDP carried out in March 2021. ������������ = ������������ 1,������������������������������������+ ������������ 2,������������������������������������ 2= 0.27+ 0.37 2 =0.32 ������������ 1−∝ is the value of the normal probability distribution corresponding to a confidence level of 1-������������. For 1- ������������ =0.95, the corresponding value is ������������ 0,95=1,64. ������������ 1−������������ is the value of the normal probability distribution corresponding to a confidence level of 1-.������������. For 1-������������ =0.80, the corresponding value is ������������ 0,80=0,84. ������������ ������������������������������������ : It is the estimated design effect (DEFF) of the survey which is 1 in this sampling plan (One-stage Simple Random Sample (SRS)). Hence ������������ ������������������������������������������������������������������������������������=1∗� 1.64∗�2∗0.32(1−0.32) + 0.84∗�0.27(1−0.27)+0.37(1−0.37) 0.27−0.37 � 2 ������������ ������������������������������������������������������������������������������������=267 There is indeed a non- response rate (generally set at 1.1, corresponding to 10%, but may change depending on the context). ������������ ������������������������������������������������������������������������������������ ������������������������������������������������������������������������������������ 267∗1.1=294 ������������������������������������������������������������������������������������������������������������������������������������������������������������ - The calculation of the sample size for Pointe-à-Raquettes ������������ ������������,������������������������������������= 0.12 proportion of the population with a less severe rCSI in baseline. ������������ ������������,������������������������������������= 0.27 proportion of the population with a less severe rCSI in Anse -à-Galets in 3rd MDP carried out in March 2021. ������������ = ������������ 1,������������������������������������+ ������������ 2,������������������������������������ 2= 0.12+ 0.27 2 =0.195 ������������ 1−∝ is the value of the normal probability distribution corresponding to a confidence level 1- ������������. For 1-������������ =0,95, the corresponding value is ������������ 0,95=1,64. USAID EFSP 19 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation ������������ 1−������������ is the value of the normal probability distribution corresponding to a confidence level 1- ������������. For 1-������������ =0,80, the corresponding value is ������������ 0,80=0,84. ������������ ������������������������������������ : It is the estimated design effect (DEFF) of the survey which is 1 in this sampling plan (One-stage Simple Random Sample (SRS)). Hence ������������ ������������������������������������������������������������������������������������ =1∗� 1.64∗�2∗0.195(1−0.195) + 0.84∗�0.12(1−0.12)+0.27(1−0.27) 0.12−0.27 � 2 ������������ ������������������������������������������������������������������������������������=85 There is indeed a non- response rate (generally set at 1.1 corresponding to 10% but may change depending on the context). ������������ ������������������������������������������������������������������������������������ ������������������������������������������������������������������������������������ 85∗1.1=94 ������������������������������������������������������������������������������������������������������������������������������������������������������������ As for each commune, to be able to measure FCS, HHS, rCSI in this final evaluation, we need at least 339 units, therefore the sample size per commune is adjusted to 339. So, overall we should survey on the island of La Gonâve at least 678 beneficiary households in the two project intervention communes in this final evaluation. The following table breaks down the total number of beneficiaries by commune, communal section (including target communities): Table 1 : Number of beneficiaries by commune and communal section Communes Communal Sections/Town Target Communities Number of Beneficiaries Anse-à-Galets Ville Ville 875 1 st Palma Palma 168 2 nd Petite Source Fortuna, Mare Sucrin, Port Frégard 566 3 rd Grande Source Bois Brûlé, Bois Noir, Les Étroits, Nan Café, Plaine Mapou, Grande Source 1063 4 th Grand Lagon Trou Louis Jeune, Zabricot 866 5 th Picmy Picmy 97 6 th Section Petite Anse Petite Anse 164 Total 3799 Communes Communal Sections/Town Target Communities Number of Beneficiaries Pointe-à-Raquettes 1 st Section la Source Plaine La Source, Latanier 84 2 nd Section Grand Vide Grand vide, Tamarin 456 USAID EFSP 20 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 3 rd Section Trou Louis Dan Griyen, Deux Frères, Morne Trou Louis, Port- de-Bonheur, Port- Trou-Louis 621 4 th Section Pointe-à-Raquettes La Palmiste, Lotoré, Plaisance, Terre Sèche, Ti Palmiste, Bois Pin 539 5 th Section Gros Mangle Bel Platon, Gros Mangles 134 Town Town of Pointe-à-Raquettes 137 Total 1971 Grand total 5770 By considering the lists of beneficiaries by commune as the sampling basis, the sample of households to be surveyed was drawn according to a simple random selection by grouping the target communities by communal section. The aléa () function in MS Excel was used in the lists to select households to be surveyed. This approach ensures a geographical coverage more representative of the beneficiaries’ location. To access the randomly selected households, it was expected that GPS coordinates be used to locate them. However, these not being correct, we had to rely on guides who know the communities well to identify the randomly selected households. The table below shows the allocation of households to be surveyed by communal section/town using the Probability Proportional to Size (PPS) method: Table 2: Breakdown of the number of beneficiary households to be surveyed in Anse-à-Galets by communal section Communal Sections/Town Target Communities Number of Beneficiaries Demograph ic Weight Sample allocation by communal section Number of surveys actually carried out Town Ville 875 0.23 78 78 1 st Palma Palma 168 0.04 15 16 2 nd Petite Source Fortuna, Mare Sucrin, Port Frégard 566 0.15 51 55 3 rd Grande Source Bois Brûlé, Bois Noir, Les Étroits, Nan Café, Plaine Mapou, Grande Source 1063 0.28 95 95 4 th Grand Lagon Trou Louis Jeune, Zabricot 866 0.23 77 77 5 th Picmy Picmy 97 0.03 9 14 6 th Section Petite Anse Petite Anse 164 0.04 15 30 3799 1.00 339 365 USAID EFSP 21 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Table 3: Breakdown of the number of beneficiary households to be surveyed in Pointe-à-Raquettes by communal section Communal Sections/Town Target Communities Number of Beneficiaries Demographic Weight Sample allocation by communal section Number of surveys actually carried out 1 st Section la Source Plaine La Source, Latanier 84 0.04 14 10 2 nd Section Grand Vide Grand Vide, Tamarin 456 0.23 78 80 3 rd Section Trou Louis Dan Griyen, Deux Frères, Morne Trou Louis, Port-de- Bonheur, Port-Trou-Louis 621 0.32 107 114 4 th Section Pointe-à- Raquettes La Palmiste, Lotore, Plaisance, Terre Sèche, Ti Palmiste, Bois Pin 539 0.27 93 94 5 th Section Gros Mangle Bel Platon, Gros Mangles 134 0.07 23 34 Town Town of Pointe-à-Raquettes 137 0.07 24 29 1971 1.00 339 361 Note: 726 beneficiaries were surveyed instead of the planned minimum quantity 678. This surplus is explained by the fact that each interviewer was asked to interview 3 to 5 respondents as a precaution. In fact, when clearing and processing the data, the research team may need to eliminate forms deemed to be inconsistent. To measure the GAM indicator (Prevalence of Global Acute Malnutrition), one of the 3 approaches proposed by the FFP protocol was used, in this case approach B. We looked at the official statistics for the West department on the proportion of the population of children under 5 years old. This proportion is 0.12 (Statistical Report, MSPP, November 2019) and the respective average household sizes for Anse-à-Galets and Pointe-à-Raquettes are 2.2 and 2.4 (according to the list of beneficiaries by commune). Using these parameters for each commune in the USAID’s sample size calculator (USAID, population- based survey sample size calculator) 5 , we then have the respective minimum sizes of children under 5 to be considered for the measurement of the GAM indicator in each of the communes: - n (Anse-à-Galets) = 109 children from 0 to 59 months - n (Pointe-à-Raquettes) = 118 children from 0 to 59 months These samples of children under 5 years were sought among the samples of surveyed households. It should be noted that, within the same household, more than one child under 5 could be considered. So the samples of children were broken down according to the demographic weight of households by commune. The following tables break down the number of children to be considered to measure the GAM by commune: 5 https://www.usaid.gov/documents/1866/population-based-survey- sample- size-calculator USAID EFSP 22 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Table 4: Breakdown of the sample of children aged 0-59 months to be surveyed in Anse-à-Galets Communal Sections/Town Target Communities Demographic Weight Allocation of less than 5 years old samples by communal section Number of children less than 5 actually observed Town Town 0.23 25 30 1 st Palma Palma 0.04 5 7 2 nd Petite Source Fortuna, Mare Sucrin, Port Frégard 0.15 16 25 3 rd Grande Source Bois Brûlé, Bois Noir, Les Étroits, Nan Café, Plaine Mapou, Grande Source 0.28 30 60 4 th Grand Lagon Trou Louis Jeune, Zabricot 0.23 25 39 5 th Picmy Picmy 0.03 3 12 6 th Section Petite Anse Petite Anse 0.04 5 13 Total 1.00 109 196 Table 5: Breakdown of the sample of children aged 0-59 months to be surveyed in Pointe-à-Raquettes Communal Sections/Town Target Communities Number of Beneficiaries Demographic Weight Allocation of less than 5 years old samples by communal section Number of children less than 5 actually observed 1 st Section la Source Plaine La Source, Latanier 84 0.04 5 9 2 nd Section Grand Vide Grand Vide, Tamarin 456 0.23 27 66 3 rd Section Trou Louis Dan Griyen, Deux Frères, Morne Trou Louis, Port-de - Bonheur, Port-Trou- Louis 621 0.32 37 76 4 th Section Pointe- à-Raquettes La Palmiste, Lotoré, Plaisance, Terre Sèche, Ti Palmiste, Bois Pin 539 0.27 32 48 5 th Section Gros Mangle Bel Platon, Gros Mangles 134 0.07 8 19 Town Town of Pointe-à- Raquettes 137 0.07 8 16 Total 1971 1.00 118 234 b) Processing and analysis of quantitative data The processing and analysis of quantitative data helped respond mainly to issues of project effectiveness, impact, sustainability and relevance and also check whether there are unexpected outcomes. Moreover, the indicator values were established in this final study. The values USAID EFSP 23 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation found were compared with those established during the study baseline. A comparison test of means or proportions was performed on MS Excel to see whether there are statistically significant differences or not to actually confirm if the program reached the desired effects and impact as mentioned in section 3.1. Here are the steps followed in implementing the test: 1) H0: P1=P2, there is no significant difference between both values H1: P1≠P2, there is a significant difference between both value s P1: indicator value at baseline P2: indicator value during the final study 2) Materiality threshold: α=5% 3) Conditions to implement the test: case of large samples n1>30 and n2 >30 4) The test statistics are: ��������������1−P�2�−(p1−p2) � ������������1������������1 ������������1 + ������������2������������2 ������������2 ~ N(0,1) (follows a centered and reduced normal). 5) Decision rule: if the test statistic is greater than 1.96 (Table value of N (0.1) for a risk α = 5%), then no rejection of the null hypothesis, i.e. H0. Or we use the pvalue by comparing it with the risk α = 5% . A pvalue> α = 5% => DH0 (non- rejection of the null hypothesis). In EXCEL, the pvalue is obtained using the function: NORMSDIST (test statistic) 6) Decision and conclusion: We can therefore conclude, with a risk of error α = 5%, that the proportion of the final evaluation is not significantly different from that of the baseline. The difference observed between the two is due to sampling fluctuations. In order to have the best possible data quality, technological tools were used to guarantee the validity, reliability, integrity, accuracy and availability of the information collected. In the case of quantitative data, the Ona digital survey platform was used. The survey questionnaire was designed using the XLSForm syntax 6 and then uploaded to the platform. Each enumerator was therefore provided with an Android digital tablet on which the data collection tool was installed. Once the collected data was cleaned and validated, the analysis phase was launched. The main data analysis tools that were used during this phase are the SPSS software 7 and MS Excel. 3.2. Qualitative Survey The qualitative approach, in the final evaluation, was essential not only to complete the quantitative data collected from project beneficiaries but also to validate the findings of the study through information triangulation. It is also important in order to better identify the changes induced by project interventions. It included focus groups and key interviews using the following tools: Guide for focus group (or discussion groups) interviews and Guide for key stakeholder interviews (List of data collection tools in the appendix) . 6 https://xlsform.org/en/ 7 https://www.ibm.com/products/spss-statistics USAID EFSP 24 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation a) Focus Groups Focus groups were held with vulnerable groups of project beneficiaries (pregnant women, nursing women, people with reduced mobility, etc.). They helped identify trends and perceptions from these different categories of relevance, impact, sustainability, performance, efficiency and ownership of project interventions. The team made sure to have the various categories of vulnerable groups represented in the focus groups. Thus, 7 focus groups bringing together 6 to 10 people were held in each of the communes (Table 6). Table 6: Number of focus groups per commune and category of beneficiaries Category of Beneficiaries Number of Focus Groups Total Number Anse-à-Galets Pointe-à-Raquettes Mothers Clubs-Nursing Women 1 1 2 Nursing Women-People with Reduced Mobility 1 1 2 Member of Saving & Credit Groups (S4T) and Agents 2 2 4 Youth 1 1 2 ASCP (Community Health Worker) 1 1 2 Food Vouchers for Work (FVFW) 1 1 2 Total 7 7 14 b) Key Interviews Individual interviews were conducted with project’s key actors in order to deepen key aspects in the analysis of relevance, efficiency, effectiveness, impact and sustainability. Thus, 25 key interviews were conducted as presented in the following table: Table 7: Number of key interviews by commune and category of stakeholder Category of Stakeholder Number of Key Interviews Total Number Anse-à-Galets Pointe-à-Raquettes Manager - - 1 Component Managers (Distribution-Livelihoods- Monitoring and Evaluation) - - 2 Representantive of NGO implementing a similar program (Concern Worldwide) - - 1 MSPP Representative - - 1 MAST Representative - - 1 Local Authorities (CASECs- Mayors) 4 5 9 Mothers-Leaders 2 2 4 USAID EFSP 25 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Vendors (at least 2 suppliers having a contract with the project in each commune) 3 2 5 Wesleyan Hospital focal point 1 - 1 Total 10 9 25 c) Evaluation of the Program’s Effects on the Markets World Vision has provided the evaluation team with the price products monitoring reports in key local markets in order to establish a price fluctuation line (quantitative analysis of price changes) and the seasonality of basic food products included in the vouchers given out to beneficiaries. This analysis helped answer the following questions: • Has the market experienced significant variations (in availability, demand, prices, stakeholders present) since the start of the program? And if so, what are the causes? The team sought to isolate the activity’s endogenous causes from the exogenous causes. How has the program responded to seasonal fluctuations, volatility and possible price differences? • Have the difficulties/constraints/distortions that could be attributable to the program been identified? Then, the team made an analysis of the program’s effects on the markets, which was done qualitatively with major wholesalers and retailers. This completed the quantitative survey of beneficiaries. The team held key interviews with local market actors serving project communities. d) Processing and analysis of qualitative data The analysis of qualitative primary data focused on identifying the various findings of the evaluation by combining the various results and syntheses resulting from the review of secondary data and major trends related to the themes developed in the qualitative surveys. To analyze the data, we followed the following steps: 1) We have defined an analysis plan by identifying the main themes arising from data collection tools. 2) On the basis of these themes, we note the major trends emerging from the focus groups by category of participants and by commune, on the basis of reports provided by focus groups facilitators and recording 8 loaded onto the Atlas.ti qualitative data analysis processing software. 3) We also summarized the key interviews by commune and category of respondents on the basis of the study’s main themes. We identified the relevant information to identify this evaluation’s problematic; 4) Lastly, we made sure to triangulate the findings with existing quantitative data on the defined themes in order to draw conclusions and recommendations. 8 Each pair of facilitators was required to record the focus groups from their phone’s voice recorder. USAID EFSP 26 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 4. Constraints and Limitations of the Study The study was carried out in a particular context marked by strong political instabilities and security challenges, among others. We had to use several strategies to ensure the training and deployment of enumerators in the field because certain areas are completely cut off from the rest of the country. Some had to take the online training by video conference. On the start day, August 14 th , 2021, there was an earthquake that shook the country early in the morning. Despite everything, our team took the measures to go to the field in order to collect the information. Other challenges were those related to traveling and communicating on the island. Entire areas are difficult to access because of the roads condition and the lack of means of transportation on site. On the other hand, the communication issues and unavailability of adequate 3G/4G signal on the island made it impossible to do real-time synchronization of data collected in the field. In terms of limitations, in the post-earthquake context, some judgments from interviewees can be altered and amplified by too much expectancy from them. Following such a disaster, people tend to exaggerate their situation of precarity in order to get help or something in return. As the island of la Gonâve was not very much affected by the earthquake, it is assumed that this had no influence on the responses from beneficiaries. 5. Analysis of the Final Evaluation’s Findings In this section, the findings of the final evaluation are analyzed on the basis of qualitative and quantitative data collected in the field. For that purpose, the broad characteristics of beneficiaries are presented, various evaluation questions are examined one by one according to the OECD criteria and finally, the effects of the project on local markets and its impact on certain interest groups are analyzed. 5.1 Broad Characteristics of Beneficiary Households In this section we present the characteristics of beneficiary households in this survey. The structure of beneficiary households in the two project intervention communes can be seen using the age pyramid ( Figures 1 to 2 ). This graph shows that among project beneficiary households, the 0 to 24 years age groups are the most represented in the 2 intervention communes. We understand that these are mostly very rural communities that are very vulnerable. However, the general trend in Haiti, in recent years, is this urban migration of youth to large cities which occurs very early from the age of 25. Data also show that, in the 2 communes, over 24 years of age, there are slightly more women than men in beneficiary households. USAID EFSP 27 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Chart 1 : Age pyramid of EFSP Program beneficiaries in the commune of Anse-à-Galets Chart 2: Age pyramid of EFSP Program beneficiaries in the commune of Pointe-à- Raquettes In this sample of beneficiaries surveyed, the majority of respondents are women in the 2 communes (Chart 3). Yet, the status of households shows that they are mostly headed by men and women, at the same time, which means that both husband and wife are present in the household (Chart 4). More than 25% of households are in fact headed by women only with slightly more cases in Anse- à-Galets. Chart 3 : Distribution of respondents by commune according to their sex (n=726) Chart 4 : Distribution of respondents by commune according to household status (n= 726) Source : EFSP Project final evaluation in La Gonâve, World Vision Haiti, August 2021 As found in the baseline, households are headed by people with no or low education levels (Chart 5). According to figures from the office of the island of La Gonâve’s School District, in 2014, there were not 20 public schools in total in the 2 communes (12 public schools in Anse- 10.0%5.0%0.0%5.0%10.0%15.0% 0-4 10-14 20-24 30-34 40-44 50-54 60-64 70-74 80-84 FH 15.0%10.0%5.0%0.0%5.0%10.0%15.0% 0-4 10-14 20-24 30-34 40-44 50-54 60-64 70-74 80-84 FH 34% 40% 66% 60% Anse-à-galets Pointe-à-Raquette Male Female 31% 27% 64% 65% 4% 7% Anse-à-galets Pointe-à- Raquette Children No Adult (CNA) Male adult No Female adult (MNF) Female and Male adult (F&M) Female adult No Male adult (FNM) USAID EFSP 28 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation à-Galets and 7 in Pointe-à-Raquettes). With such low number of public schools, the probability of a child from a vulnerable household going to school is therefore very low . Chart 5 : Distribution of heads of households by commune according to their education level (n=726) Source : Final evaluation of the EFSP project at La Gonâve, World Vision Haiti, August 2021 Statistics related to marital status confirm that program beneficiaries largely come from common-law or married households, which means they face the need to feed their children or dependents. In fact, in Anse-à-Galets as in Pointe- à-Raquettes, more than 70% of surveyed beneficiaries are in a common-law relationship or married (Chart 6). Households with at least one orphan are very much in the minority, i.e. 11% in Anse-à-Galets and 6% in Pointe-à- Raquettes (Chart 7). Chart 6 : Distribution of heads of household by commune according to their marital status (n= 726) Chart 7 : Distribution of respondents by commune according to whether or not there is at least one orphan in the household 34% 39% 36% 36% 4% 7% 23% 16% 2% 2% Anse-à-galets Pointe-à-Raquette Illiterate Incomplete primary Complete primary Incomplete secondary Complete secondary Vocational school Incomplete university Complete university 4% 17% 34% 17% 44% 53% 7% 8% 11% 6% Anse-à-galets Pointe-à- Raquette Widower Separated/divo rced Common-law partner Maried Single 89% 94% 11% 6% Anse-à-galets Pointe-à-Raquette No Yes USAID EFSP 29 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 Less than 10% of surveyed households have a pregnant woman, whether in Anse-à-Galets or Pointe-à-Raquettes (Chart 8). This statistic could have surprised us, given the nature of the program that includes a nutritional component targeting pregnant women. But we understand that, after 15 months of implementation, women that were pregnant are no longer in this situation. As for people living with a disability, the samp le of households surveyed account respectively for 13% in Anse-à-Galets and 11% in Pointe-à-Raquettes (Chart 9). This is still a considerable statistic since the program also targeted this subgroup. Chart 8 : Distribution of respondents by commune according to whether or not there is a pregnant woman in the household Chart 9 : Distribution of respondents by commune according to whether or not there is at least one disabled person in the household Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 Chart 10.1 : Distribution of respondents by commune according to whether or not there is a lactating woman in the household Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 92% 94% 8% 6% Anse-à-galets Pointe-à-Raquette No Yes 87% 89% 13% 11% Anse-à-galetsPointe-à-Raquette No Yes 29% 10% 71% 90% Anse-à-Galets Pointe-à-Raquette YesNo USAID EFSP 30 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation The homes of beneficiaries are mainly with tin roofs, i.e. 77% in Anse-à-Galets and 87% in Pointe-à-Raquettes respectively (Table 8). It is obvious that in rural areas this type of habitat characterizes a certain level of vulnerability. But we admit that the lowest level takes into account cottages or other types of thatched houses. Table 8 : Distribution of respondents by commune according to the type of habitat (n = 726) Type of habitat Communes Anse-à-Galets Pointe-à-Raquettes Grand Total Mud house 1% 4% 2% Hovel/Plank house 5% 0% 2% Cottage 7% 4% 5% Tin house 77% 87% 82% Simple low house 8% 4% 6% Two-storey house 0% 0% 0% Other (Concrete, thatched house …) 2% 1% 2% Grand Total 100% 100% 100% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 It was also essential to look at program beneficiaries’ main activities. As is always the case in rural areas, agriculture and trade are the most common activities (Table 9). We were still surprised to see how uncommon fishing was, since we are on an island. Unfo rtunately, even the production of wooden charcoal (6% in Anse-à-Galets and 7% in Pointe-à-Raquettes) exceeds fishing (4% of fishing practitioners in Anse-à-Galets and 6% in Pointe-à-Raquettes). Table 9 : Distribution of respondents by commune according to the head of household’s main activity Main Activities Communes Anse-à-Galets Pointe-à-Raquettes Grand Total Agriculture 37% 42% 40% Construction 2% 2% 2% No Activity 17% 5% 11% Breeding 1% 1% 1% Trade 20% 25% 23% Fishing 4% 6% 5% Transportation 2% 1% 1% Charcoal production 6% 7% 6% Teaching 3% 1% 2% Daily labor sale 5% 9% 7% Mechanic 1% 0% 0% Other (Cabinetmaking, laundry service, sailing…) 2% 2% 2% Grand Total 100% 100% 100% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 As a second economic activity, breeding and charcoal production (Table 10) emerge alongside the other two previously mentioned (Agriculture and trade). Given the condition of the Haitian USAID EFSP 31 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation environment and the natural disasters linked to the climate risks we are facing; it is more than urgent to launch an awareness campaign to reduce the rampant cutting of trees. But it is clear that the scale of such activity testifies to the seriousness of poverty in the intervention communities of this World Vision program. Table 10 : Distribution of respondents by commune according to the head of household’s second activity Secondary Activities Communes Anse-à-Galets Pointe-à-Raquettes Grand Total Agriculture 13% 22% 17% Construction 1% 2% 1% No other activity 51% 25% 38% Breeding 12% 9% 11% Trade 7% 12% 9% Fishing 1% 3% 2% Transportation 0% 1% 1% Charcoal Production 8% 17% 13% Teaching 1% 0% 1% Daily labor sale 5% 7% 6% Mechanic 0% 0% 0% Other (Cabinetmaking, laundry service,…) 1% 1% 1% Grand Total 100% 100% 100% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 When analyzing the expenditure of beneficiary households, food, health care and children’s school fees (Charts 10 and 11) are the most cited items. These justify this type of emergency intervention by World Vision in these vulnerable communities. Making more public schools available to relieve the expenses related to sending children to school is more than essential. We understand the low number of beneficiaries with a good level of education among respondents. Moreover, these vulnerable households do not have the economic means to support school costs. USAID EFSP 32 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Chart 11 : Distribution of respondents by commune according to the first item of annual household expenditure Chart 12 : Distribution of respondents by commune according to the second annual household expenditure item Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 The beneficiaries’ monthly income brackets are very low, i.e. largely less than 5,000 gourdes (approximately USD 50) in the 2 communes (Chart 12). This statistic puts almost all beneficiaries in the category of extreme poor people, living with less than 1.90 USD/day. Chart 13 : Distribution of respondents by commune according to the monthly household income bracket Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 90% 93% 7% 6% Anse-à-galets Pointe-à- Raquette Other Clothes Health care School Food 10% 7% 65% 63% 17% 24% 5% 4% 1% 2% Anse-à-galets Pointe-à- Raquette Other types of work equipment No other expenses House rent Toiletries and laundry Garden Clothes Health Care School 79.7% 70.4% 17.5% 26.9% Anse-à-galetsPointe-à-Raquette More than 30000 25001-30000 20001-25000 15001-20000 10001-15000 5000-10000 Less than 5000 USAID EFSP 33 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 5.2 Analysis of Evaluative Questions In this section, the evaluation questions of the terms of reference are analyzed one by one by crossing data collected in the field from the various sources concerned by the study. 5.2.1 Relevance For this criterion, the summary of answers given to the questions considers the way in which the project was able to consider beneficiary priorities in the implementation of activities. Question 1: What are the views of interested parties on the nature and quality of project implementation? WV through SIMAST (a data management tool linked to social protection activities) identified the vulnerability issues and the poorest families in order to help make a more efficient resource allocation to the neediest and increase aid effectiveness towards poverty reduction. The relevance of project’s actions was widely affirmed by most of our interlocutors – community leaders, implementing partners (ASCP, mother-leaders, and vendors) and beneficiaries. Interviewees stressed that the island faces structural and growing food insecurity just like the rest of the country. The repeated and increasingly frequent cycles of drought bring, as a corollary, the degradation of natural resources and production potential. Access to basic foodstuffs by populations whose livelihood assets seem quite limited and fragile is the main issue when it comes to food and nutritional insecurity. Most of them are convinced that the program’s targeting strategy has actually made it possible to identify vulnerable families (both those whose survival is threatened and those who lack the financial means) and help them to provide to their urgent food needs during critical periods of the year (lean season) and also to make investments in productive activities in order to strengthen their resilience. “Even before designing the project, WV contacted us (CASEC) to understand the food insecurity situation that prevails in our communities, to see and define with us some criteria of vulnerability adapted to social conditions” Toman Jean Philippe, CASEC from Petite Source, Anse-à-Galets In addition, the conjunction of “voucher” interventions with nutritional activities (food supplements, monitoring and care of malnourished children, awareness sessions and cooking demonstration) is well perceived and appreciated by communities because it strengthens food diversification to cover the nutritional needs of target groups such as pregnant and nursing women and children from 6 to 59 months (physical and psychological development of newborns). Savings and credit activities were highly valued and appreciated since, according to the groups, they are a significant means of increasing their resilience in the face of future shocks by strengthening their ability to invest in alternative income activities. And, by the time of future shocks, when many affected families will have to sell their assets, reduce their food consumption or withdraw their children from school to cope with the shock, members of savings groups will have cashed in their earnings to access resources – hence the expected future impact: USAID EFSP 34 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation “Before, we did not know that saving only 25 gourdes could make such a difference” Testimony of a savings group member during the FGDs organized by the evaluation team in Pointe-à- Raquettes. Focus group with members of a community savings and credit group The quantitative surveys confirm the beneficiaries ’ satisfaction level in relation to the quality of actions developed in this project in the 2 communes. In Anse-à-Galets or Pointe-à-Raquettes almost 100% of surveyed beneficiaries are either satisfied or very satisfied with the activity components developed in the project ( Chart 13). The reasons for this unanimous satisfaction revolve around the idea that the food aid provided by the project came at the right time when these vulnerable households were deprived of the means to meet their needs. Financial savings mechanisms are also cited among the main reasons for satisfaction. Chart 14 : Distribution of respondents by commune according to their level of satisfaction with project activities USAID EFSP 35 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 Question 2: Were relevant government officials involved ? In the key interview with the project manager, he admitted that there was an ongoing and close relationship with MAST apart from the use of the SIMAST database during the project targeting process. As for the MSPP, the project’s relationship with this government entity was more regular, especially through the ASCPs and mobile clinics that took place in a few communities. In the communal sections, the project relied heavily on local authorities (CASEC, ASEC) according to the manager. Thi s aspect was confirmed in the key interviews with local authorities, because the latter mentioned the importance WV gave to the strengthening of their capacities in order to guarantee the sustainability of project actions and activities, except for the downtown localities, the 3 rd and 5 th sections of Pointe-à-Raquettes where they stressed they did not receive the necessary support from the program to better understand and serve their communities. However, the coordination with local authorities was strong and consistent throughout the program, the CASECs declared having been consulted and made aware of all the implementation stages during the scoping, design and implementation phases. This, in a way, helped disseminate project-related information to their communities (especially selection criteria), to report problems to the WV team during consultation and scoping meetings (for example the need to revalue vouchers according to the local variation of the exchange rate and the price of food products on the local market). Question 3: Has the project implementation strategy been adjusted to adapt to the realities on the ground? If yes, how? The project implementation strategy was indeed adjusted to suit the realities on the ground. Initially , it was planned to choose 3,000 beneficiaries from the SIMAST database of the Ministry of Social Affairs and Labor. But this already old system posed some difficulties for us to find vulnerable households, mainly because of the migratory flow. Many names of people in this database have gone to the Dominican Republic or Chile. The project had to use a participatory targeting protocol taking into account the presence of children under 5 in the household – the presence of pregnant or nursing women – households with an elderly person 40% 37% 60% 62% Anse-à-galets Pointe-à-Raquette Very satisfied Satisfied More or less satisfied Totally insatisfied USAID EFSP 36 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation as their head – households with one (many) people living with a disability – households with young mothers – households not receiving any assistance from other institutions. The lists were therefore validated with the local authorities . Second, according to the project manager, all the vendors could sell anywhere and to any number of beneficiaries. After noticing a monopolistic tendency with one seller, especially downtown Anse-à-Galets, a ceiling of 200 vouchers was set by the project as the maximum quantity of sales for the vendor. Third, in the supervision of voucher exchanges, M&E prompted the project to put a monitor on the transaction table to ensure that beneficiaries receive the exact value of the voucher in food items. Question 4: Did the project meet the needs and priorities of the most vulnerable people and the targeted pregnant and nursing women? The WV team carried out formative research (market and needs analysis) to identify the needs and priorities of vulnerable communities, assess the local food insecurity situation, analyze whether local markets are functional and capable of absorbing a sharp increase in demand and justify the strategies and response mechanisms for food vouchers. A participatory community targeting approach was used where a more detailed database, namely SIMAST, was used to identify and register households at home. This speeded up the registration process. Community Identification Committees (CICs) were set up and worked in conjunction with local leaders 9 to approve or check the status of selected households. In fact, the local authorities consulted confirmed their integration during the process of identifying community needs and validating the vulnerability criteria specific to the intervention areas. WV then conducted a household survey to verify their eligibility, triangulate the information received and check 20% of registered households to determine their current status and priority was given to households meeting at least three criteria according to government approaches for social protection and lessons learned from the Kore Lavi program . The final eligible households were entered into the Last Mile Mobile Solution (LMMS) system to generate electronic food stamps and distribute project identity cards where each household was given a unique identification number. It is essential to note that the SIMAST database is not updated (last update in 2016) while demographic data in La Gonâve change very quickly, mainly because of the migratory flow, which led the WV team to cross data with the assessments of local authorities. This is one of the biggest lessons learned from the targeting strategy: involve communities because they have more information about their vulnerabilities. To put a score in relation to the achievement of this criterion as a summary of the answers to the various questions, we consider this satisfaction scale for each criterion: 1 = Not at all satisfied 2 = Not satisfied 3 = Moderately satisfied 4 = Satisfied 5 = Very satisfied . Thus, as far as relevance is concerned, we can say the project got 5: Very satisfied level for this criterion. 9 Mayors, CASECs, Local Leaders, Agricultural Office (BAC) and Civil Protection managers USAID EFSP 37 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 5.2.2 Profitability Question 1: Did the project have adequate and appropriate resources (human, financial and capital) for its implementation? If not, how was it addressed? The project manager was affirmative that there were enough financial resources to implement the project. However, the concern was more with human resources for the nutritional component. The slow recruitment process meant that there were 2 nursing positions that were not filled. The nutrition level activities were not developed on time. In terms of logistics, there was also some slowdown. For example, there was significant delay in the delivery of checks to vendors. Hence the influence of internal systems/procedures related to finance and operations on the implementation of activities. The project had to rely on other nurses from other WV projects in La Gonâve, because some staff were not willing to go to work in La Gonâve due to the difficult context, and it was particularly impossible to find good human resources in the localities themselves. This delayed the implementation and turned out to be a challenge throughout the project despite the fact that WV offered an additional incentive to attract technicians. This is a practice that bore fruit but is not sustainable. Moreover, WV also had recourse to other sectoral departments in other regions to support project activities in La Gonâve. Question 2: Were quality control and accountability measures in place and consistently applied during the review, approval, funding disbursement, monitoring, and reporting phases? The manager’s key interview confirms there were indeed quality control (due diligence) and accountability measures that were systematically applied during the review, approval, funding disbursement (SUN system), beneficiary monitoring (LMMS) and reporting. These systems provide WV with reasonable control processes ensuring compliance with donors. The monitoring and evaluation system provided the right framework to capture and monitor feedback from communities and individuals. Budget forecasts are made every month for a 30- day activity period and disbursement requests are thus sent to the Finance department. The verification process took longer than expected. The longer the payment processes, the longer the implementation is delayed, as suppliers also need time to replenish for the next distribution cycle. Question 3: Do recipient comments indicate widespread instances where funds (vouchers) were taxed or stolen, or where receiving a voucher represented a protection risk? The project manager was clear that there was no suspicion of voucher fraud with project staff. Unanimously, the key interviews carried out with the Local Authorities confirm they have not heard or experienced any situation of fraud or theft of funds intended for beneficiaries. In one of the nursing women’s focus groups, one participant bluntly stated “We did not have to pay people to participate”. USAID EFSP 38 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Given the delays in the nutritional aspect, the slow delivery of vendors’ checks, we can say the project got 2: not satisfied for this criterion. 5.2.3 Effectiveness The analysis of the program’s effectiveness helped see to what extent the objectives of the program were achieved. Question 1: To what extent were the objectives achieved? To respond to this evaluation question, the impact and effect indicators, as presented in the terms of reference, were analyzed through the quantitative survey carried out among beneficiary households, namely: - The dietary diversity score (HDDS) , - The food consumption score (FCS) , - The hunger scale (HHS) - Percentage of food utilization by type (household consumption, sale, exchange, livestock feed ) - Average number of meals per day in the household according to age, - Reduced coping strategies index (ISSr/rCSI) - Prevalence of global acute malnutrition (GAM) - Percentage of targeted households using and/or benefiting from community assets created/rehabilitated - Percentage of targeted households declaring an increase in the amount saved thanks to the program - Percentage of households adopting best practices in health and nutrition Overall, based on the data collected from the quantitative survey, for the indicators mentioned above, the results obtained seem a little mixed. Out of 10 indicators presented in the terms of reference, the objective has been reached for 4, namely: - Percentage of households where adults and children eat at least 2 meals a day (adults and children), - Percentage of targeted households using and/or benefiting from community assets created/rehabilitated, - Percentage of targeted households declaring an increase in the amount saved thanks to the program, - Prevalence of global acute malnutrition (GAM) . The target was not reached for the other 6, but the results obtained are progressing considerably towards the expected threshold. These are mainly the following indicators: USAID EFSP 39 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation - Percentage of targeted households with an acceptable food consumption score (FCS) - Prevalence of households with little or no hunger (household hunger scale - HHS) (score 0-1) - Reduced Coping Strategies Index (% of households with a less serious or moderate index). - Percentage of households adopting best practices in health and nutrition - Proportion of households consuming at least 6 food groups during the previous month, - Percentage of food use by type (household consumption) It should be noted that the percentages of progression for Reduced adaptation strategy index (% of households with a less severe or moderate index), Proportion of households consuming at least 6 food groups during the previous month and Percentage of Food use by type (household consumption) are considerable, we could even try to include them among the indicators whose objective has been reached because they are at more than 89% of their target. Table 11 : Key indicators measured and their progress against target Indicators Values at the final evaluation Island of La Gonâve (%) ± CI Calculated values Target % of achieved target Goal Percentage of targeted households with an acceptable food consumption score (FCS ) 46%±7.2% 75% 61.33% Not reached Prevalence of households with little or no hunger (Household Hunger Scale - HHS) (score 0-1) 47.3%±7.3% 70% 67.57% Not reached Proportion of households consuming at least 6 food groups in the previous month 89.5±2.2% 100% 128% Reached Percentage of food use by type (household consumption) 98.9% ±1.6% 100% 141.28% Reached Percentage of households where adults and children eat at least 2 meals per day (adults and children) 95% ±3.1% 70% 135%% Reached Reduced Coping Strategies Index (% of households with a less serious or moderate index) 68.8±6.45% 70% 98.28% Not reached Percentage of targeted households using and/or benefiting from community assets created/rehabilitated. 71.6±3.3% 60% 119% Reached Percentage of targeted households reporting an increase in the amount saved thanks to the program. 87.2±10.5% 60% 145% Reached USAID EFSP 40 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Percentage of households adopting best practices in health and nutrition 31%±6.70% 60% 51.66% Not reached Prevalence of Global acute malnutrition (GAM) 5.8±2.2% 10% 158% Reached a) Household Food Consumption Score (FCS) For this study, the household food consumption score takes into account dietary diversity, frequency of food consumption and the relative nutrient intake of different food groups. It is calculated based on the consumption frequency for the different food groups consumed by a household during the 7 days prior to data collection. For this indicator, the goal is to reduce the percentage of households with low food consumption (0% at the end of the project) or acceptable borderline (10% at the end of the project). By comparing the results obtained for this evaluation against the target threshold at the end of the project, we can say that the goal was not reached because 23.6% of surveyed households have low food consumption and 30.4% have an acceptable borderline consumption. Table 12 : Food consumption score of project beneficiaries by commune Indicator: Food consumption score Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % Poor food consumption (Percentage of households with FCS from 0 to 21) 122 365 33.4% 49 361 13.6% 171 726 23.6% Borderline food consumption (Percentage of households with FCS of 21.5-35) 100 365 27.4% 121 361 33.5% 221 726 30.4% Acceptable food consumption (Percentage of households with SCC> 35) 143 365 39.2% 191 361 52.9% 334 726 46.0% Average food consumption score 31.75 38.13 34.92 Food consumption standard deviation 18.33 15.49 17.26 Confidence interval at 95% of the average [29.87;33.63] [36.53;39.73] [33.67;36.18] Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 Looking at the findings displayed in the table above, the situation seems less critical in the commune of Pointe-à-Raquettes. In fact, 13.6% of surveyed households have low food consumption in Pointe-à-Raquettes while in Anse-à-Galets, they represent 33.4%. b) Hunger Scale Score/Household Hunger Index (HHI/HHS) USAID EFSP 41 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation This indicator is used to provide information on an important dimension of food security, namely access to sufficient food in the household. In fact, this indicator is assessed when the prevalence of households with severe or moderate hunger has been considerably reduced. Regarding the prevalence of households with severe hunger, the target was not reached (reduced to 0%) but the findings are more or less satisfactory by comparing the values of this indicator for the baseline with their value for the final evaluation (21.05% against 6.9%). In the communes, the situation seems slightly improved in Anse-à-Galets. Table 13 : Hunger score of project beneficiaries by department Indicator: Hunger Scale or Household Hunger Score (HHS) Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % No or mild hunger in households (scores 0-1) 162 365 44.4% 181 361 50.1% 343 726 47.3% Moderate household hunger (scores 2-3) 180 365 49.3% 153 361 42.4% 333 726 45.9% Severe household hunger (scores 4-6) 23 365 6.3% 27 361 7.5% 50 726 6.9% Average Household Hunger Index 1.72 1.51 1.61 Household Hunger Index standard deviation 1.29 1.42 1.36 Confidence interval at 95% of the average [1.58;1.85] [1.36;1.65] [1.51;1.71] Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 However, with regard to the prevalence of households with moderate hunger, the goal was reached but caution is required in the continuity of activities. Because, compared to the threshold set at the end of the project (55%) for this indicator, the result obtained during the evaluation is 45.9 ± 3.65%. In the communes, the situation seems slightly more favorable in Pointe-à-Raquettes. c) Household Dietary Diversity Score Household dietary diversity for this assessment is calculated from the number of different food groups consumed (nine groups in total) by households in project intervention area over the 30 days or four weeks prior to the survey. At the end of the project, for a diversified food consumption (consumption of 6 or more food groups), the focus was on 100% of project beneficiaries. By comparing the results obtained for this indicator during the final evaluation (89.5%) with the threshold set at the end of the project (100%), the goal was not reached. On the other hand, by comparing this result with the baseline, significant progress was noted (73.9 ± 2.8% compared to 89.5 ± 2.2%). The situation has considerably changed in Pointe -à- Raquettes, no household surveyed has a poor diet. USAID EFSP 42 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Table 14 : Dietary Diversity Score by department Indicator: Dietary Diversity Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num 10 Den 11 % Num Den % Num Den % Low dietary diversity 22 365 6.0% 0 361 0.0% 22 726 3.0% Average dietary diversity 27 365 7.4% 27 361 7.5% 54 726 7.4% High dietary diversity 316 365 86.6% 334 361 92.5% 650 726 89.5% Average Dietary Diversity Score 7.54 7.55 7.55 Standard Dietary Diversity Deviation 1.97 1.31 1.67 Confidence Interval at 95% of the Average [7.34;7.74] [7.41;7.68] [7.42;7.67] Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 d) Percentage of food use by type (household consumption, sale, exchange, livestock feed) The largest number of respondents to the survey in the commune of Anse-à-Galets declare that agricultural production is used for household consumption, i.e. 98%, 21.9% of them declare to have sold their production to increase household income and 2.7% of these households use this production for livestock feed. In Pointe -à-Raquettes, agricultural production is used for household consumption (99%) and sale (57.9%) and only 2 households declare having exchanged their production for other products. In general, the agricultural production of households on the island of La Gonâve is used fo r household consumption (98%) and for sale (39.8%). Table 15 : Percentage of food use by type of project beneficiaries for the island of La Gonâve and the communes Indicator: Use of production Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % Household consumption 360 365 98.6% 358 361 99.2% 718 726 98% Sale 80 365 21.9% 209 361 57.9% 289 726 39.8% Exchange 5 365 1.4% 2 361 0.6% 7 726 1.0% Livestock feed 10 365 2.7% 4 361 1.1% 14 726 1.9% e) Percentage of households where adults and children eat at least 2 meals a day (adults and children) The average number of meals per day in a household is considered an indicator of the level of food security or insecurity and this, in the context of this study, is set at 2 or more to interpret this indicator. As a target, at the end of the project, a percentage of 70% was set. The table 10 Num : Numerator 11 Den : Denominator USAID EFSP 43 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation below, made from the data collected from the quantitative survey, indicates a percentage of 95% for this indicator over the entire project intervention area, which proves the significant achievement of this goal. No significant difference was observed for this indicator in the communes. Table 16 : Percentage of households where adults and children eat at least 2 meals a day (adults and children) Indicator: Percentage of households where adults and children eat at least 2 meals a day (adults and children) Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % Households where adults and children eat at least 2 meals a day 349 365 95.6% 341 361 94.5% 690 726 95.0% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 f) Reduced Coping Strategies Index (ISSr/rCSI) For this index, the higher the score, the more food insecure the household in question is. This means that it uses negative strategies more frequently and/or that the strategies used are more severe compared to a household whose score is lower. The maximum score for the index is theoretically 56. So, as part of this project, the goal is to see a high percentage of households used negative strategies less often to meet their needs. By the end of the project, we hoped to reach 70% or more beneficiaries using positive strategies. Table 17 : Reduced Coping Strategies Index (ISSr/rCSI) Reduced Coping Strategies Index (ISSr/rCSI) Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Average 16.22 11.29 13.77 Standard Deviation 13.62 9.20 11.89 Confidence Interval of the Average [14.82 ; 17.62] [10.34 ; 12.23] [12.90 ; 14.63] Median 16 8 8 Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 Following the findings from the quantitative survey, the goal was not achieved because, if we refer to the threshold used to interpret this index, 68% of surveyed households in the 0-18 class now enjoy a less serious (0-4) or moderate (4- 18) situation. Alm ost 30% of surveyed households use negative strategies throughout the intervention area. More beneficiary households use negative strategies in Anse-à-Galets compared to those in Pointe-à-Raquettes. Chart 15 : Layer of Reduced Coping Strategies Index (ISSr/rCSI) USAID EFSP 44 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 g) Percentage of targeted households using and/or benefiting from community assets created/rehabilitated During the implementation of the project, activities were carried out directly (training, orientation session, etc.) with beneficiaries and others (soil conservation, construction of dikes, etc.) indirectly within their community for more support. This indicator helps see, from the community assets created/rehabilitated, the households that have used or benefited from these assets. For this indicator, the goal was reached because, compared to the expected threshold at the end of the project (60%), 71.6% of surveyed be neficiaries in the quantitative survey over the entire project intervention area use or benefit from community assets created or rehabilitated in the community. Table 18 : Percentage of targeted households using and/or benefiting from community assets created/rehabilitated Indicators : Percentage of targeted households using and/or benefiting from community assets created/rehabilitated Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % Targeted households using and/or benefiting from community assets created/rehabilitated 253 365 69.3% 267 361 74.0% 520 726 71.6% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 In the communes, the beneficiaries located in Pointe-à-Raquettes (74.0%) are slightly more represented than those in Anse-à-Galets (69.3%) among surveyed households who declared having used or benefited from the assets created/rehabilitated in the community. h) Percentage of targeted households reporting an increase in the amount saved through the program 44.4% 18.0% 31.3% 29.0% 72.3% 50.6% 26.6% 9.7% 18.2% Anse-à-Galets Pointe-à-Raquette Ile de La Gonâve Severe(>=19) moderate(4-18) Less severe(4<) USAID EFSP 45 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation To help beneficiaries from the project intervention area better withstand shocks and difficulties, the project encouraged them to integrate the inclusive savings and credit associations. The project sought to diversify the income of beneficiaries so that they can better respond to their priority needs or use fewer negative strategies. At the end of the project, the expected percentage of target households reporting an increase in the amount saved through the program was 60%. By observing the results obtained for this indicator from the quantitative survey, 87.2% of surveyed households in the project intervention area declared having seen an increase in the amount saved. Thus, it can be clearly stated that the goal was significantly achieved . Table 19 : Percentage of targeted households reporting an increase in the amount saved through the program Indicators: Percentage of targeted households reporting an increase in the amount saved thanks to the program. Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % Targeted households reporting an increase in the amount saved through the program 28 32 87.5% 6 7 85.7% 34 39 87.2% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 i) Percentage of households adopting best practices in health and nutrition In relation to the nutritional component developed in the project, it was essential to consider the extent to which households adopted one or another of these practices, namely when there is a pregnant woman present, taking into account at least one of the following aspects: • Iron and folic acid supplementation • Calcium supplementation • Multiple micronutrient supplementation • Direct food aid from fortified/specialized food products (i.e. CSB +, Super Cereal Plus, RUTF, RUTF, etc.) As for the presence of children under 5 in the household, one or more of these aspects can be considered: • Immediate, exclusive and continuous breastfeeding • Appropriate, adequate and safe complementary foods from 6 to 24 months • Vitamin A supplementation in the last 6 months • Zinc supplementation during episodes of diarrhea • Multiple micronutrient powder supplementation (MNP) • Treatment of severe acute malnutrition • Treatment of moderate acute malnutrition • Direct food aid from fortified/specialized food products (i.e. CSB +, Super Cereal Plus, RUTF, ASPE, etc.) These practices were noticed more often in the commune of Anse- à-Galets in comparison with the commune of Pointe-à-Raquettes ( Table 19). USAID EFSP 46 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Table 20 : Percentage of households adopting best practices in health and nutrition Indicators : Percentage of households adopting best practices in health and nutrition Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % Households adopting best practices in health and nutrition 160 365 44% 63 361 17% 223 726 31% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 j) Prevalence of Global Acute Malnutrition (GAM) To measure and assess the nutritional status of children, data on their height, weight and age were collected so that anthropometric indices such as height-for-age, weight-for-height and weight-for-age could be calculated. In this study, the main emphasis is on low weight -for-height or wasting, which is considered a measure of acute undernutrition and the consequence of inadequate nutrition during the period immediately before investigation. It measures body mass in relation to height or length and describes the current nutritional status. Children with a weight-for-height Z-score below two or more standard deviations (-2SD) from the median of the reference population are considered to be thin (wasted) or acutely malnourished. The expected outcome at the end of the project is that less than 10% of children in beneficiary households suffer from acute malnutrition. According to the study’s findings, throughout the project intervention area, among surveyed children aged 0 to 59, 5.8% of them, or 25 out of 430, suffer from acute malnutrition. In relation to the expected outcome, we can say the goal was reached for this indicator. In terms of gender, the results seem more satisfactory for boys (5.1%) than for girls (6.7%). In the communes, children growing up in Pointe -à-Raquettes (6.4%) suffer more from acute malnutrition than those in the commune of Anse-à-Galets (5.1%). For girls living near Pointe- à-Raquettes, special attention should be paid to adopting best practices in health and nutrition with parents, especially mothers. Table 21 : Prevalence of acute malnutrition by commune Indicator: Prevalence of acute malnutrition in children by department (6-59 months) Anse-à-Galets Pointe-à-Raquettes Island of La Gonâve Num Den % Num Den % Num Den % % of male children under five (0-59 months) with a weight-for-height Z score less than - 2 standard deviation and/or with edema 6 106 5.7% 6 129 4.7% 12 235 5.1% % of female children under five (0-59 months) with a weight-for-height Z score less than - 2 standard deviation and/or with edema 4 90 4.4% 9 105 8.6% 13 195 6.7% Percentage of children with a weight-for- height Z score below 2 standard deviation and/or with edema less than five (0-59 months) 10 196 5.1% 15 234 6.4% 25 430 5.8% Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 USAID EFSP 47 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation The data from the quantitative survey therefore helped establish the values of these key project indicators in this final evaluation. We have seen the progress of key indicators compared to their target. Only 4 did not reach 100% of their target! It is interesting to note that all of them reached at least 50% of their target. Overall, this implies that many indicators reached at least 50% of their target. In the impact analysis section, the values for key indicators are compared with those found in the project baseline. Question 2: What were the main factors that affected (or didn’t affect) the objective ’s achievement? Certain external factors affected the achievement of all of the project’s objectives. First and foremost the COVID- 19 context that, according to the project manager, slowed down activities especially when government measures limited crowd gatherings. We understand then that this impacted the gathering of people for distributions. Secondly, socio-political instability often causing road blockages. Even when, on the island of La Gonâve, there was very little political turbulence, project vendors purchase from wholesalers located in Arcahaie, Saint-Marc or even Port-au-Prince. They are still impacted by the difficult macroeconomic situation of the country. Question 3: Did the monitoring and evaluation system provide quality, appropriate and reliable information when measuring planned indicators? The key interview with the project manager was affirmative with regard to this question in the sense that the Monitoring and Evaluation system had monitoring tools on the distribution sites (Post-Distribution Monitoring) which helped identify any temporary dissatisfaction. The project monitoring and evaluation manager explained that to ensure data quality for the measurement of indicators, training sessions were organized for the project staff who were more involved in collecting data related to output indicators. As for effect and impact indicators, World Vision investigators were well trained in using PDM survey forms. Question 4: How effective was the project model in terms of design, adequacy, management and accountability? The distribution of beneficiaries in the communities within the 11 communal sections was raised by the manager as a problem in relation to this question. He also stressed that he arrived after the start of the project. Some very vulnerable communities did not have enough beneficiaries out of the 5,770 households considered. Question 5: How effective was the project in terms of program delivery (coordination, cooperation, efficiency, standardization)? The monitoring manager drew the connection with the measured output indicators to confirm that the project managed to achieve the expected outcomes despite the various constraints (country lockdown (“peyi lòk”), COVID-19, other political unrest, etc.). This indeed shows some efficiency in the execution of actions in this difficult global environment with difficult health and political risks. USAID EFSP 48 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation In terms of coordination, deadlines were generally respected apart from the nutritional aspect that had delays due to the absence of a nurse in the staff. The unsuccessful recruitment process was influencing the coordination of nutrition activities. Question 6 : Were humanitarian standards respected and humanitarian principles followed? (Sphere, HAP, Codes of conduct)? Efforts were made within the program team to comply with SPHERE standards. The key interview with the manager revealed that the proximity between vendors and beneficiaries was taken into account to keep the latter from walking several kilometers. When distributing vouchers, the field team made sure beneficiaries did not line up in the sun. There were facilities at the distribution centers to avoid this situation. Finally, in distribution centers, priority was given to pregnant women, people with reduced mobility and the elderly. The statistics from the quantitative beneficiary survey confirm the efforts mentioned in the key interview with the project manager. Indeed, 100% of beneficiaries in the commune of Anse-à- Galets indicate their dignity was not affected in the distribution process and 84% said the same in Pointe-à-Raquettes (Chart 14). As for the time to access a vendor who is part of the project, 68% of Anse-à-Galets beneficiaries said they needed less than 45 minutes and 41% of those in Pointe-à-Raquettes indicated the same amount of time (Chart 15). We noticed that more beneficiaries in Pointe-à-Raquettes needed more time to find a vendor to redeem their voucher. Chart 16 : Distribution of beneficiaries according to whether their dignity was affected or not by commune Chart 17 : Distribution of beneficiaries according to their access time to a project vendor Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021 Question 7: What measures were taken to identify and reduce possible negative effects? Local market prices were regularly monitored with the aim of detecting possible negative effects from the project. According to the project manager, it was found, during the payment of 100% 84% 0% 16% Anse-à-galets Pointe-à-Raquette No Yes 68% 41% 18% 47% 11% 10% 3% 2% More than 2h15 mns 1h31mns- 2h15mns 45 mns- 1h30mns Less than 45 mns USAID EFSP 49 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation redemptions that there was some decline in sales from other vendors who are not from the market. This made these vendors reluctant to give the true price of their products to WV collection agents who showed up with their tablet to do price monitoring. To correct this, the project team avoided using this form of formal price collection. Reliable vendors were also identified to be contacted by phone for this information. Based on the previous analysis of project indicators, one would therefore be tempted to assign a score of 4 = Satisfied on the previously established scale to determine the extent to which the OECD criteria were fulfilled by the project. 5.2.4 Impact Question 1: Did the project reach the expected number of beneficiaries and geographic coverage? Yes, the project reached the number of beneficiaries as well as the planned geographical coverage. This information was confirmed in the interviews with the manager and the monitoring and evaluation manager. However, in the first distribution cycle that began in May because of COVID19, the 5,770 beneficiaries were not reached at the same time. The situation was subsequently addressed with the application of barrier measures. Question 2: To what extent has the project contributed to reducing the vulnerability levels of beneficiaries? The project’s interventions related to living conditions and income-generating activities promoted by community savings and credit structures helped reduce the vulnerability levels of beneficiaries. The project created 15 of these structures and reinforced 35. This form of financial tool coming from solidarity economy carries the features having the most positive impact on the lives of beneficiaries. According to mothers-leaders, the project helped significantly reduce malnutrition cases in their community. Food vouchers saved several families some money. Question 3: What are the unintended positive and negative impacts of project implementation? In relation to this question, one positive impact is this citizen engagement which was noticed with the project’s participatory approach involving beneficiaries in committees for the sustainability of assets created such as stone lines, dikes, distribution committees in the communities as well as Local Authorities. This has promoted these government officials before the population. There is now some trust between these two entities, according to the project manager. Local Authorities praise the positive impact of vouchers on the poorest families. At the same time, they indicate the perverse effects of vouchers on local commerce: they undermine the sale of small traders and planters. The distribution agents made the same observation. There was this unintended negative impact characterized by this drop in sales previously mentioned during USAID EFSP 50 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation the payment of redemptions and the monopolistic trend that was created downtown Anse-à- Galets with this vendor who was able to attract most beneficiaries through his marketing actions. The project was able to address this by setting a ceiling in the sale of vouchers by vendors. We also analyzed the impact through the indicators presented in the ToRs. By comparing the values for these at the start of the project (in the baseline) and those in this final evaluation, the difference test shows there is no change for 4 indicators in Anse-à-Galets and 2 in Pointe-à- Raquettes (Table 22) . Table 22 : Comparison of Baseline and Final Evaluation Results Indicators Baseline Values Final Evaluation Values Difference Test, in EL Anse-à-Galets (%) ± CI Pointe-à- Raquettes (%) ± CI Anse-à-Galets (%) ± CI Pointe-à- Raquettes (%) ± CI Anse-à- Galets Pointe-à- Raquettes Percentage of targeted households with an acceptable food consumption score (FCS) 56.1 ± 9.0% 40.3 ± 8.9% 39.2 ± 10% 52.9 ± 5.2% p= 0.99>0.0 5, no difference p= 0.000<0. 05, there is a difference Prevalence of households with little or no hunger (Household Hunger Scale - HHS) 23.2 ± 7.7% 7.8 ± 4.9% 44.4 ± 10.2% 50.1 ± 10.3% p= 0.000<0. 05, there is a difference p= 0.00<0.0 5, there is a difference Proportion of households consuming at least 6 food groups in the previous month 82.6 ± 6.9% 65.2 ± 8.7% 86.6 ± 7.0% 92.5 ± 5.4% p= 0.06>0.0 5, no difference p= 0.00<0.0 5, there is a difference Percentage of food use by type (household consumption ) 91.8 ± 1.8% 98.3 ± 2.4% 98.6 ± 2.4% 99.2 ± 1.8% p= 0.006<0. 05, there is a difference p = 0.129>0 .05, no difference Percentage of households where adults and children eat at least 2 meals a day (adults and children) 72.3 ± 8.1% 62.1 ± 8.8% 95.6 ± 4.2% 94.5 ± 4.7% p= 0.000<0. 05, there is a difference p= 0.000<0. 05, there is a difference Percentage of targeted households using and/or benefiting from community assets created/rehabilitated. 0% 0% 69.3 ± 9.4% 74 ± 9.1% Obvious difference Obvious difference Reduced coping strategy index (% of households with a less severe or moderate index) 59.8 ± 8.9% 65.2 ± 8.7% 55.6% ± 10.2% 82 ± 8% p= 0.88>0.0 5, no difference p = 0.000<0. 05, there is a difference Percentage of targeted households reporting an increase in the amount saved thanks to the program. 0% 0% 87.5 ± 23%% 85.7 ± 40% Obvious difference Obvious difference Percentage of households adopting 0% 0% 43.8± 10.2% 17.5± 7.9% Obvious difference Obvious difference USAID EFSP 51 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation best practices in health and nutrition Prevalence of global acute malnutrition (GAM) 6% (West) 5.1 ± 6.2% 6.4 ± 6.2% p= 0.69>0.0 5, no difference p= 0.40>0.0 5, no difference The indicators that showed no change in Anse-à-Galets are: • Percentage of targeted households with an acceptable food consumption score (FCS), • Proportion of households consuming at least 6 food groups during the previous month • Reduced coping strategy index (% of households with a less serious or moderate index) • Prevalence of global acute malnutrition (GAM) While those that show no change in Pointe-à-Raquettes: • Percentage of food use by type (household consumption) • Prevalence of global acute malnutrition (GAM) Question 4: How satisfied are the communities with the intervention? Overall, data from the quantitative survey show that the beneficiaries are almost 100% satisfied or very satisfied with project activities (Chart 18). Question 5: Did the program require more time from the women than from men? The program, in fact, required more time from women than from men according to the manager’s analysis. This is explained by the fact that alongside unconditional or conditional distribution activities, specific actions were developed in the nutritional component for pregnant and nursing women. For example, we can mention actions such as mothers’ clubs and mother-leaders. In addition, in rural communities in La Gonâve, women are more available at home while men go work in the fields. Question 6: What gender issues were addressed ? The project made sure, in the committees created in the communities, to encourage women to hold decision- making positions. Awareness did not stop with a female presence in the committees but first and foremost their participation . Question 7: Did the voucher project affect the market and context in any way (did the voucher assistance have an impact on inflation)? The increase in food prices on the island of Gonâve responds to factors external to the food voucher program implemented by WV. For example, on the international market, according to the bulletin presented by the CNSA, in February 2020 12 , there was a relatively large increase in the prices of rice and wheat over the last 4 to 6 months, 11% and 19% respectively, which 12 https://fscluster.org/sites/default/files/documents/foodbasket_fevrier_2020_final_.pdf USAID EFSP 52 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation had negative effects (caused an increase) on local prices for these products imported into Haiti and La Gonâve. Besides this, the food basket in Haiti increased in annual basis by 40% and 14% in a monthly basis in November 2019 because of the deterioration of the exchange rate, the socio- political unrest of October and November 2019 (“Peyi lòk”), and, in early 2020, the negative effects of the Covid-19 pandemic where the closure of local and international borders greatly affected food availability and access (restrictions of movement) (more details in section 5.3 Analysis of the project’s effects on local markets) . Question 8: Did the voucher aid affect food availability on the markets? How did the voucher aid affect local trade? Interviewed vendors explained they had no shortage during the implementation of the program and that local markets continued to operate and procure from the usual suppliers in Carriès, Port-au-Prince and Arcahaie. However, community leaders said they observed a decrease in the number of vendors who are not part of the project, following the reduction of clients mainly due to the decline in the communities’ purchasing power and the new demand created in the market by the project and focused on specific vendors. Hence the need for WV to carry out a more in-depth study on the vendors’ sales volume and the structure of the existing commercial dynamics on the island (Reference in section 5.3 Analysis of the effects of the project on the local markets). By considering the answers given to the various questions and the difference test carried out on the indicators at the start and at the end of the project (6 indicators out of 10 showed there was an impact of the project’s actions), we can deduce the impact criterion got the following score: satisfied. 5.2.5 Sustainability Question 1: To what extent will project benefits continue after donor funding ends? Are positive effects sustainable? Community organizations and individuals should also be aware, from the outset, of their post- program roles and responsibilities. The savings and credit activities were identified as the flagship activity that has the most potential for continuity in communities beyond the WV funding, because communities found financial interests and mutual aid (monetary contribution) in those. In addition, the agents from the savings groups claim to be able to continue to support and create new groups within the communities even if they lack didactic materials promoting a good transmission of the knowledge acquired from the program . The watershed protection activities are replicable because producers see the need to protect their agricultural production and productivity and working techniques have been assimilated by farmers. They are already claiming to suffer even more severe episodes of drought or hurricanes every year. However, they emphasized that no local structure was hired to ensure the continuity of soil conservation activities through the reproduction of training (training of trainers ). For the health care and supervision provided by ASCPs, it is not clear that these activities will continue in the communities with the same rigor and commitment shown during the implementation of the project. The ASCPs, by their proximity to the communities, say they want to continue serving. However, with the absence of MSPP and logistical and financial USAID EFSP 53 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation means beyond project grants, they deem impossible to continue assisting pregnant women, nursing women and children from 0 to 59 months. No running costs or supervision were provided to them by MSPP, they are generally assisted and supported by international NGOs through community health interventions. For their part, entrepreneurial activities with young people and management training require more technical prerequisites, thus limiting any local capacity to continue these cascading trainings. But the young people interviewed said the capacity building received helped them better invest and make their businesses grow. Monetary subsidies cannot cont inue without WV or other similar programs, but the businesses created or strengthened have continued to operate until today. A quarterly monitoring of these companies is recommended in order to assess future impacts and economic benefits in target communities. The mother-leaders emphasized their enthusiasm to keep meeting with pregnant and nursing women, to refer malnourished children and to assist them. But they are aware of the challenges related to care and especially the logistical means to continue assisting vulnerable groups and the limited local purchasing power to continue paying for health care services. Even during the implementation of the program with WV, they had shortages in drugs and teaching materials. Awareness- raising, cooking practices and demonstrations also remain knowledge that is anchored in the communities. They better understand the importance of a balanced diet and required nutritional intake for pregnant and nursing women and children 0 to 59 months. However, due to the ir modest purchasing power, communities continue adopting survival strategies and other eating habits where local nutritious products are sold and replaced by substitutes or other less nutritious and cheaper foods. Question 2 : What sustainability drivers are obvious (local ownership, partnership, transformed relationships, household and family resilience)? ASCPs, mother-leaders and savings groups were strengthened and reactivated within the communities, the training received was very appropriate although there are barriers (highlighted previously) to their full application and development by families. Their actions and services rendered to the communities during the implementation of the project made them better known and accepted. This has transformed pre-existing relationships, and these connections are likely to continue beyond WV funding and contribute to the resilience of target communities. The partnership with the Wesleyan hospital was not formal for this project, no MoU was signed to define the roles and responsibilities of each actor. The communities did not really understand the operational role played by the Wesleyan Hospital and complained that the management of malnutrition cases in children from 0 to 59 months was not effective, even with referral from ASCP or mother-leaders. Services are not free and no care is provided without the payment of a “deposit”. On top of the already limited purchasing power of vulnerable families, pregnant women were unable to complete the 4 prenatal consult ations (ANC) recommended by the WHO for safe motherhood, nor to get the preventive care necessary for maternal and fetal health. In addition, the watershed rehabilitation and conservation works, although useful for communities, will be difficult to continue beyond external funding given there is no responsible local structure that took it over. Local authorities (CASEC, ASEC) are not in a position to USAID EFSP 54 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation finance such work from public funds. It is th erefore necessary for WV to understand that projects respond to specific needs, within set deadlines and within a specific budget. And that a clear plan for sustainability must be developed, shared and understood by the project’s communities, partners and stakeholders from the start . Question 3: To what extent did the project take into account factors that, in experience, have a major influence on sustainability such as economic, ecological, social and cultural aspects? In relation to this question, the manager underlines the living conditions of beneficiaries that were strengthened in the project by the community assets created thanks to the food voucher for work (FVFW) works, the soil conservation works, the dikes helping protect the watersheds. To maintain these structures, sustainability committees were created precisely at the community level to help maintain them or take them over in the event of natural disasters. However, it is necessary to question the way these sustainability committees are funded and their link with local civil protection committees. Community savings and credit groups are also cited as tools to support beneficiary households in carrying out income-generating activities. The manager wanted to mention that group members were able to make loans to stock up with seeds. By analyzing the project’s sustainability aspects, apart from the community group savings and credit activity, despite the project’s effort, the often absent governmental intermediary gives rise to concerns in terms of maintaining certain good dynamics triggered by the project as mentioned above. For this, we believe the project got the score: moderately satisfied for the sustainability criterion. 5.2.6 Linkages, Overlaps and Exit Strategies Question 1: To what extent did the project take advantage of other US Government (USG) and non- USG investments in the same area to facilitate linkages with complementary services, overlaying previous investments and implementing exit strategies to minimize reliance on external support? WV capitalized on lessons learned from USG-funded interventions in the North East and North as well as Kore Lavi 13 also focused on maternal and child health and nutrition interventions for pregnant and nursing women and children under two. Targeting strategies, mother -leaders, the community participatory approach, and the use of SIMAST as a tool for identifying vulnerable families and the use of locally produced foods as part of the food voucher system have helped significantly increase the contribution and impacts of the program. The addition of community savings and credit groups as a complementary component of the project was very successful during Kore Lavi. And these groups have had the most success when they were combined with other activities such as voucher distribution. The community 13 The Kore Lavi project is funded by USAID, and implemented in partnership with the World Food Programme, World Vision, and Action Against Hunger in 24 communes of 5 departments of Haiti. The program also focuses on improving access to locally produced foods among vulnerable households, maternal and child health and nutrition interventions for pregnant and lactating women and children under two years of age” USAID EFSP 55 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation savings and credit groups are considered a sustainable element that continue making an impact long after the project, for many households. Also, WV capitalized on the recommendations laid down by the government in its National Social Protection Policy (PNSP) developed following the positive impact of Kore Lavi (In fact, it is because of the Kore Lavi activities that the various social protection actors became more receptive to the drafting of the policy). Although the recommendations for the World Vision’s initiatives in the North East and in Kore Lavi stressed the need to have a clear sustainability plan from the design phase of the project, a plan that is shared and transmitted to the communities. Local actors were not made aware of the steps for the transmission of the project to communities (handover of the project). The ownership strategy for the sustainability plan included in the project narrative was not fully defined. In addition, the local bodies or institutions that can sustain the actions were not provided with a capacity building plan, for example, the representatives of the local MSPP, a plan to have the ASCPs supported by the supervisory department before the end of the project, the intentional strengthening of local and farmer CBOs on soil conservation practices and techniques, the support and structuring of mother-leaders. The sustainability plan presented by WV should at least include (Rogers and Macias, 2004) 14 : • Decisions on the approach (phase-out, gradual replacement); • Explicit benchmarks for progress and deadlines; • A clear division of responsibilities; • Graduation criteria and the gradual withdrawal of free inputs; • An emphasis on building the capacity of local community and government organizations to gradually take over the management and delivery of maternal and child health services; • And the development of alternative incentive structures (for example, initiatives on agricultural production to create increased resilience among beneficiaries to self-finance maternal and child health and other services. To ensure greater sustainability of livelihoods activities, more emphasis should be placed on private sector participation). Question 2: To what extent has the project aligned and integrated with the host country’s service provision strategy/policy for social protection? As part of the health system reform in Haiti, the Ministry of Public Health and Population (MSPP) adopted the Community Health strategy 15 with the goal of increasing the accessibility of health services to the whole population and strengthening first-level health institutions by integrating ASCP 16 . This is in line with the approach adopted by WV in orbiting health activities around ASCPs as those in charge for the facilitation and provision to households of a set of basic services through rally posts, mothers’ clubs and home visits. In addition to playing an interconnecting role, and working with nurses to refer beneficiaries to other appropriate services. WV strengthened the capacity of ASCPs and integrated them into the local health 14 Beatrice Lorge Rogers and Kathy E. Macías.2004. Program Graduation and Exit Strategies: Title II Program Experiences and Related Research. 15 https://mspp.gouv.ht/site/downloads/Plan%20Directeur%20de%20Sante%202012%202022%20version%20web.pdf 16 Indeed, Haiti has a significant potential of 5,500 community relays among which there are 4,411 ASCPs, i.e. 1 ASCP for 2,720 inhabitants, which is much lower than current MSPP standards: 1 ASCP/2,500 inhabitants in the urban area and 1/1,000 inhabitants in the rural area. USAID EFSP 56 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation system by giving them clear responsibilities and ensuring their connection with the community referral hospital (HCR) under the supervision of a nursing assistant. The project also supports the national risk and disaster management plan (PNGRD) 17 in its “Community participation” component, by supporting local efforts to prepare for and adapt to climate shocks and by coordinating community responses. “Food vouchers” interventions as a mechanism to fight against food and nutritional insecurity and strengthen resilience is already recognized in the National Policy for Social Protection and Promotion (PNPPS) 18 and the Child Protection 19 in force in Haiti through MAST and the Institute for Social Welfare and Research (IBESR). Through this program, WV achieved promotion and social integration by strengthening the capacities of households in situations of poverty and socioeconomic vulnerability, to generate income independently, on the basis of activities of production of goods and services while ensuring for them continuous access to community health care and a variety of foods to meet their nutritional needs. This mostly appears by considering the criteria for selecting target groups such as the elderly, people with reduced mobility and pregnant women. According to the mana ger, the treatment given by the field team such as serving these groups first in the distribution sessions also demonstrates the project’s alignment with this National Policy for Social Protection and Promotion (PNPPS). Moreover, nutritional and awareness-raising activities are in line with the National Nutrition Policy (PNN) in the sense that these initiatives sought to improve the nutritional situation of communities based on a preventive approach to all forms of malnutrition through the promotion of healthy, nutritious and suitable food. After observing the elements put forward to evaluate this project component called “Linkages, Overlaps and Exit Strategies”, project, we can grant the following score: satisfied. To recap, we therefore have the following summary table of satisfaction scores for each criterion: Table 23: Level of satisfaction in terms of meeting criteria Criteria Score Awarded* Relevance 5 Profitability 2 Efficiency 4 Impact 4 Durability 3 Linkages and exit strategies 4 *1 = Not at all satisfied 2 = Not satisfied 3 = Moderately satisfied 4 = Satisfied 5 = Very satisfied 17 https://www.preventionweb.net/files/72907_plannationaldegestiondesrisquesdeds.pdf 18 https://d2s5011zf9ka1j.cloudfront.net/sites/default/files/2020- 07/20200500_HTI_PNPPS_VF.pdf 19 http://www.haiti-now.org/wp-content/uploads/2017/05/Ibesr_Strategie-National-De-protection-de-l-enfant-2015.pdf USAID EFSP 57 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 5.3 Analysis of the program’s effects on local markets and certain interest groups (women and men; youth population; boys and girls, etc. ) The impact study was based on interviews, a review of existing documentation and a price analysis. This section summarizes the findings in relation to the following points:  Evolution of prices for the food vouchers products distributed (rice, corn, beans, flour, pasta-spaghetti or macaroni, oil and fresh products such as vegetables, food and meat). In general, in La Gonâve, the nominal cost of the food basket exceeds the benchmark national average by 14.5%, with major variation peaks exceeding 16% during the months of May, June and December corresponding respectively to the lean period and the end- of-year holidays. That means the La Gonâve population spends more money to get the same quantities of food products in the local markets compared to other shopping centers in the country, since vendors mainly source from other markets in neighboring regions such as Arcahaie/Carriès, Artibonite and the urban centers of Port-au-Prince such as the Titony market, which adds additional costs to the average product prices (WV EFSP, La Gonâve baseline, 2020, page 51). The local authorities and beneficiaries considered the value of vouchers insufficient (suggesting rather between 7,500 and 10,000 gourdes). The project began the first round of redemption in early May 2020 with a voucher value of 4,750 gourdes for a family of 5, which only covers 43% of their food needs and completed the distribution cycles in February 2021 reaching a voucher value of 5,000 gourdes ($50), significantly lower than CNSA forecasts 20 on national outlook for the supply and market of basic food products, from July 2020 to June 2021 (1,922 gourdes per person or 9,610 gourdes for a family of 5). Also, local families have the social practice of sharing food through group cooking (Bitasyon) or donations to non- beneficiary families. However, this pooling suggests that the quantities of food consumed are smaller for individuals making up each household and that the vouchers cover needs for a shorter time. All the surveyed vendors mentioned this increase in product prices on local markets. Indeed, the food basket underwent a significant rise with an annual increase of 40% due to the surge of the exchange rate, the drop in production and a socio-economic situation not conducive to stability and progress. (OCHA, 2020 21 ). Moreover, the harmful effects of restrictive COVID- 19 measures (closure of ports, airports and border crossings) taken locally and internationally in countries such as the United States and the Dominican Republic have had negative repercussions on local economic activities, employment, transfers and imports of certain products such as rice and flour (EFSP La Gonâve, PDM report). WV included vendors in the pricing process for vouchers. However, those in Pointe-à-Raquettes pointed out that the estimates made by the WV team were sometimes below market prices due to additional transportation costs, especially during COVID-19 restriction periods when travel was more expensive than usual. This caused friction with the beneficiaries 20 https://fews.net/sites/default/files/documents/reports/HAÏTI_Perspectives_de_lOffre_et_du_marche_Septembre_2020_Juin_2021_0.pdf 21 https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/hti_hno_2020-fr.pdf USAID EFSP 58 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation regardless of contractual requirements with WV which kept them from reducing the quantity of products to be supplied during the exchanges of beneficiaries’ vouchers. For the M&E manager, monitoring product prices on the local markets and strategies for valuing vouchers remain aspects of the program needing improvements and especially requiring to move from paper vouchers (the issue of traceability of aid) to electronic vouchers that have the advantage of empowering beneficiaries, that help monitor the quantity of food received and the supply from vendors. On site, to address this issue, the project team, on the recommendations of M&E, had to set a “monitor” at the transaction table to ensure beneficiaries received the exact value of the food voucher.  The supply capacities of vendors in terms of quality and sufficient quantity to meet demand. The vendors said they received an average of more than 20 beneficiaries a day during the food voucher exchange schedules, and it took them between 10 to 20 minutes to serve each beneficiary, which, according to several of them, was an overload requiring the use of additional labor during voucher exchange peak times. All the vendors reported ha ving hired at least three additional people and the project’s positive impact helped them significantly increase their portfolio by ensuring a fixed share of local customers. In Pointe-à-Raquettes, observations made by local authorities noticed a decrease in the number of vendors not included in the program, due in particular to the system and nature of the local economy. The vast majority of vendors in this commune buy their goods on credit from Anse- à-Galets wholesalers and have to repay the loan after the sale. Some of these vendors had to give up their activities and wait for the project to close to avoid getting into debt. More in -depth analyzes on sales volumes must be made to really conclude whether it is about creating monopolies. Vendors in the program have all claimed to have met and absorbed the new demand and have not encountered difficulties with on-time supply or transportation except during COVID-19 restrictions when flows slowed down but did not affect trade relations on the island. The beneficiaries also confirmed their satisfaction with the quality of food products received and said they were able to find all the products listed on the vouchers at the vendors’ except sporadic instances where fresh products such as vegetables (especially related to the seasonality of local production) and meats were lacking. Beneficiaries say they continue getting supplies from the usual markets and they walked an average of 30 minutes to redeem their vouchers. So there were no case of relocation of beneficiaries identified which would represent additional destabilization risks for local markets and inflationary effects. The findings from household surveys and follow-up visits to beneficiary communities showed that the vast majority of participants prefer the food voucher modality in the 2 communes (Graph 18). However, cash comes in second in Pointe-à-Raquettes. With the latter modality, they feel empowered to make their own decisions about household needs and, in so doing, ensure the most efficient allocation of resources. However, given that the project’s objective is food diversification and food intake by vulnerable groups in order to combat malnutrition, the evaluation team understands and validates the choice of directed food vouchers. USAID EFSP 59 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Chart 19 : Distribution of EFSP project beneficiaries according to preferred aid modalities by commune Source : Final evaluation of the EFSP project in La Gonâve, World Vision Haiti, August 2021  Impacts on certain interest group s Haiti remains one of the most unequal countries with a Gini coefficient (assessing household income inequality) of 0.676, the highest index in the region. Rural women are more disadvantaged than their male counterparts, and gender differences are greater among poor and vulnerable groups (OCHA, 2021) 22 . In 2019, Haiti ranks 170th out of 189 countries according to the Gender Inequality Index (GII), a ranking reflecting the inequalities and challenges women face. Gender-related socio-economic inequalities constitute one of the structural factors relating to the feminization of poverty and its concentration in rural areas (AlterPresse, 2020) 23 . The WV voucher project is intended to ensure a diversified diet, necessary for the prevention of malnutrition in pregnant and nursing women and children 0 to 59 months. Since markets and communities are complex systems, WV realized and anticipated that food vouchers cannot be used independently, but combined with other components, such as support for income - generating activities, access to prenatal and community health services, awareness and family nutrition education. WV intentionally targeted pregnant and nursing women, people with reduced mobility and single parent families as vulnerable groups to benefit from unconditional food vouchers. In most cases, the women collected and kept the food vouchers received from the program and also decided on the amount, frequency and ratios of food exchanged at the program’s vendors’. This did not provoke any discussion within families as some roles are considered “acceptable” for women at home 24 (e.g., feeding the family, looking after children, household chores and small family businesses selling small-scale agricultural products). However, decisions about the use of cash, family savings, and payment for health care are made by the husband. Hence the need to integrate men through awareness campaigns for a more responsible and inclusive masculinity. 22 https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/hti_hno_2021-fr.pdf 23 https://www.alterpresse.org/spip.php?article26496#.YTErfS1h2L0 24 https://banyanglobal.com/wp- content/uploads/2017/07/USAID-Haiti-Gender-Assessment.pdf 80% 71% 10% 6% 8% 23% Anse-à-galets Pointe-à-Raquette Direct distribution of food products Cash Voucher Money to buy what I prefer Food Voucher USAID EFSP 60 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation During the conditional voucher phase, working hours were selected so as not to harm (Do no harm) the usual activities of women. Moreover, nutritional practices and cooking demonstrations enabled women to understand the importance of a balanced diet within families and limit cases of malnutrition in children from 6 to 59 months . Focus groups participants in Anse-à-Galets mentioned shortcomings in the care of people with reduced mobility within the various program activities. Of course they receive food vouchers and WV, through the ASCPs, provided them with extensive assistance but the project’s operational structures were not fully adapted to their needs. For example, there were no subsidy for medical coverage for the elderly, blind or disabled people, pregnant women, children, and families. Vendor kiosks and some training rooms (ASCP, meetings with the project team, etc.) do not have any infrastructure allowing accessibility for people with reduced mobility. The majority of program vendors are women and reported that WV provided them with training focused on managing marketing and financial skills to strengthen their sales and management capacities, provide good quality food and attract new customers. They also said they are proud to contribute to the humanitarian response and they are viewed favorably by their community. They consider themselves well positioned for other partnerships with other NGOs as they have a better technical understanding of the food voucher system. 6. Conclusion and R ecommendations T he overall objective of this final evaluation is to analyze the level of achievement of the goal, objectives, and outcomes of the EFSP pro ject and how these were done. To achieve this objective and provide answers to the evaluation questions, a dual approach combining quantitative and qualitative tools was used. Thus, a representative sample of 726 beneficiaries (365 in Anse-à-Galets and 361 in Pointe-à-Raquettes) were surveyed and 25 key interviews held with LAs, component managers of the project team, as well as 14 focus groups bringing together project target groups such as pregnant women, nursing women, people with reduced mobility, members of community savings and credit groups and young people. The analysis of the findings of this evaluation showed to what extent the program met or not the OECD criteria and the various findings to be highlighted are: Relevance • The participatory approach prioritized by the project involving communities as well as local authorities in the various implementation stages (targeting of beneficiaries, distribution committees) was a major point of success in the project; this approach valued local authorities in the eyes of the communities and a relationship of trust was established. • The action components implemented in the vulnerable communities of Anse- à-Galets and Pointe-à-Raquettes largely responded to the priorities of beneficiary households. Nevertheless, it needs to be mentioned that voucher value is estimated at less than 50% of the minimum expenditure basket. Profitability USAID EFSP 61 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation • Internal systems negatively influenced the implementation of actions. The unsuccessful recruitment of 2 project staff slowed down the activities of the nutritional component. In addition, the slow arrival of checks for vendors disturbed their operation in terms of supply. Efficiency • Despite the COVID19 context, as well as the “Peyi lok”, the project indicators have progressed to at least 50% of their target. In addition, out of the 10 indicators to be measured in this evaluation, only 1 indicator (percentage of targeted households with an acceptable food consumption score (FCS)) did not increase according to the desired trend despite the intervention of the project. The catastrophic situation in economic terms (rising inflation) of the country during the project implementation period may be one explanation for this. Impact • The project developed many activities involving women, particularly community savings and credit groups that facilitated the creation of income-generating initiatives, alongside the distribution of food vouchers. Sustainability • The takeover by the government of certain dynamics triggered by the project is not guaranteed, such as the continuity of ASCPs’ work alongside communities. ASCPs are often demotivated because of long-term wage arrears or the fact they are not included in the budget of the public treasury. Linkages, overlaps and exit strategies • The project benefited from the investments of other programs in the use of SIMAST for targeting but, above all, with the presence of community savings groups created since Kore Lavi which gave a lasting character to the overall action. A- Points to be improved Data collected showed that many beneficiaries did not use and were not aware of feedback mechanisms (CARM) either. This could be due to the low literacy level of interviewees or the fact that complaint committees were formed by a few influential people in the communities, hence the need to provide several options to communities for feedback mechanisms. Stakeholders are not aware of the program’s sustainability plan; it is essential that WV establish procurement plans from the design of interventions. These plans should, as a first step, identify which program components are appropriate for gradual implementation, and those which can be phased out so that program objectives are met without continuous input. For components that will be gradually replaced, the plan should identify the institution, group or entity that will support each program element. USAID EFSP 62 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation B- Effects on local markets Overall, vendors were able to meet the challenge of meeting the demand stimulated by the project without openly running out of dry and fresh products, except in sporadic cases linked to vegetable seasonality. There were no obvious case of relocation of beneficiary households at the time of exchanges, as the average walking time was 30 minutes, which did not cause any destabilization of local markets and potential inflationary effects. Vendors mentioned having used or needed additional help to serve voucher beneficiaries during the exchanges, so it would be important for WV to establish a reasonable quota by selecting a larger number of vendors to lessen any tendency for monopolies. The decrease in the activities of vendors not included in the program observed by local authorities as a shared consequence 25 of program actions demonstrate the future efforts the WV team need to make to conduct more in-depth studies of existing business structure and dynamics on the island. C- Access to health care for target and vulnerable groups Direct health payments represent the 3rd most important household expenditure item analyzed. Due to reduced local purchasing power and low government funding for health care, all health institutions charge user fees to patients. This lack of affordability is the main reason, for example, that pregnant women say they did not receive all of the 4 ANC recommended for a safe pregnancy or that families of malnourished children have not been able to get adequate care for their children despite having been referred by ASCPs and mother-leaders. ASCPs have played a key role in prevention and care for cases of child malnutrition, assistance to pregnant women, and training on essential primary care issues. However, it is es sential that WV, together with MSPP representatives, define a locally sustainable long-term financing plan for ASCPs (for example through the VSLA groups of pregnant and nursing women who could devote part of the profits from loans to pay for the services offered by ASCPs). There can be no development of food security and sustainable improvement of the long- term nutritional situation without a WASH component, especially in the Haitian context where access to drinking water remains an ongoing challenge affecting the health of children, pregnant and nursing women and the spread of diseases such as cholera, dysentery, typhoid fever, polio, hepatitis A and E. D- Entrepreneurial, savings and credit activities The combination of not only preventive but also developmental approaches is the most suitable in the context of a chronic food and nutritional problematic. Income-generating activities and savings groups have been highly regarded and valued in communities and have the greatest potential to continue beyond WV funding. More in -depth monitoring is recommended to continue assessing their impact on the local and family economy. Recommendations: A- In terms of food needs • For items from the proposed food basket, s trategize to estimate the value of vouchers – for example, based on the MEB estimated by the Cash Working Group. • Specific training on how to keep food fresh is advised. 25 In addition to COVID-19 effects, the decline in local purchasing power and political unrest. USAID EFSP 63 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation • Consider increasing the amount of food provided through the nutrition voucher to cover the consumption of other household members and/or neighbors. • Ensure that non- selected households also benefit from awareness sessions on best nutritional and care practices. • In addition to soil conservation activities, there is a need to adapt the permanent gardens (permagarden) strategies such as crop diversification to meet the challenges encountered during the dry season in order to contribute to long-term local food security in the country, beyond food vouchers. B- In terms of project implementation • Women should be better involved in livelihood and income generation activities to benefit from MCHN services. Better integration of women would provide additional resources or income in the household to purchase food and pay for drugs and health care services. • Provide more technical support to local CBOs, focusing on sustainable practices and services. • First implement a CAP survey (knowledge, skills and practices) in order to determine in detail the eating practices and habits of beneficiary households. • Where possible, maximize engagement between local and national organizations to ensure that knowledge and experience is fully shared and utilized. • Develop and share in advance a comprehensive and clear plan for sustainability as community organizations and individuals must also be aware from the outset of their roles and responsibilities after the program. • To ensure greater sustainability of livelihood activities, more emphasis should be placed on private sector participation. • Integrate technological innovations (T4D) into responses to address the complex challenges related to the resilience of vulnerable households. • In addition to the complementary activities carried out during this program, to ensure full resilience of vulnerable households WV should consider strengthening agricultural production, fisheries, and livestock to build sustainable growth within communities. C- In terms of access to health care • Assess the feasibility of eliminating fees associated with essential health services targeting pregnant and nursing women, and children under five. • Strengthen the capacities of ASCPs to produce detailed reports on the care and assistance provided to communities (traceability). • Work with representatives of the local MSPP to incorporate and activate ASCPs (11) in the local health system for the continuity of project impact (financing of ASCPs). • A WASH component is essential to ensure the proper implementation of nutritional advice and the prevention of diseases linked to the consumption of unsafe water. D- In terms of monitoring and evaluation • Involve communities through a participatory monitoring and evaluation system. • Improve complaint follow-up mechanisms by strengthening responsibilities. USAID EFSP 64 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation • Carry out a survey among vendors in order to determine existing disparities in the markets, and measure the project’s effects on the evolution of their activities (growth of sales, use of profits, etc.) E- In terms of “gender” integration • Gender sensitive indicators should be developed and used according to EFSP guidelines in future programs. These indicators should measure the differences in how men and women participate in or benefit from the program. • Mobilize larger investments in the coordination, collection, analysis and management of gender responsive market data. • Develop and incorporate in the project’s implementation a “responsible masculinity ” strategy by integrating men in activities and training on primary care for pregnant women, nursing women and mothers of children from 0 to 59 months . Summary of findings and recommendations Conclusions Recommendations Food vouchers partially met basic household food needs. Use the local MEB developed by the Cash Working Group to estimate the value of the coupons. Conduct a KSP (knowledge, skills, and practices) survey beforehand to determine the detailed food practices and habits of recipient households. The distribution of food vouchers should be extended over a much longer period to cushion shocks such as COVID-19. Families have a cultural practice of sharing food with non-recipients. Consider increasing the amount of food provided through the nutrition voucher to cover the consumption of other household members and/or neighbors. Working on community resilience is very complex and interventions should aim to build the capacity of people and systems to advance and protect long- term well-being, despite shocks and stresses. WV should consider strengthening agricultural production, fisheries, and livestock in addition to intervening on WASH to build sustainable growth within communities and prevent diseases related to unsafe water consumption. Integrate technological innovations (T4D) into responses to address complex challenges related to the resilience of vulnerable households such as access to climate information, agricultural extension services, and early warning services in conjunction with civil protection. The report showed that many participants neither used nor were aware of the feedback mechanisms (CARMs). Improve complaint tracking mechanisms by strengthening accountability and providing multiple options for communities to channel their complaints in addition to local grievance committees. Stakeholders are not aware of the program's sustainability plan. Involve communities in the monitoring and evaluation system. WV should establish handover plans at the design stage that identify which program components are appropriate for phasing in, and which can be phased out so that program objectives can be achieved without ongoing input. For components that will be phased out, the plan must identify the institution, group, or entity that will take over each program element. The MDPs did not cover important topics that need in-depth analysis throughout the implementation. Gender-sensitive indicators should be developed and used according to EFSP guidelines in future programs. These indicators should measure differences in how men and women participate in or benefit from the program. A decrease in the activities of non- participants vendors was observed by local authorities as a shared consequence of the program's actions. Review the beneficiary/vendor quota to better distribute the project's benefits to local markets, especially in a context where local purchasing power is falling. Carry out constant follow-ups with vendors to determine existing disparities in the markets, and measure the effects of the project on the evolution of their activities (evolution of sales, use of profits, etc.) USAID EFSP 65 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Vulnerable groups cannot afford the costs of health services at local health centers. Subsidize health services for pregnant and lactating women and children under five. Intentionally integrate women into livelihood and income-generating activities so that they can generate additional income at the household level to purchase food and pay for medicines and health care services. Develop and incorporate a "responsible masculinity" strategy into project implementation, not just set quotas for participation of men. ASCPs are not supported by the MSPP to continue providing basic health care services to the most vulnerable. Work with local MSPP representatives for the insertion and activation of ASCPs (11) in the local health system for the continuity of project impacts (ASCP funding). Lack of traceability of the services offered by the ASCPs during the implementation of the program. Strengthen the capacity of ASCPs to produce accurate reports on the care and assistance provided to communities (traceability). Although income-generating activities and savings groups have been highly valued in communities and have the greatest potential to continue beyond WV funding, quantitative impact data must be collected. Increase emphasis on private sector participation. WV should continue to monitor and evaluate the impacts of the businesses created or strengthened on the family and local economy. 7. Bibliographical References 1- OCHA, 2019. Humanitarian Needs Overview Haiti. (Consulted online 07/16/21) https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/2019/0 2/HNO- Haiti_2019- Summary-EN_0.pdf 2- Integrated Food Security Phase Classification (IPC) Haiti, 2019. (Consulted online 07/16/21). http://www.ipcinfo.org/ipc-country- analysis/details- map/en/c/1068538/?iso3=HTI 3- FEWSNET, 2019. Haiti Food security Outlook. (Consulted online 07/17/21) https://fews.net/sites/default/files/documents/reports/HT_OL_02-2019_final_EN.pdf 4- World Vision, Humanitarian Response Info. (Consulted online 07/17/21) https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/assess ments/la-gonave-one-pager.pdf 5- USAID Feed the Future. Population-based survey sample size calculator. (Consulted online) https://www.usaid.gov/documents/1866/population- based-survey-sample- size- calculator 6- Enquête SMART, UNICEF: https://www.unicef.org/haiti/communiqu%C3%A9s-de- presse/les-taux-de-malnutrition-sont -en-hausse-en-ha%C3%AFti-r%C3%A9sultats- pr%C3%A9liminaires Downloaded on September 8 th , 2021, at 9:54 AM. USAID EFSP 66 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation 8. Appendices Appendix I: FCS Indicat or These groups include different types of foods such as: Group No. Food Group Weighting 1 Staple food: Corn, rice, sorghum, other grain, roots and tubers (potatoes, yucca, yam, sweet potatoes, large breadfruit, small breadfruit) and plantain 2 2 Legumes: White beans, black beans, red beans, pinto beans, green beans, nuts, peanuts (and other similar foods) 3 3 Vegetables/Leaves: Lyann panye, spinach, chives, cabbage, pumpkin, tomatoes, onions, broccoli, radishes (and all kinds of similar vegetables) 1 4 Fruits: Mango, papaya, guava, apricot, cantaloupe, pineapple, orange, melon, watermelon, quince, cherries, lemon, grapefruit, avocado, banana, apple, plum, tamarind, strawberry, pear (and all kinds of fruits) 1 5 Meat, poultry and offal: goats, pigs, sheep, cows, horses, chickens, turkeys, guinea fowls, pigeons, liver, kidneys, hearts, intestines, offals, brains, (and all other types of meat) Seafood: Fresh fish, salted fish, salted cod, crabs, shrimp, (and all kinds of seafood) 4 6 Milk and dairy: Cow’s milk, powdered milk, canned milk and batch milk, yogurt (and all other similar products) 4 7 Sugar and honey: White sugar, red sugar, honey (and all other similar products) 0.5 8 Oils and fatty products: Vegetable oil, olive oil, butter, shortening, fat (and all other similar products) 0.5 9 Spices/drinks: Coffee, tea, spices (parsley, thyme, garlic, clove), salt, fish powder, creamer 0 These scores were established based on information or experience from other surveys around the world. For the purposes of this survey, the following thresholds were considered: Poor food consumption: score between 0 and 21 Borderline food consumption: score between 21.5 and 35 Acceptable food consumption: score above 35 USAID EFSP 67 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Appendix II: HHS Indicat or Households will be asked about their experiences of hunger in the last four (4) weeks/30 days preceding the survey. Their answers allow us to classify them into three (3) categories: mild or no hunger, moderate hunger and severe hunger. The first level of the scale is considered an acceptable or normal situation from the food access standpoint. Three (3) frequency answers (Never=0, Seldom or Sometimes=1, Often=2) A score is calculated for each household (summing the three (3) responses), with a minimum possible score of 0 and a maximum possible score of 6. Three (3) categories of hunger are thus defined: a. “No or mild hunger in households” (scores 0- 1) b. “Moderate hunger in households” (scores 2- 3) c. “Severe household hunger” (scores 4- 6) USAID EFSP 68 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Appendix III: Reduced Coping Strategy Index (rCSI) Survival strategies were categorized according to their severity: Category Behavior Weighting Stress Strategy Buying or borrowing food on credit 2 Borrowing money 2 Spending savings 2 Use more casual work than usual 2 Crisis strategy Selling productive assets 3 Removing children from school 3 Reducing health and education expenses 3 Emergency strategy Sending household members to beg 4 Selling the last breeding females 4 Migration of the whole household 4 In the so- called reduced strategy, only five (5) standard elements (standard strategies) are taken into account with their weighting, which tells their severity. No Strategy Weighting 1 Eat cheaper but less preferred food 1 2 Borrow food or money from friends or family 2 3 Reduce portion sizes in meals 1 4 Reduce adult consumption so that children eat more 3 5 Reduce number of meals a day 1 The maximum possible value of the score is 56 since a household uses all 5 strategies over all 7 days. The situation was evaluated according to the index value: Group Index Value Less serious Less than 10 USAID EFSP 69 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Moderate Between 10 and 19 Serious Between 20 and 29 Very serious More than 30 Source: CNSA, 2017 USAID EFSP 70 Initial Report of the WV Haiti La Gonâve EFSP Final Evaluation Appendix IV: Data collection tools (in separate files) 1. Quantitative survey-Final evaluation questionnaire 2. Focus group guide for women from mothers’ clubs -nursing women 3. Focus group guide for pregnant women- people with reduced mobility 4. Focus group guide for savings and credit group members (S4T) and VSLA agents 5. Focus group guide for youth trained in small business development 6. Focus group guide for ASCP 7. Focus group guide for FVFW beneficiaries 8. Interview guide for the Project Manager 9. Interview guide for the Distribution Manager/FVFW 10. Interview guide for the Monitoring & Evaluation Manager 11. Interview guide for the representative of an NGO implementing a similar program (Concern Worldwide ) 12. Interview guide for the MSPP 13. Interview guide for the MAST 14. Interview guide for Local Authorities (CASECs-Mayors) 15. Interview guide for Mother- Leaders 16. Interview guide for vendors 17. Interview guide for the Wesleyan Hospital Focal Point