Rapport final d'évaluation de l'activité pilote d'alimentation scolaire par voucher (VSFP)

Photo 1 Source: Friends of Paradis des Indiens, Haiti VOUCHER SCHOOL FEEDING PILOT (VSFP) ACTIVITY FINAL EVALUATION REPORT AUGUST 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Gladys Mayard (Team Lead), Amy Porter, Wesner Antoine, Louisena Louis, and Jennifer Mandel, for Social Impact’s Haiti Evaluation and Survey Services Project. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. FINAL PERFORMANCE EV ALUATION OF VOUCHER SCHOOL FEEDI NG PILOT ( VSFP ) ACTIVITY FINAL EVALUATION REP ORT AUGUST 2018 Evaluation Mechanism Number: AID - 521 - C - 17 - 00002 Haiti Evaluation and Survey Services for USAID /Haiti USAID.GOV FINAL EVALUATION REPORT | i ABSTRACT The United States Agency for International Development ( USAID )/ Haiti Voucher School Feeding Pilot ( VSFP ) A ctivity supports school canteens in Abricot providing two nutritionally balanced meals daily at Fondation Paradis Des Indiens (FPDI) support ed schools. Th is evaluation answer s thr ee evaluation questions (EQs), regarding VSFP ’s influence on parents’ school enroll ment decision (EQ1), the probable impacts of pilot discontinuity on enrollment (EQ2), and canteen sustainability after VSFP ends (EQ3). The evaluation team (ET) conducted a desk review, 17 key informant intervie ws , and 25 focus group discussions. Alongside school data on student enrollment, attendance , and performance, the ET used these multiple data sources to analyze and dra w findings, conclusions , and recommendations about VSFP . For EQ1, the evaluation concluded that school canteen s are closely connected to parents’ decision to enroll their children. Other motivating factors include low tuition and education quality. However, for the most vulnerable, canteen s factor significantly in to families’ economic and livelihood decisions. For EQ2, the ET concluded that the canteen has a clear impact on attendance. Whether or not a child eat s at school factor s strong ly in to parents’ decisions about how frequently children go to school, and motiva t es students to arrive before school , and stay through out the day. Overall, the ET found that school attendance frequency would decline if VSFP end s . For EQ3, stakeholders considered ways to continue the school feeding activities after VSFP . FPDI is active ly seek ing support for VSFP ’s continuation, however, area families face difficult economic situation s , limiting most community stakeholders ’ ability to materially contribute to ensure program continuity. USAID.GOV FINAL EVALUATION REPORT | ii TABLE OF CONTENTS Abstract ................................ ................................ ................................ ................................ ................................ ................. i Table of contents ................................ ................................ ................................ ................................ ............................... ii List of tables and figures ................................ ................................ ................................ ................................ .................. iii Acronyms ................................ ................................ ................................ ................................ ................................ ............. v Executive Summary ................................ ................................ ................................ ................................ ........................... 1 Activity Summary ................................ ................................ ................................ ................................ .......................... 1 Evaluation Purpose and Questions ................................ ................................ ................................ ........................... 1 Evaluation Design, Methods , and Challenges ................................ ................................ ................................ .......... 1 Findings and Conclusions ................................ ................................ ................................ ................................ ............ 1 Findings for EQ1 ................................ ................................ ................................ ................................ ....................... 1 ConclusionS for EQ1 ................................ ................................ ................................ ................................ ............... 2 Findings for EQ2 ................................ ................................ ................................ ................................ ....................... 3 Conclusions for EQ2 ................................ ................................ ................................ ................................ ............... 3 Findings for EQ3 ................................ ................................ ................................ ................................ ....................... 3 Conclusions for EQ3 ................................ ................................ ................................ ................................ ............... 5 Recommendations ................................ ................................ ................................ ................................ .................... 5 VSFP Activity Background ................................ ................................ ................................ ................................ ................ 7 Activity Summary ................................ ................................ ................................ ................................ .......................... 7 Theory of Change ................................ ................................ ................................ ................................ ..................... 9 Evaluation Background ................................ ................................ ................................ ................................ ................... 10 Evaluation Purpose And audience ................................ ................................ ................................ ........................... 10 Evaluation Questions ................................ ................................ ................................ ................................ .................. 10 Evaluation Team and Support Structure ................................ ................................ ................................ ................ 10 Evaluation Methods and Challenges ................................ ................................ ................................ ............................ 11 Phase One: Planning ................................ ................................ ................................ ................................ ................... 11 Phase Tw o: Field Data Collection ................................ ................................ ................................ ........................... 12 Key Informant Interviews ................................ ................................ ................................ ................................ ..... 12 Focus Group Discussions ................................ ................................ ................................ ................................ ..... 13 Phase Three: Data Processing and Analysis ................................ ................................ ................................ .......... 14 Challenges and Mitigation Strategies ................................ ................................ ................................ ...................... 15 Time and Geographical Challenges ................................ ................................ ................................ .................... 15 Behaviour Effect ................................ ................................ ................................ ................................ ...................... 15 Overlap in respondent categories ................................ ................................ ................................ ...................... 15 USAID.GOV FINAL EVALUATION REPORT | iii Availability of Respondents ................................ ................................ ................................ ................................ .. 15 Findings and Conclusions ................................ ................................ ................................ ................................ ............... 16 EQ1: To What Extent was the Decision to Send Children to School Influenced by VSFP ? ..................... 16 Findings ................................ ................................ ................................ ................................ ................................ ...... 16 Conclusions ................................ ................................ ................................ ................................ .............................. 19 EQ2: To What Ext ent Might the Frequency of School Attendance Change Once the Program Ends? 20 FINDINGS ................................ ................................ ................................ ................................ ................................ 20 Conclusions ................................ ................................ ................................ ................................ .............................. 22 EQ3: Have the VSFP stakeholders considered ways to continue the school feeding activities beyond the pilot’s period of performance? ................................ ................................ ................................ .......................... 22 Findings ................................ ................................ ................................ ................................ ................................ ...... 22 Conclusions ................................ ................................ ................................ ................................ .............................. 24 Recommendations ................................ ................................ ................................ ................................ ........................... 25 Annexes ................................ ................................ ................................ ................................ ................................ ............. 27 Annex A: Evaluation Matrix ................................ ................................ ................................ ................................ ...... 28 Annex B: expected preliminary results ................................ ................................ ................................ .................. 30 Annex C: Documents consulted in Desk Review ................................ ................................ ............................... 31 Annex D: List of Schools ................................ ................................ ................................ ................................ ........... 32 Annex E: scope of work ................................ ................................ ................................ ................................ ........... 34 Eval uation Purpose and Questions ................................ ................................ ................................ .............................. 34 SUMMARY INFORMATION ................................ ................................ ................................ ................................ ........ 34 Description of the Problem and project background ................................ ................................ ............................. 34 EVALUATION QUESTIONS ................................ ................................ ................................ ................................ ........ 35 EVALUATION DESIGN AND METHODOLOGY ................................ ................................ ................................ . 35 DELIVERABLES AND REPORTING REQUIREMENTS ................................ ................................ ......................... 35 EVALUATION TEAM COMPOSITION ................................ ................................ ................................ .................... 36 EVALUATION SCHEDULE ................................ ................................ ................................ ................................ .......... 36 FINAL REPORT FORMAT ................................ ................................ ................................ ................................ ............ 37 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT ................................ ................... 37 Annex F: Evaluation Tools ................................ ................................ ................................ ................................ ........ 38 LIST OF TABLES AND F IGURES Figure 1: Overview of the VSFP Activity ................................ ................................ ................................ ...................... 7 Figure 2: VSFP Theory of Change ................................ ................................ ................................ ................................ .. 9 USAID.GOV FINAL EVALUATION REPORT | iv Figure 3: Data Collection Methods Used and Gender Breakdown ................................ ................................ ..... 12 Figure 4: Student Performance during Project Implementation ................................ ................................ ........... 18 Figure 5: Enrollment over the project life in sampled schools ................................ ................................ ............. 19 Figure 6: Attendance at VSFP Schools ................................ ................................ ................................ ........................ 20 USAID.GOV FINAL EVALUATION REPORT | v ACRONYMS ACRONYM MEANING ASEC Assemblée des Sections Communales CASEC C onseil Administration des Sections Communales DPSNAS Documents de Politique et Stratégie Nationales d'Alimentation Scolaire EQ Evaluation Question ESS Evaluation and Survey Services ET Evaluation Team FGD Focus Group Discussion FPDI Fondation Paradis des Indiens GOH Government of Haiti IFOS Institut de Formation du Sud IR Intermediary Results KII Key Informant Interview MENFP Ministère de l'Education Nationale et de la Formation Professionnelle NGO Non - Governmental Organization PNCS Programme National Cantine Scolaire PRRNU Plan Reduction des Risques Naturels en zone Urbaine SI Social Impact SOW Scope of Work TL Team Leader ToC Theory of Change UNICEF United Nations International Children's Emergency Fund USAID United Agency for International Development VSFP Voucher School Feeding Pilot WASH Water Sanitation and Hygiene WFP World Food Program USAID.GOV FINAL EVALUATION REPORT | 1 EXECUTIVE SUMMARY ACTIVITY SUMMARY The United States Agency for International Development ( USAID ) /Haiti financed the Voucher School Feeding Pilot ( VSFP ) implemented by Fondation Paradis des Indiens (FPDI) under the contract amount of 1,767,145 US dollars to provide an alternative and improved approach to school feeding programs that minimize s the burden on school staff , while ensuring that poor students are provided with a nutritious snack an d meal each school day. EVALUATION PURPOSE A ND QUESTIONS This final evaluation ’s purpose is to inform future USAID /Haiti programing in school canteens. Specifically, the evaluation assessed to what extent VSFP made or is on a path to making positive contributions to students’ learning experience. The evaluation addressed the following evaluation questions (EQ’s): 1. To w hat extent was the decision to send children to school influenced by the VSFP activity? 2. To what extent might the frequency of school attendance change once the program ends? 3. Have the VSFP stakeholders considered ways to continue the school feeding activi ties beyond the program’s period of performance? EVALUATION DESIGN, M ETHODS, AND CHALLENG ES T o ensure the reliability and validity of findings , t he evaluation used qualitative data collection methods to gather information from a diverse range of sources , provid ing strong evidence to answer the evaluation questions (EQ). The design included a desk review, 25 f ocus g roup d iscussions ( FGD s ), and 17 k ey i nformant i nterviews ( KII s ). Data collection occurred within targeted schools across four commu nal sections of the municipality of Abricots. S even out of nine schools operating the VSFP activity were select ed using c onvenience sampling . The ET recorded responses from 209 individuals (98 women and 111 men). Respondents included key partners (FPDI) , l ocal governance authorities, Conseil de d'Administration de la Section Communale (CASEC), Assemblée de la Section Communale ( ASEC), municipal official s ; school staff (principals and teachers), local private food vendors, and parents. The primarily qualitative evaluation design mean t that limitations were expected; such as, interviewer effects and the need to rely on informants ’ availability and willingness to fully participate in KIIs and FGDs. To mitigate these limitations, the E valuation T eam (ET) applied a variety of methods (i.e., desk review; data on school enrollment, attendance , and performance; KIIs, and FGDs), to allow for data triangulation collection through the coding and analysis process , which increased the reliability and val idity of findings, conclusions , and recommendations. FINDINGS AND CONCLUS IONS FINDINGS FOR E Q1 The ET found that school canteen presence contributes to parents’ motivation to enroll their children in school. Teachers and school principals confirmed that p arents are attracted to the school canteen, stating that parents usually ask about the canteen while enrolling students. KIIs and FGD s with teachers, school principals, and parents all cited the correlation between the school ’s feeding program and USAID.GOV FINAL EVALUATION REPORT | 2 parents’ decision to send their children to school. Substantiation of VSFP ’s impact on enrollment decisions comes from school enrollment data collected throughout VSFP ’s life cycle. The ET also drew on existing literature on VSFP to con firm the data collected during field work in Abricots. These multiple data sources triangulate our key finding that the VSFP activity had a significant impact on parents’ decision to send their children to school. Most parents interviewed in this study suf fer economically, especially since Hurricane Matthew. Even knowing it is the ir duty to feed their children every morning before school, parents have limited economic capacity since Mathew destroyed the commune resources . T hey are forced to make difficult c hoices within their limited economic means. From FGDs, the ET found that appreciation for canteen increases with the number of children that parents have (up to four sometimes), and the ir poverty level. VSFP is well appreciated in this regard, and these b enefits affect parents’ decision - making in terms of their children’s enrollment in school. For parents unable to feed their children, teachers and school directors described VSFP as being “like a gift from heaven” and “a relief.” The effect of VSFP ’s nutri tious and assorted meal provision s was recognized by many parents in their comments related to changes in their children’s health. Food quality has a positive impact on children’s health, according to parents . When a child receive s a nutritious meal, there are positive outcomes in his or her health. Teachers and principals also reflected on VSFP ’s contribution to student health, which they associated with student learning. It is important to note that the canteen was one of multiple reasons parents gave as to why they decided to enroll their children in a VSFP school. The FPDI schools also did not require parents to purchase school supplies (as other schools do) and charged fees that parents stated were lower than those at other schools. Additionally, FPD I schools were understood by parents to be of good quality. While teachers and school principals believe that school quality was parents’ primary motivation for school enrollment decisions, parents emphasized that the food provision could not be separated from quality education. To parents, food provision is a primary concern directly linked to any subsequent assumptions of educational quality. To a lesser extent, school proximity and extracurricular activities also factored into parents’ decision to enroll their children. These multiple factors must also be considered in school enrollment decisions. CONCLUSION S FOR EQ1 The presence of a school canteen is closely connected to parents’ decision to enroll their children in school. Following Hurricane Mathew, parents in Abricot s became more vulnerable and expressed appreciation for relief from the burden of feeding children before sending them to school. With the program, students eat regularly during school days , which improve s their learning experience and school performance. There are several advantages of a school canteen , which influence parents’ enrollment decisions. Th e school canteen benefits parents, allowing them to save money on food, as well as students, allowing them to be fed and have a better day in school. Indirectly, VSFP also benefits teachers, whose classrooms are filled with more attentive students, and dir ectors , who can proudly manage schools with improved student performance. School canteens are not the only factor that parents’ value when deciding to send their children to school. Other important factors include low tuition fees and education quality. However, for the most vulnerable parents, a school canteen factors significantly in their families’ economic and livelihood decisions. USAID.GOV FINAL EVALUATION REPORT | 3 F INDINGS FOR EQ2 Multiple pieces of evidence assert both the impact that VSFP had on student attendance (and tim eliness ) and demonstrate how parents, teachers, principals, and others anticipate decreas es in attendance frequency at VSFP ’s end. Teachers and school principals confirmed that with VSFP , variation in the number of students attending class is greatly reduc ed. Regular attendance during VSFP is a marked difference from school attendance frequency before VSFP and, as will be detailed below, difficult to maintain without VSFP . As meals are served at a specific time, students arrive at school very early to avoid missing the first meal. The parents reported that the school canteen not only motivated students to go to school, but also to do so on time and stay for the anticipated second meal service. Parents are typically forced to choose between school and domesti c or agricultural labor for their children . W ith VSFP , this equation shifts towards a preference for sending their children to school. Without VSFP , parents react differently, generally negatively, to sending their children to school if they will not be fe d. Some parents reported that they would only send their children to school if food is available. Teachers concurred that students would only attend school when their parents have a way to feed them. Some male parents also reflected that without the assura nce of food being provided , they would send their children to school irregularly. Finally, and relatedly, children’s health is expected to suffer if they are not able to receive food at school. This presents a cycle in which not having access to nutritious food increases the likelihood of a variety of illnesses, which , in turn , make s it less likely that a child (who is suffering) will attend school. Respondents describe d illnesses related to malnutrition suffered by their children prior to VSFP and anticipate these conditions may return if VSFP ends. CONCLUSIONS FOR EQ2 The canteen has a clear impact on attendance. It is a strong motivational factor for parents to send th eir children to school and for students to attend school on time and for the full duration . Overall, the ET found that school attendance frequency would decline if VSFP were to end. Additionally, benefits of timeliness, better student health, and academic improvements were expected to revert to their prior status. If the canteen ends, attendance frequency will be reduced , as parents may not send children to school on a regular basis and keep them home to help with domestic work or agricultural labor. Studen t s’ performance at school will decrease and fewer students will advance to the next grade level. The ET concludes that there will be a significant decrease in school attendance frequency with VSFP ’s end. F INDINGS FOR EQ3 The ET found that while FPDI leadership had successfully collaborated with international donors to ensure that the school feeding activity would continue, other stakeholders within the Abricots commun e were not sure how to maintain the program: 21/25 FGDs with parents, teachers, vendors, and canteen supervisors found that stakeholders have not considered continuing school feeding activities beyond the program. A t the time of fieldwork and analysis, the means to sustain VSFP beyond the period of performance was highly uncertain am ong those in the Abricots commune . The lack of action among parents, local authorities, teachers , and others reflects the current socio - economic situation in Abricot s , rather than disinterest in seeing the program continue. Life in Abricot s is very diffic ult, having worsened with Hurricane Mathew ’s passage . The economic challenges respondents raised that led the ET to conclude that VSFP strongly impacts parents’ decision to send their children to school, and school attendance frequency, also means that par ents and local community actors are generally unable to sustain VSFP without external support. Various stakeholders reflected on their capacity to support the Activity’s continuation and their suggestio ns for maintaining the program: USAID.GOV FINAL EVALUATION REPORT | 4 PARENTS While both male and female parents noted a willingness to support the program, offering, for example, to provide products from their own gardens, they also emphasized that they had little themselves and were materially unable to support the program. FPDI F PDI officials think that parents should work with them to see how to continue the program, due to the activity’s importance and impact on children ’s education . Officials added that they also plan to contact various foundation partners, to see what other st rategies they might be able to adopt to allow them to continue the meal program at its current level (a snack and a meal each school day). Securing FPDI partnerships outside the community Negotiations and activities intended to ensure continued program funding from outside the Abricots commune were undertaken by FPDI leadership, USAID , and several international donor organizations. The ET learned about these activities through USAID : s takeholders in the Abricots commune either did not know about, or chose not to disclose, these potential funding sources to the ET. After fieldwork and an alysis, the ET learned from USAID that efforts to engage actors outside the Abricots community had been successful. FPDI received funds from Swiss NGO HEKS - EPER, to support its nine schools from February 2019 to June 2020. USAID will support a no cost exte nsion bridge funding for FPDI school feeding activities through fall 2018 , so that there will be no gap in services for students served by the school feeding activity . Additionally, HEKS - EPER will finance school feeding programs following the VSFP activity model in 16 remote localities across the communes of Jérémie and Roseaux, in addition to FPDI schools in the Abricots commune. SCHOOL P ERSONNEL School personnel such as teachers, principals , and canteen supervisors declared that they could do nothing t o materially support the Activity’s continuation. The ET also found that improving program sustainability will require that key stakeholders, including teachers, feel that their interests are being considered within the school feeding program’s design. CATERERS In line with other stakeholders’ comments, caterers believe that the activity is important and has contributed greatly to relieving the parents’ heavy burden and helping children progress in school. However, food vendors felt that local actors sho uld not be relied upon to make this activity permanent. LOCAL A UTHORITIES Abricot s’ local authorities shared parents ’ and food vendors ’ opinion s as well. The only way for them to contribute to the activity’s sustainability is by offering technical support, such as advice during meetings aimed at highlighting new strategies to be adopted in VSFP . SUSTAINABILITY THROU GH I MPROVED E FFICIENCIES Parents and school officials noted that strengthening food distribution supervision, possibly through a committee of pa rents, students, teachers , and school directors would help to improve the current VSFP . Their responses point to potential inefficiencies in the pilot’s existing operational structure that could be resolved to reduce costs – making the program sustainable with lower funding requirements . Improving efficiencies might also increase stakeholder support for the activity , especially among teachers who seemed, at times, displeased at being left out of the pilot’s design and implementation. Multiple stakeholders s uggested that raising awareness about VSFP and sharing successes with parents, agriculture producers, and the local community was also a potential way to build support for this type of activity . USAID.GOV FINAL EVALUATION REPORT | 5 CONCLUSIONS FOR EQ3 Based on the above findings, FPDI staff, with the support of actors outside Abricot, including USAID and Swiss NGO HEKS_EPER were able to ensure that school feeding activities would continue beyond the pilot’s period of performance. However, Abricot s commune members with whom the ET conducted research did not disclose or make statements to indicate that they were a wa re of the high - level efforts undertaken by USAID to secure funding from outside the commune. Financial support from outside the commune was critical, as other stakeholders within the Abricots commune believed that the difficult economic situation that families in the area were facing would not allow them to contribute to the continuation of the activity. R ECOMMENDATIONS To build upon gai ns , address the shortcomings of VSFP performance , and ensure sustainability, the ET offers the following three recommendations: First, considering the successes of VSFP in Abricot s, per the conclusions related to EQ1 and EQ2, the ET recommends that VSFP be continued in Abricot s and possibly expanded to the Grande Anse and/or South department s . These neighboring communities, with similar demographics, would also benefit from this type of school canteen program. Hurricane Mathew exacerbated parents’ vulnerability in the Grande Anse and South departments, and they, like those in Abricot s , are still in recovery mode. Specific recommendations on how to continue the program include: • USAID and others considering new or ongoing investment in school feeding programs should take into consideration EQ3 findings and conclusions regarding stakeholder engagement. While this pilot endeavored to reduce the burden on school staff, the result was that some ( i.e. teachers) expressed frustration with being left out of the design and implementation of a n activity, which was so closely connected to their responsibilities for the students in their classrooms. • EQ3 conclusions also noted potential areas for improving efficiencies and partnering to improve on the activity (e.g., sharing successes broadly within the local community and potential partnerships based on the program’s use of clean energy for food production). The ET learned that such activitie s had been introduced shortly before and after fieldwork and analysis for this report. For example, promotional efforts , including site visits to FPDI schools, attended by a variety of national and international actors , including the World Food Programme, elevated the visibility of the VSFP activity , which ultimately secured financial support for the activity beyond the pilot period of performance. • School feeding programs benefit all students, and play a significant role in students’ food security, especia lly among vulnerable parents with multiple children. School feeding programs should target vulnerable parents, who are most in need of a school canteen, to amplify the activity’s impact. This could be done by identifying schools in the most vulnerab le areas and targeting those that are not at capacity , as the ET found that schools with canteens in Abricots experienced increased student enrollments. While finalizing the report, the ET learned that the project will be continued, with donor support (det ailed below) , under revised guidelines that cover all fees for very vulnerable parents (20 - 30% of parents) who cannot afford to pay for the canteen . USAID.GOV FINAL EVALUATION REPORT | 6 • Continuing to w ork alongside the Programme Nationale de Cantines Scolaire ( PNCS ) will help to ensure that school feeding programs benefit local school communities across Haiti, and could be a strategy for sustainably expanding VSFP activities in the region more systematically. Second, given that school canteens were one of multiple fac tors parents considered in their decision to enroll children in school, the ET recommends considering interventions that touch on the multiple factors that influence enrollment. • The ET encourages future programming to consider additional benefits that woul d increase enrollment and attendance such as school materials ( e.g. textbooks) and charging a low (or scaled) tuition rate. These additional advantages would make schools even more attractive to parents. Future programming could tailor support to schools b ased on neighborhoods ’ and families’ vulnerability to maximize student attendance and performance. Third, VSFP has made notable positive impacts in the Abricots commune . H owever , the program’s forecasted closure was a source of concern among the local community whose limited means are not positioned to maintain the program without external support. Because of this, the ET recommends that greater emphasi s on sustainability be integrated into school feeding programs at their inception . • Given the inability of the local community to financially support VSFP ’s continuation beyond the pilot, future school feeding pilots should consider developing a sustainability plan and/or an exit strategy. As part of such a plan, partn erships and linkages to organizations that could increase the local capacity, efficacy, and quality of VSFP services should be identified and engaged from the pilot’s start. Even if pilots were not continued, these actions would positively stimulate commun ity support and engagement, increasing the likelihood of successful implementation and sustainability beyond the pilot period. • Positively, the ET learned that F PDI received funds from Swiss NGO HEKS - EPER to support the continuation of its school feeding p rogram from February 2019 to June 2020 . USAID will support a no cost extension bridge funding for FPDI school feeding activities through fall 2018. The ET welcomes this support , which aligns with its recommend ation to contin ue the pilot model , which at this point requires financial support from outside the Abricots commune. Additionally, news that HEKS - EPER will finance school feeding programs following the VSFP activity model in 16 remote localities across the communes of Jérémie and Roseaux , in addition to Abricots , aligns with the ET’s recommendation to consider expanding the program to neighboring vulnerable communities . USAID.GOV FINAL EVALUATION REPORT | 7 VOUCHER SCHOOL FEEDI NG PILOT ACTIVITY BACKGROUND A CTIVITY SUMMARY Figure 1 : Overview of the VSFP Activity To offer an alternative and improved approach to providing poor students with access to nutritious snacks and meals at school , the United States Agency for International Development ( USAID ) financed the Voucher School Feeding Pilot ( VSFP ) activity implemented by Fondation Paradis Des Indiens ( FPDI ) under the contract Amount: $1,767,145 USD , which was awarded with a period of performance from October 2015 to September 2018 . VSFP , with support from the Government of Haiti (GOH), seeks to establish a safety net that builds demand for locally produced foods. This is done by building linkages and coherence with the Feed the Future initiative ’s agricultural production activities. By institutionalizing a safety net system that recognizes and embraces GOH leadership and creates a platform for multi - donor investments , VSFP play ed an important cross - sectoral role in supporting rural Haitian communities . VSFP incorporate d USAID f orward principles in an innovative manner that forged non - traditional relation ships with local partners, linked current social safety net programming gains , and integrated past learning in a novel design approach . School feeding is particularly important for the Haitian Government because it is located at the intersection of two of the n ation ’s economic and social development priority sectors : e ducation and a griculture . 1 D e fin ing a national school feeding policy is a sine qua non to unlock ing the potential of both these sectors. 1 Programme National De Cantine Scolaire (PNCS), Document de Politique et de Stratégie Nationales d’Alimentation Scolaire (DPSNAS) - Janvier 2016 ; Ministère de l’Education Nationale et de la Formation Professionnelle (MENFP) USAID.GOV FINAL EVALUATION REPORT | 8 The GO H recognizes school feeding as a top priority . E vidence of its prioritization is realized in the fact that the n ational s chool f eeding p rogram , Programme Nationale de Cantines Scolaire (PNCS ), the Haitian Government’s implementation and oversight body for school feeding , receives an annual budget allocation of approximately eight million US dollars from the Haitian Treasury. 2 The fact that the government allocates resources to the PNCS sends a strong signal of political will and commitment to school feeding programs. The PNCS , wh ich is supposed to have the capacity to coordinate and plan overall school feeding efforts , as well as to take th e lead in the implementation of a national program as the “ national coordination body ” has no rules or regulations that mandate its status. As a result, there are significant variations from year to year in the numbers of school children who access school meals and there are no targeting criteria established regarding school selection , nor are there standards related to the quality, diversity, regularity , or timeliness of meals. Per national policies and priorities, the PNCS ’ mandate is rooted in education, welfare , and agriculture - related objectives. Its objectives include: an inten tion to reach the greatest number of school children in the country , to increase the capacity of the government to manage and implement the progr am independently , and to increase the amount of local agricultural production purchased for the program. With the direct support received by PNCS , the number of children receiving meals in schools var ies from year to year , falling between 600,000 and 1.1 m illion children , out of an estimated 2.3 million school aged children . 3 School feeding programs do not generally procure commodities in bulk , instead sourcing from local markets and relying heavily on imported in - kind food aid. Although school feeding prog rams are implemented throughout the world in both developed and developing countries, Haiti’s requires an approach that is tailored to its education al context. Several factors , described below, necessitate a unique approach . As noted above, a lthough PNCS is seen as the “national coordination body,” there are no rules or regulations that mandate PNCS’ status , n or are there regulations or legislation framing the way school feeding programs are implemented. M ost schools are private/ non - public wi th widely differing standards, norms , and education quality. Despite the presence of PNCS, most of Haiti’s school meal programs are N on - Governmental Organization ( NGO ) /donor/international organization financed and implemented. Thus, s chool feeding activiti es include many implementers, each with their own objectives and rationales for school feeding programs, with no overarching national system of management or oversight. VSFP worked with PNCS to test the design of a developed approach , taking Haiti ’s low e ducation quality and overwhelmed teachers and school administrative personnel into consideration . The pilot eliminate d the direct role of school personnel, including school directors and teachers, in commodit y transportation and storage and in the preparat ion and delivery of meals. P rivate food preparers handled m eal p reparation in the targeted communities (traditionally, mostly women have this role) outside of the school and the prepared cost - effective meals were delivered to schools daily in the morning around 7:30 2 Voucher School Feeding Program, Project Design; USAID 2014 3 Document de Politique et Stratégie Nationales d’Alimentation Scolaire (DPSNAS), Ministère de l’Education Nationale et de la Formation Professionnelle (MENFP), 2016 USAID.GOV FINAL EVALUATION REPORT | 9 am and 11:30 am. Overall management of the transporta tion, storage, preparation , and delivery of meals to the school s was , therefore , part of the contracted food prepar er’s responsibilities, as was the case for any restaurant or caterer. By removing this role from the school and placing it in the hands of ex isting community fo od preparers, the pilot relieved the burden of managing these processes from the school , while increasing and improving local community food preparers’ business opportunities. The contracted c aterers , who prepared m eals using fresh local products , received basic training in hygiene, nutrition, conservative energy (improved cook stove), micro - entrepreneurship and water, sanitation , and hygiene ( WASH ) . They were community member s , small restaurant owners , or street vendors in the targeted schools’ neighbor hoods Figure 2 : VSFP Theory of Change C ontracted food vendors prepare d and deliver ed daily meals to schools. S tudents ate a snack early in the morning and a hot meal over their lunch break. The project reached 2,350 students and 40 local food providers in 10 beneficiary schools in the commune of Abricot s , department of Grand’Anse, Haiti. T HEORY OF CHANGE The VSFP Theory of Change (ToC) posited that co ntracting local private vendors and training them on how to prepare and hygienically serve nutritious and ba lance d meals to school children w ould lead to school children consuming two nutritious meals per school day. In doing so, VSFP lead to improve s tudent le arning outc omes as students w ere enrolled in school and attended with greater frequency . VSFP operationalized this ToC through two Intermediate Results (IRs) : 1. Ensure students have access to two meal s each school day 2. Improve the nutritional quality of students ’ meals As part of its effort to increase school attendance and improve achievements in the 10 VSFP schools in the targeted communes of Abricot s , the pilot activity approach ed school feeding as a safety net activity. In addition, this pilot focused on reinf orcing the linkages between the private - sector food provider s and USAID.GOV FINAL EVALUATION REPORT | 10 the school s to reduc e the operational burden that has fallen to school staff other school feeding programs. In the emergency resulting from the aftermath of H urricane Matthew , VSFP ’s strategy was reviewed and adap ted to the operating context. The local food service providers were given tools and financial resources from USAID , delivered through the implementing partner Cooperative for Assistance and Relief Everywhere , to provide meals to the entire community, as all members of the community were in severe need after the region was destroyed by Hurricane Matthew. The local private food vendor s w ere directed to support the community with community - wide food distributions from the time of the hurricane (September 2016) , during which time the VSFP schools were used as community shelters. EVALUATION BACKGROUN D E VALUATION PURPOSE AND AUDIENCE At the request of USAID , the Haiti Evaluation and Survey Services (ESS) project conducted the final performance evaluation of VSFP . The purpose of this final evaluatio n was to inform future USAID programs based on lessons learned. Specifically, the evaluation assess ed to what extent V SFP made or is on a path to making positive contributions to students’ learning experience. The evaluation results are intended to inform USAID and implementers’ programs within the education sector in Haiti. The primary stakeholder for this evaluation is USAID /Haiti. As appropriate, the results may be shared with other stakeholders such as the GOH , PNCS , and World Food Program (WFP), among othe r s. E VALUATION QUESTIONS The final per formance evaluation of VSFP address ed the following evaluation questions (E Q’s): 1. To what extent was the decision to send children to school influenced by the VSFP activity ? 2. To what extent might the frequency of school attendance change once the program ends? 3. Have the VSFP stakeholders considered ways to continue the school feeding activities beyond the p ilot ’s period of performance? E VALUATION TEAM AND S UPP ORT STRUCTURE Through a competitive bidding process, Haiti ESS hired the Institut de Formation du Sud ( IFOS ) , a local research firm, to conduct the VSFP performance evaluation. To carry out the full breadth of the assignment, IFOS proposed a Haitian team qualified and experienced in project evaluation. The team’s experience is mainly in the evaluation of education projec ts and has worked in collaboration with IFOS in this domain for more than five years. The Team Lead (TL), Ms. Gladys Mayard, graduated in Social Sciences and Sciences Development at the State University of Haiti - Faculty of Ethnology. Ms. Mayard is an Ant hropologist/Sociologist with a Master of Science in Development. She is a Lawyer with experience training in Human Rights. She has skills and strong experience in qualitative and quantitative research. She also has conceptual and methodological skills to d evelop action and development plans, either locally or regionally, from participatory approaches with communities in analyzing their own problems, the environment and USAID.GOV FINAL EVALUATION REPORT | 11 solutions proposed. Ms. Mayard has been part of the IFOS research team since 2009 and has led several socio - economic research studies for IFOS in nutrition, civil society participation, economy and violence against women across Haiti. Ms. Mayard prepared instruments and fi nalized them through pretesting, developed field staff manual s and data collection protocols, led qualitative data collectors’ training, and supervised data collection and quality assurance for all these processes. As a member of the Haitian research clearance body ( Comité National de Bioéthique ), Ms. Mayard submitted applica tions for local research clearances and obtained relevant authorization (as needed). Three focus group discussion (FGD) facilitators and three notetakers supported the TL to increase the efficiency and accuracy of data collection and analysis. IFOS hired three language experts and translators for the instruments and FG D transcript translation, who are highly qualified and experienced translators and interpreters in Haiti. They translated the data collection tools and training materials with the approach se t out above and used in the past by IFOS, as requested in the Scope of Work ( SOW ) . While many evaluation projects utilize local firms exclusively for data collection, as TL, Ms. Mayard was additionally responsible for data cleaning, analysis, and drafting all evaluation deliverables (i.e., work plan, interview guides, out briefing presentation, draft and final reports, etc.). She was supported in these efforts by ESS local staff who worked with her on the data analysis and SI HQ staff, specifically the Project Director, who provided extremely det ailed feedback as par