Gras pou ranfòse kapasite estatistik nan edikasyon an Ayiti (P161160) - Rapò sou Achevman ak Rezilta Aplikasyon an

Document of The World Bank FOR OFFICIAL USE ONLY Report No: 158227 - HT IMPLEMENTATION COMPLETION AND RESULTS REPORT TF 0A5007 ON A SMALL GRANT IN THE AMOUNT OF US $0 .5 MILLION TO THE REPUBLIC OF HAITI FOR A Statistical Capacity Building in Education Gr an t (P161160) April 23 , 2021 Education Global Practice Latin America And Caribbean Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Regional Vice President: Carlos Felipe Jaramillo Country Director: Tahseen Sayed Regional Director: Luis Be nvenis te Pr actice Manager: Emanuela Di Gropello Task Team Leader : Yves Jantzem ICR Main Contributor : Claudia Lopez ABBREVIATIONS AND ACRONYMS CPF Country Partnership Framework DDE Departmental Directorate of Education ( Direction Départe menta le de l’E ducation ) DPCE Directorate of Planning and External Cooperation ( Direction de la Planification et de la Coopération Externe ) EMIS Education Management Information System FCV Fragility, Conflict and Violence GDP Gross Domestic Product GoH Gove rnmen t of Haiti GPE Global Partnership for Education IHSI H a i t i a n Institute f o r S t a t i s t i c s a n d I n f o r m a t i c s ( Institut H aïtien de S tatistique et d' I nformatiqu e ) M&E Monitoring and Evaluation MENFP Ministry of National Education and Vocational Traini ng ( M inistè re de l’Éducation Nationale et de la Formation Professionnelle ) PEQH Providing an Education of Quality in Hai ti Project (the Project Implementation Unit) PIU Project Implementation Unit QAS Quality Assurance System SA Social Assessment USI Infor mation S y ste m Unit ( Unité du Système d’Information ) WBG World Bank Group The World Bank Haiti Statistical Capacity Building in Educat ion Grant (P161160) Page 1 of 31 TABLE OF CONTENTS I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ................................ ....................... 4 II. OUTCOME ................................ ................................ ................................ .................... 12 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 18 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RI SK TO DEVELOPMENT OUTCOME .. 19 V. LESSONS LEARNED AND RECOM MENDATIONS ................................ .............................. 20 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ................................ ........................... 21 ANNEX 2. PROJECT COST BY COMPONENT ................................ ................................ ........... 26 ANNEX 3. RECIPIENT COMMENTS ................................ ................................ ........................ 27 The World Bank Haiti Statistical Capacity Building in Educat ion Grant (P161160) Page 2 of 31 DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P161160 Haiti Statistical Capacity Building in Education Grant Country Financing Instrument Haiti Investment Project Financing Original EA Category : C R evised EA Category : N A Organizati ons B orrower Implementing Agency MENFP The World B a nk Project Development Objective (PDO) Original PDO The objective of the Pr oject is to support the MENFP in establishing a sustainable data collection strategy for the school census which delivers qua lity d ata comparable to international standards; des i gning a MENFP strategy for strengthening its monitoring and evaluation syste m; and initiating implementation of said strategy. FINANCING KEY D ATES Approval Effectiveness Original Closing Actual Closing 02 - Jun - 2017 0 2 - J u n - 2 0 1 7 3 1 - A p r - 2 0 1 9 3 0 - O c t - 2 0 2 0 Orig inal Amount (US$) Revised Amount (US$) Actual Disbursed (U S$) D onor Financing Total 5 0 0 , 0 0 0 - 49 7 , 4 2 7 . 2 8 The World Bank Haiti Statistical Capacity Building in Educat ion Grant (P161160) Page 3 of 31 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed ( US$M) Key Revis ions 30 - Apr - 2019 0 . 1 9 Change in Results Framework C h ange in Components and Cost Change in Loan Closing Date(s) 1 7 - J u n - 2 0 2 0 0 . 3 0 C hange in Results Framework C hange in Components and Cost Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performan ce M&E Qu ality Satisfactory H i g h l y Satisfactory Sub s tanti a l RATINGS OF PROJECT PERFORMANCE IN ISRs No. Date ISR Archived DO Rat ing IP Rating Actual Disburs ement s (US$M) 01 05 - Sep - 2018 Moderately Satisfactory Moderately Unsatisfactory 0 . 0 6 02 27 - Mar - 2 019 Mo derately Satisfactory Moderately Satisfact ory 0 . 1 9 03 21 - Sep - 2019 Moderately Satisfactory Moderately Satisfactory 0 . 2 8 04 2 2 - Apr - 2020 Moderately Satisf actory Moderately Satisfactory 0 . 3 0 0 5 2 2 - O c t - 2 0 2 0 Satisf actory Moderately Satisfactory 0 . 4 0 A DM STA FF Role At Approval At ICR Regional Vice Pre s ident: J orge Familiar Calderon Carlos Felipe Jaram illo C ountry Director: Mary A. Barton - Dock Tahseen Sayed Director: J a i m e S a a v e d r a Luis Benveniste Practice M anager: R e e m a N a y a r Emanuela Di Gropello Task T eam Leader(s): Juan Baron, Melissa Ann Adelman , Yves J antzem ICR Contributing Author: C l a u d i a L o p e z The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 4 of 31 I. PROJECT CONTEXT AN D DEVELOPMENT OBJECTIVES Context C ountry Context at A pprai sal 1. In 2017, Haiti was in a classic situation of institutional a nd soc ial fragility , exacerbated by a series of ext e rnal sho cks in the decade prior tha t had hampered i ts development . After the c atastrophic earthquake of 2010 , which caused the death of app roximately 3.5 percent of the population , Haiti endured a widespr ead ch olera epidemic that same year . Six years late r , during the preparation of th is Ha iti Statistical Capacity Building in Educat ion Grant (the Project) , the devastating H urricane Matthew st ruck the country , affecting an estim at e d 2.1 million people. With a Gro ss Domestic Product ( GDP ) per capita of US$76 6 1 in 201 7 , the country remained one of the poorest in the world, with almost 6 0 percent of the Haitian population living below the nationa l poverty line . I nequality was high, with wealth and economic oppo rtunit y concentrated around Port - au - Prince. Access t o basic services was limited, parti cularly in rura l areas, which translated in to low human development indicators; Haiti rank ed 169 th out of 189 countries in the Human Development Index . 2. In addition to the ex ternal shocks and the subsequent deterioratin g economi c situation , political unce rtainty contrib ut ed to weak governance , whi ch complicat ed policymaking and exacerbated instability in t he already fragile state . C ontested presidential election results in Oct ober 2015 led to the appointment of a transit i onal gov ernment and , eight months l ater, to the an nulment of the initial round of elections . The election process stretched for a total of 14 months , with new presidential elections planned and postponed due to the damage sustained during Hurricane Matthe w in Octo ber 2016 . F inally held in l ate November 20 16 , t he presidential electio n 's results were declared official in January 2017 . Therefo re , at the time of Project appraisal, the c ountry had been governed by a transitional government for more than a year, contribu ting to an uncertain politi cal climate , d ifficulties in set ting politi cal priorities , a lack of clarity on institutional roles , a nd a lack of policy coherence among public sector actors . Country Contex t during Implementation 3. During Project imple m entation , a socio - p olitical crisis and the COVID - 19 pandemic further undermine d the recent development al gains in Haiti . After a short pe riod of relative stability following the 2017 elections, corruption alleg ations against government officials attracted public a ttention . W idespread protes ts against the g overnment began in several Haitian cities in July 2018, causing massive disruptions in services and economic activities . This unrest persisted and develo ped in to a socio - political crisis, intensifying in 2 019 and culminating in a period of total lockdown from September 2019 through December 2019 , during which economic producti on, schools i n or near cities, businesses, and the government and its services we re lar gely closed. While the socio - political situat i on stabi lized, on March 19, 2020, t he G overnment of Haiti (G oH ) announced its first two cases of COVID - 19 and declared a state of emerge ncy. The virus has since continued to take a toll on human health an d life in 1 World Bank Data The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 5 of 31 the country, with a rapid increase in cas e s and de aths between mid - June and m id - July 2020 ( from 3,662 con firmed cases to 6,727, and from 56 deaths to 141). Sectoral Context 4. At the time of Project appraisal in 2016 , b oth public and non - public s chools offer ed a low - quality education, while acces s to prim ary education was at risk . The quality of teaching wa s variable, and o ften low. T he average Haitian child enter ed the first grad e nearly two years late, due to a combination of (i) household finan cial c onstraints that delay school entry ; and (ii) s chools o ften demanding that childre n receive at least one year of pre - primary in order to prepare for first grade. Once in school, 13 per cent of first graders repeat ed the year, and only about half w ould reach the sixth grade . 2 A 2018 study of a fourth - gr a de math assessment by the Ministry of Education – Ministère de l’Education et de la Formation Professionnelle (MENFP) in both public and non - public schools found that student scores were only slightly abo ve wha t would be achieved by random guessing . 3 Whil e access to primary education had co nsistently i mproved since 2000 , gains we re at risk of being lost and school dropouts at risk of increa sing. The net primary enrollment rate had r i se n from about 60 perce nt in the early 2000s to between 70 and 80 percent i n 2012, in large part due to the im plementation of programs promoting access f unded by the GoH . 4 However, lack of public financing for th e sector, as well as the absence of a sustainable strategy for publ ic fin ancing of non - public education , resulted in t h e discon tinuity of tuition waiver a nd subsidy p rograms for non - public schoolin g as of 2016 . 5. Ch allenges in terms of education quality an d access were partly explained by governance issues in the sector , especi ally as they related to constraints in data g e neration and use. I n 201 7 , the plan ning functio ns of the MENFP were limited, a nd it lacked a National Strategy for the Development of Sta tistics. The school census strategy , moreover, was outdated and ine fficie nt , which resulted in a lack of timely and re l iable in formation on schools, teach ers and stud ents, as well as a lack of stru ctured data collection systems . The MENFP also lacked a pla n to strengthen the capacity to analyze data that could inform poli cy dec isions on quality and access. In the meantime , technic al and financial partners s uch as the W orld Bank and Inter - American De velopment Bank ( I A DB ) were financing major operations to su pport access and quality , whose gains were at risk of being lost wi thout a proper monitoring and evaluation system to t rack pro gr ess, allocation of resour ces , and inf orm ed decision - making processes in the context of limited public financing of the primary sector. The lack of robust statistical approaches and processes res trict e d the MENFP’s ability to make well - informed d e cisions on how to spend public reso urces and im prove service delivery and thre atened future external funding opportunities in the absence of a n effective tracking system . 2 Off to a Ba d Star t: Drivers of La te Entry i nto Primary Scho ol in Haiti (W orld Bank 2015). 3 Résultats de l’Evaluation Nationale 2017 en Haïti (Inte rnational Association for the Evaluation of Education Achievement 2019). 4 In recent years, the GoH prioritized the fina ncing of access to pri mary educa tio n, contributi ng to substant ial increases in school participation. The GoH financed tuition waivers to non - publ ic providers through: (i) the do nor - funded Education for All Project’s (EFA) Tuition Waiver Program beginning i n 2007 , and (ii) the G overnment - fun ded Universal , Free, and Co mpul sory Education Program ( Programme de Scolarisation Universelle Gratuit e et Obli gatoire ; PSUGO) beginning in 201 1. At their combined peak between 2011 and 2014, these two programs financed th e scho ol fees of over 60 percent of all primary students in Ha iti, with PSUGO representing about 20 percent of domestic public spending on educat ion. The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 6 of 31 Bank Rationale for supporting t he Pro ject 6. Reliable, timely, and accessible statis t ics on s tudent s, teachers and schoo ls were cons idered essential to allow the M ENF P to deploy its limited resources in the most cost - effec tive way. This Project aimed to develop a sustainable data collecti on str ategy, paired with a strong Education Managem e nt Infor mation System ( EMIS ) . I t al so aimed to strengthen the MENFP’s ability to analyze and use data , which would , in the long term, sup port improved evidence - based policy and decision - making and ultimat ely im prove education quality while maintaining acc e ss in Ha iti. This Project was align ed with the MENFP’s sector strategy — the Pl an Opérationnel 2016 - 2020 ) , which was being prepared at the time of Project p reparation — Specifically i t s first pillar (govern ance) a n d the objective of “reinforcing its plannin g , monito ring, and evaluation capabi lities . ” 7. The need for a n Education Manage ment Information System was not tackled comprehensive ly . At time of preparation, there was no on - going and comprehensive exter nally funded initiative on the EMIS and strengtheni n g of the school census. Only one pa rtner ( UNESCO ) was supporting the dialogue on SDG indicators , through the UNESCO ’s Institute fo r Statistics (UIS) but was not providing direct support to the EMIS . The I nforma tion System Unit ( U n i t é d u S ystème d ’ I n f o r m a t i o n , U S I ) of the MENFP had only been established in 2015 , and this Project was seen as a catalyst in strengthening USI and external partners’ technical and financial sup port around these priorities . 8. The u ncerta in political environment in 201 6 and competin g Governm ent priorities following Hu rricane Matthew led the MENFP to request th at the Project be executed by the World Bank on i t s behalf (Bank Executed on Behalf of t he Recipient ) . The volatile operating enviro nment , accompanied by numerous changes in edu c ation mi nisters , largely contribute d to this decisi on. The World Bank was well placed to fulfill this request because its Haiti education team was already engaged with the GoH on topics related to data co llecti on at t he time of Project appraisal . In parti c ular, th e team had initiated policy dialogue on the establishment of a Quality Assurance System (a set of tools to monitor students and school progress) for schools and was thus strongly positioned to imp lement this g rant. 9. The P roject wa s aligned with t h e System atic Country Diagnostic and the World Bank’ s Education Portfolio in Ha iti, which, at the time of Project preparation, was composed of the Education For All II (P124134 , Grant number IDH 7400 ) and P rovidi ng An E ducation Of Quality In Haiti ( PEQH, P1 5 5191 , Gr ant number D 1230 and D4700 ) projects . The P roject was also aligned wit h the Statistical Capacity Building Project (P157531) that focused on the Housing Census , and synergies between the two Projec ts wer e explo red during Project preparation. Proje c t Develo pment Objectives (PDOs) 10. Th e objective of the Project was to support t he MENFP in establishing a sustainable data collection strategy for the school census which deliver ed quality data comparable to int erna tional standards; designing a MENFP strat e gy for s trengthening its monitoring and evaluation system; and in itiating impl ementation of said strategy. The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 7 of 31 Key Expected Outcomes and Outcome Indicators 11. The PDO - level Results Indicators for the Project we re as fo llows: • PDO Indicator 1: New school census data col lection process officially a dopted by MENF P • PDO Indicator 2: Strategy for strengthening monitoring and evaluation (M&E) system officially adopted by MENFP • PDO Indicator 3: Technical prerequisite s for a fun ctional Education Management Information S y stem (EM IS) are in place 12. The long - t erm expected o utcome was that the MENFP st aff w ou l d access, analyze, and use the information in the EMIS to support their decisions , thus contributing to the higher - l evel obj ect ive of improving evidence - based policy and decision - making . 13. The PDO - level ind icators remaine d the same for the duration of the Project. Components 14. The Project had four components : Component 1: School census data collection strategy ( US$ 125 ,000) 15. T he objective of this component was to support the MENFP to def ine a revised s chool ce nsus data collection strateg y . It encompasse d support for, inter alia : i. A diagnostic of the existing data collection processes for the school census, including con sultatio ns with relevant stakeholders from the education sector in the Recipient’s t e rritory ; ii. T echnical support for the c reation of standards and templates of ed ucat ional data, in accordance with international standards ; iii. The design, piloting, and finaliz ation of a data collection process for the school census that improve d timeliness an d reliabi lity of data and took into ac count previous experiences ; and iv. T raining and workshops for MENFP national and departmental staff on the new the data collection proces s for th e school census . Component 2: MENFP M onitoring & E valuation (M &E) system st r engtheni ng strategy ( US$ 50,000) 16. The objective of this component was to suppo rt th e MENFP in develop ing an M&E system strengthening strategy . It included p rovision of supp ort for, inter alia: i. A diagnostic of the existing statistical infrastr ucture, infor m ation ma nagement processes and data ut ilization practices of the ii. A n evaluati on o f the implementation of national learning iii. A study tour for MENFP staff to learn from othe r countr ies’ experiences on setting up EMIS ; and The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 8 of 31 iv. The d esign, review, a dopti on , and d i sseminat i on of a strategy for strength ening the M&E system of the MENFP. Com ponen t 3: Implementation of the MENFP M&E s trat eg y ( US$ 265,000) 17. The objective of this compon ent was to support the i mplementation of the MENFP M&E system strengthe ning strategy . At appr oval, this component included p rovision of support for, inter alia: i. W orks hops and training session s for MENFP staff on the customization, population, and maintena nce of d atabases; survey design; data collection; data management; info rmation disse m ination; and data analysis for policyma king ; and ii. W orkshops and training for t he st aff the Haiti Institute for Statistics ( Institut H a ï tien de S tatistiques et d' In formatiqu e , IHSI ) staff as part of preparation s for the population census. Comp onent 4: Proj e ct imple mentation support (US$60,000) 18. This component encompassed support for the administration, implementation, monitoring, and evaluation of the Project. Project Restr ucturing 19. To adapt to a challenging country context and external shock s, and to the p ace of i mplementation , the P roject was restructured twice , in April 2019 and June 2020. First restructuring : April 2019 20. The Project underwent a level 2 restructuring , w hich was approved on April 30 , 20 19 and did not change the PDO or PDO - level results i ndicator s. The amount disbursed at the time of the restructuring was US$ 214,1 75 ( 4 2 percent of the grant amount ). The 2019 restructuring entailed the extension of the Proj ect clos ing date, a change to components and costs, and change s to the result s frame w ork at t he intermediate indicator level . 21. The changes are summarized below: i. E xtension of the Closing Date . The Project experienced initial setbacks in the implementatio n of C om ponent 1, 2 and 3 activities due to : (i) slow progress in en gaging with the p lanning directorate in charge of educati on statistics ( Direction de la Planif ica tion et de la Coopération Externe , DPCE ) i n the revision of the data collection process ; an d (ii) s ocio - political unrest during 2018 and early 2019. T hese chal lenges , however, were add ressed or mitigated, and impleme ntation gained momentum and progress e d s ignificantly towards P roject objectives in 2019 . The one - year extension (from April 30, 201 9 to Jun e 30, 2020) was requested to make up for the initial delays , allow for the c o mpletion of ongoing activities , and full y achieve the Project’s objectives . ii. Result s Framework. The intermediate indicator “Training fo r IHSI staff as part of preparation for popu lation census completed” w as deleted, since the workshops and training for I H SI staff as The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 9 of 31 part of preparation for th e population census were eliminated fro m the Project. iii. Cha nge s to components and cost . Initially, the Project included workshops and t raining for the H a i t i a n Institute f o r S t a t i s t i c a n d I n f o r m a t i c ( Institut H aïtien de S tatistiq ue et d' I nformatiqu e , IHSI ) , as part of C omponent 3. These activities were removed from th e Project, h owever, when the IHSI received another source of funding to c over the m. In addition, t he MENFP revised its data collection strategy and reduced t h e scope of Component 3 ( implementation o f the M&E strategy ) to emphasize the design of the M&E str ategy itsel f and to increas e the budget for piloting the new data collect ion stra tegy . Budget a ll ocations by component were revised to reflect these changes ( see tabl e below) . In addition to the act ivities above - mentioned that were rem oved, the evaluation of the implementation of national learning assessments , that was envisage d in the initial design of the Project , was not pursued. Indeed, there was a change o f strate gy on t his topic with the M inist ry and this activity, that was insert ed in component 2 as a placeholder to support potential efforts on the learning assessment sys tem was not carried a s part of this Project. Since this activity was not directly li n ked to t he outp ut of component 2 (nor pa rt of the result framework ) , it was n ot included as part of the restructuring itself . Pr oject Components Original Cost (US$) Re vi sed Cost (US$) 1. School census data collection strategy 125,000 160,000 2. MENFP M&E s y stem str engthening strategy 50,000 150,0 00 3. Implementation of the MENFP M&E s ystem strengthening strategy 265,000 140,000 4. Implementation Support 60,000 50,000 Total Pr oj ect Cost s 500,000 500,000 Second Restructuring : June 2020 22. In order to mitigate d e lays cau sed by the COVID - 19 pandemic , t h e Project underwent a second lev el 2 restructuring on June 17 , 2020 , which did not change the PDO or PDO - level results indicators . T he amou nt disbursed at restructuring was US$ 304,828 ( 61 percent of the grant amount ) . The re structuring entailed the extensi on of the closing date, changes to co mponents and costs and changes to the result s framework at the intermediate level . Those cha nge s are d etailed below: i. Extension of the closing date . The closing date was extende d by four months , until October 30, 2020, to factor in delays cause d by t he CO VID - 19 pandemic . For several weeks, g overnment buildings and schools were closed, resulting in MENFP s taff ’s inability to meet and organize workshops to finalize the school censu s data co llection strategy and related tra ining . Furthermore, t he GoH ann ounce d restrictions on large gatherings and travel. Th e four - month extension allowed the MENFP to po stpone the training and workshops that contributed to the achievement of Project’s o bjective s by Project clos ing. ii. Changes to the result s framework . Due to unce rtainty on how the COVID - 19 pandemic would evolve and when schools w ould reopen, the intermediat e indic ator “Pilot of the new data collection process for the school census complet e d” w as d eleted, and the intermediate indi cator “Training for MENFP national and departmental staff in the new data collection process completed” was replaced by “Design of th e tra ining content and materials on the new school data collection process comple t ed . ” The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 10 of 31 iii. Ch ange to components and costs . The follo wing activities were revised : o Data collection: Given that data collection activities on the grou nd could not be completed d ue to C OVID - 19 related travel restrictions, the pilot of the data collection strate g y was re moved from the Project and C ompone nt 1 a ctiv ities were adapted to f ocus on in - depth desk work . A consultant was hired to extract lessons learned from the previous s chool c ensuses and update the school census data collection strategy ; this approach mitigate d the impacts of the pandemic on P roject interventions and was less costly than the originally planned activities . This work was accompan ied by a revision of the sc hool ma pping tools. The methodology, developed with the support of other developmen t partner s (I A DB and more recently, the Spa nish A gency for International Dev elopment [ Agencia Española de Cooperación Internacional para el Desaro llo AECID ]), matches the da ta from the school census on education supply with economic information and project i ons on s tudent demand and needs in a given area. Together with the revised data collection strategy, these tools would serve as a basis for train ing on micro - planning at th e subna tional levels of the MENFP. o Training sessions: Furthermore, the implementat i on of th e training sessions on the new dat a coll ection activities that were meant to be implemented once the data collection strategy was develop ed were removed from the sc ope of the Project because of the restrictions on large gatherings. Instead, this a c tivity w as replaced by the design of a ful l trai ning program for MENFP cent ral and local staff , as well as a roadmap for implementation for the t raining sessions . o Costs: T he cost s avings from C omponent 1 were re allocated to C omponent 3 to focus on the im p lementat ion of the M&E strategy, specifica lly al locating more resources for the hiring of tec hnicians to finetune the implementation of the techn ical prerequisites for a n E MIS. An international consultancy in Pro ject management was also financed to move f o rward th e EMIS agenda and organize the EMI S work plan within the MENFP . Chan ges in cost by component related to these revisions are detailed in th e table below . Project Co mponent s Original Cost (US$) Revised Cost (US$) 1. School census data collection st r ategy 16 0,000 60,000 2. MENFP M&E system str engthening strategy 150,000 15 0,0 00 3. Implementation of the MENFP M&E system strengthening strategy 140,000 250,000 4. Implementat ion Suppor t 50,000 40,000 Total Project Cost 500,000 500,000 Theory of Change 23. The Project’ s theory of change is summarized b elow: Main activities Outputs P DO Outcomes Long - Term Outcomes - Evaluating the existing data collection processes for the - E valuation of the existing data collection MENFP establishes a sustainable data collect i on The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 11 of 31 scho ol census; - Providing technical support for the development of the data collection s t rategy ; - Carry ing out workshops for MENFP national and departmental staf f on the new data collection process. processes (SAB ER - EMIS); - R evised data collection protocol; - I nstruments and new protocols for d a ta colle ction and verification; - Revised school mapping tools (aligned wi th the data collection protocols) , presented to MENFP; - W orkshop on the revised strategy with ME NFP nat ional and departmental staff. strat egy for the school census , which d elivers quality data comparable t o international standards New school census data collection process officially adopted by MENF P ( PDO I nd icator 1) Production of high - quality data for improved e v idence - b ased policy and decision - makin g. - Carry ing out a diagnostic of th e current statistical infrastructure, information management processes and data utilization practi c es; - Carry ing out a study tour for MENFP staff to learn from other countries’ e x perience on setting up the EMIS; - Design ing , review ing , adopting and diss em inating a strategy for strengthening the M&E system of the MENFP. - Diagnostic on the statistic a l infr astructur e, information management processes , and practices developed wit h t he suppo rt of technical experts and circul ated; - Study tour in Burkina Fa so for MENFP managers and experts; - Revised strategy and action plan for the EMIS developed, cir c ulated , and adopted. Designing a MENFP strategy for strengthening its monitoring a nd evalu ation system Strategy for strengt hening the M&E system officially ad opted by MENFP (PDO I ndicator 2). - Workshops and trainings for MENFP staff on the customizat i on, po pulation, and maintenance of databases; survey design; data collection; dat a managem ent; information - Technic a l supp ort for the establishment of secure connectivity and intranet at the subnat i onal and central levels of the MENFP; I nitiating the implem enta tion of the M&E strategy Technic a l prerequisites for a The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 12 of 31 dissemination; an d data analysis for policymaking. - Required resources identified in the strategy document for the EMIS are established. - St udent and school databases struct ur ed; - Single identification numbers created; - Databases from externally funded project integrat e d in t he EMIS; - Training program developed for the MENFP’s technician s and develo p ers on t he EMIS. functional EMIS are in place (PDO I ndicator 3) . Critical assumptions: • The GoH commits to revis ing its data collection process and strengthen the use of the EMIS for decision - making; • Trained tech nical staff t o appl y their new s kill s in data collection, management and analysis; • Key sector stakeholders, including development pa rtner s, support and coordina te their interventions on the EMIS. II. OUTCOME Asse s sment of Achievement of E ach Objective/Outc ome 24. This section assesses Proje ct performance in relation to the PDO. The PDO contained three parts, as follows: Support the MENFP in: i. Ob jective 1: E stablishing a sustainable data collection strategy for the sc h ool censu s which delive rs quality data compa rable to international standard s ii. Objective 2: D esigning a MENFP strategy for strengthening its monitoring and evaluation system iii. Ob j ective 3: Initiating implementation of said strategy 25. Each outcome is evaluated s eparately to arrive at an overall outcome ra ting in the next section. O bj ective 1: Establish a sustainable data collection strategy for the school census which delivers qu ali ty data comparable to international standards The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 13 of 31 26. Achievement of PDO1 was measure d by PDO1 Indicator “New school census data c ollection process officially a d opted by MENFP . ” The baseline was “No , ” and the end target “Yes . ” The PDO1 Indicator was achieved and , as ou tlined below, additional data /observations suggest that the established d ata colle ction strategy is sustainable and w ill deliver quality education s ystem data . 27. T he Project successfully implemented interventions that led to the adoption of a revi se d susta inable school census data collection strate gy. In particular, the MENFP o rganized , with Project support, a workshop i n October 2020 to present the n ew data collection strategy as the backbone of decision - maki ng and strategic - p lannin g p rocesses for t he sect or . In April 2020, the General Director sent an official letter to the W o rld Bank, approving the EMIS operation al pla n including the new data colle c tion strategy as the first pillar of the EMIS pla n . 28. T his revised data collection strategy aims a t r educing data gaps that limit the Ministry’s ability to conduct data - driven deci s ion - makin g. T o do so, the strategy include s processes to collect the follo w ing data: (i) student s administrative data by age, gender and grade along with a single tracking n umb er ; (ii ) staff administrative data by grade and gender along with a single trac k ing numbe r , and (iii) s chool administrative data, such as school type alon g with a unique tracking number as well . The strategy include s a tim eline for the design of the dat a c ollecti on form s and a description of the consultations that will need to be org a nized wit h relevant stakeholders. The forms will then be piloted to determ i ne whether data providers easily comprehend the language and framin g of the questions 5 . I n addition, the str ategy includes d ata quality assurance mechanis m s both at the local and central levels to ensure that the quality of the data collected is complet e, accurate, clear and properly disaggregated . V erification workshops will be orga n ized th e week followi ng t he data collection campaign . During these worksho p s, school inspectors familiar with the schools , will review data school by school in order to mini mize human errors. At the end of the data collection campaign , r aw data on the edu c ation sys tem will be availab le at the Minist ry level, and aggregate data o n enrollment and completion rate, as well as teachers, will be available to various stakeholders to offer a data - based perspective on education policy . 6 This will encourage part icip a tion at a ll levels of the sy stem a nd foster commitment of sector stakehold ers. The strategy also includes mechanism s to involve the whole education system in a comprehensive , structured, and systemic manner during the preparation stage, the data collec tio n itself, the data val idation , the compilatio n , and the dissemination stage s. 29. The new approach is considered sustainable because it was designed to be executed on a yearly ba sis, using the MENFP’s own resources in a cost - effective way. In addition, it do e s build u pon a solid system architecture, a sound policy framework for th e E MIS (developed under component 2 of the Project), a strong organizational structure and includes st rategies to improve human resources capacity . While the first yea rs of the EMIS p lan w ill require cont inued i nitial investmen t from development partners, the action plan is designed to be taken over by the MENFP in five years, and select activities are alr eady being financed by t he Public Treasury . D evelopment partners are now also 5 A prelimina ry step for the data collection , described in the revised strategy , will be to mer g e and cle an existing school data set s in or der to put together a baseline fo r the data collection for each department. Then, in order to ensure that data i s collected in a tim ely and efficient manner, data will be collected at the local level by th e decentr a lize d depa rtment of education and data en try wil l also be done locally, fee d ing directly in the EMIS database . The data collection will leverage the use o f mobile tools, and training sessions are already planned for MENFP staff and collecting agents on the ground . 6 Access level and anon ymit y rul es will be defined at a later stag e. The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 14 of 31 al i gning the ir intervent ions to finance this an nual action plan, which will ult imately strengthen alignment of sector interventions on data collection. The data collection itself will be implemented as part of the first component of the PEES Project, that wi l l finance technical a ssistan ce, training, an d logistics costs for sub - nat ion al levels of the Ministry to adapt and carry said data collection strategy and carry the first revi sed school census process. 30. Activities carried out under t he Project toward thi s objectiv e include: i. T he re vis ion of the MEN FP school census data collec tio n strategy (with the support of a local consultant firm) and establishment of a new strategy using new technologies ; ii. W orkshops to consult with key stakeholders and the validat ion of the st rategy ; and iii. T he de sign of a trainin g program for MENFP staff on ne w data collection processes . Specifically, the training program includes modules for back - end compu ter engineer s to ensure that MENFP staff is able to exchange databases between d i fferent a pplications once da ta from the schoo l census is collected and fe d i nto the EMIS . 31. T h ese achievements are particularly relevant in a context where the dialogue betwee n development partners and the MENFP on the improvement of the school census pro c ess had s talled since 2016, the year during w hich the last school census was implemented . After 2016 , external partners stopped financing the school data collection process , w hich was considered neither cost - efficient nor sustainable . Meanwhile, the MENFP did not h ave the nati onal re sources to fund t he school census, so both th e s ector dialogue on education system data collection and school census activities were interrupte d . U nder Component 1 , t he Project ’ s also supported the creation of a National Techni c al Commit tee for the EMIS, w hich oversee s the data collection and EMIS st rat egy , as well as workshops to finalize and disseminate the strategy . Associated intermediate result s indicators (IRIs ), namely “IRI1: Diagnostic of existing data collection process e s for the school cens us” and “ IRI3: Design of the training content and ma ter ials on the new school data collection process completed , ” were also achieved. 32. Other outcomes of the Project under this component include (i) the revamping of a policy and a tec h nical dia logue on dat a quali ty, collection an d use; and (ii) strengthen in g the governance structure overseeing the EMIS, making the data collection process an integral part o f the MEFP’s information system and reducing silos between technical directorates (specific ally the Inf ormatio n System Unit and the Directorate for Statist i c s and External collaboration, in charge of the school census). O bjective 2 : D esign a MENFP strat egy for strengthening its monitoring and evaluation system 33. Achievement of PDO2 wa s measured by PDO2 Ind icator “Strategy for str engthening the M&E system of ficially adopted by MENFP . ” The baseline was “No” and the end target “Yes , ” which was achieved by P roject closing. 34. The strategy for the EMIS encompasse d a diagnostic of the EMIS in f rastructu re , carried in 2018 , that helped ass ess Haiti ’s specific needs i n terms of data and select the required education modules for the system . Following the diagnostic , a framework document for the EMIS was developed in 2019 and , focus ed on four key d i mensions for improvem ent nam ely: ( i ) technolo gical infrastructure; ( ii ) d ata application management; ( iii ) data collection; and ( iv ) data governance . The system will ensur e The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 15 of 31 compatibility with the Ministry’s databases, integrability and security of informa t ion. In t erms of soft ware , t he decision was m ade to develop an in - house E MIS solution to facilitate adoption by all parties and ensure buy - in and sustainability at all lev els of the Ministry. This choice will ensure financial accessibility, avoid recurren t hidden m aintenance c osts , a nd allow a higher degree of customization. I t will also ensure that the system is integrated with other government systems such as human resour ce management systems and reduce security breaches. In order to be sustainable and e n sure succ essful adopt ion wit hin the Ministry , the EMIS will integrate pr e existing smaller sub - systems and align existing processes in a single system. The system will use different data centers, and data will be replicated in real tim e in the cloud. An ex t ernal ven dor will man age one of the physical servers on its own premises in order to ensure a secure additional data backup mechanism (Haiti being subject to natural disas ters, data could easily be lost without a solid contingency str ategy ). Data security will be e nsured throu gh the implementation of a virtual private network ( VPN). The plan also includes the procurement of security equipment, the improvement of workstation s, the acquisition of a cloud server, training for MENFP staff and the school data c e nsus desc ribed in com ponent 1. An EMIS budget has also been established , carefully assessing the financial requirements of the EMIS and maintenance costs as well. Finally, t he EMIS will be fully decentralized, with special attention t o local capacity buil d ing at th e subnationa l level s of the Ministry . 35. In terms of data, t he EM I S will include data on enrollment, attendance, graduation rates, collected through the revised pro cess supported under component 1 of the Project and assessments and administrative s t atistics as well . The new EM IS will better su pport the Ministry’s M&E sy s tem by providing data on enrollment, learning, teachers, fi nance and resource allocation ( thr ough the analysis of enrollment, f inancial, hu man resources, facilities, and learning assessment data ) with the objective of m onitoring the performance of the education system and produce in ternation ally - comparable Education Data . Special attention will be given to the mo nitoring of teachers’ deployment and the calculation of student s ’ gross enrollment rates. 36. Act ivities carried out related to th is objective include: i. T wo technic al consultancies to support the develop ment of a policy document and a strategy to i mplement an improved EMIS, both approved by th e MENFP ; ii. A workshop in Dec ember 2018 to disseminate the st rategy with sector partners and t rain MENFP staff on data managemen t , during which approximately 100 participants exchanged about their data needs and experienc es and learned about the current stat e of the EMIS and the MENFP priorities in terms of Informat ion Systems ; iii. A South - South knowledge sharing exchange for the Nati onal Technical Team on the EMIS in Burkina Faso in 2017 . 7 37. Intermediate result s in d icators r elated to this objective and C omponen t 2, namely IRI4 “Diagnosti c of the current statistical inf rastructure, information management processes and data utilization practi c es in the MENFP ” and IRI5 “Strategy for strengthening M&E system designed” w e re also a chieved. 7 This exchange was m anaged by the Af rican education progra m ADEA (Assoc iati on for th e Development of Education in Africa) and co - financed with the Global Partnership for Ed ucation. The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 16 of 31 38. The achievement of PDO2 al so represents a key milesto ne toward improved governance , a s t he MENFP now has framework documents at its disposal to imp rove its M&E functions and implement an efficient EMIS. These key documents also allow d e velopment partners to better align their finan cing based on the EMIS acti on plan. Overall, the Project ha s been instrumental to improv ing the EMIS , and thus the MENFP’s M&E capacities. Objective 3 : Initiate implementation of the MENFP monitoring and eval u ation str ategy 39. The Project’s third objective was to start implementing the EMIS strategy. Achievement o f t his objective was captured by the PDO3 indicator “T echnical prer equisites for a functional EMIS are in place . ” The baseline was “No , ” and the end t a rget of “ Yes” was achieved by Project closing. This objective was support ed by C omponent 3 activities, wi th achievements that fall under three main categories : i. Support to the EMIS T echnical C ommittee to implement the roadmap for the EMI S through the hir i ng of a s enior consultant providing day - to - day support to MENFP technical directors and coaching in the a reas of planning and project management ; ii. Support to implement techn ical prerequisites, including the establishment of secure connectivi ty and intranet in centra lized and decentralized levels of the MENFP , and consolidation o f student and school databases t hrough the hiring of seven technical consultants ; iii. W orkshops on the EMIS roadmap for MENFP staff and data management. The roadmap to operationalize th e EMIS str ategy covers the next five years and i ncludes a timeline for imp lementation and specific objecti ves and expected results for each dimension of the plan. 40. As part of the Project, s chool and students’ modules’ technical specifications have been de f ined by t he EMIS technical team and validated b y the National Committee f or the EMIS . Databases are curre ntly being populated with existing data and will be completed with the upcoming school census. Protocol to develop unique student and schools tracking number ar e also being finalized. Finally, a se c ure connection to internet and the Ministry’s intranet has been established in the Ministry’s subnati onal levels offices. Tra ining of the local Ministry’s staff was also provided, contributing to the achievem e nt of IRI 6 “Training for MENFP on databases, s u rvey design, data collecti on, data management, information dissemination and/or data analysis complet ed . ” 41. By establishing a function ing inter - directorate committee , the National Technical C ommittee for the E M IS, compo sed of members of different MENFP dir e ctorates , with a clear vi s ion and leadership structure, th e Project will serve as a model for other education system interven tions, including the Promoting an Efficient Education System in Haiti (PEES) Projec t financed by the GPE and whose grant agent is t he World Bank. As part of t he PEES Project, technical team s will be formed following the EMIS technical committee arrangement s in the cross - cutting areas of learning assessments and accreditation . P ioneering t hese fram ework documents for the EMIS, the MEN F P could also serve as a m o del within the broader Haitian g overnment for developing and implementing a strategic approach to o fficial statistics. Overall Outcome Rating Rating: Satisfactory The Wo rld Bank Haiti Statistical Capacity Building in Education Grant (P161160) Page 17 of 31 42. A l l three PDO in d icators a nd all IRIs were achieved . Given t hat t he operation fully achie ved its intended outcomes , with m inor shortcomings in terms of delays and sustainability , the overall efficacy of the Project is consid ered S u b s t a n t i a l . The achievement of P roject objectives is mainly attributable to the activiti es suppo rted by t h e Project . As t his is a small grant Imp lem entation Completion and Result s Report (ICR) , there is no rating on Efficiency nor Relevance ( whic h ar e combined with Efficacy ratings to calculate the overall Outcome ratings in ICRs for larger a mounts of financi n g ) . Howev er, on e can note that : (i) t h e co sts involved in achieving the Project’s objectives were deemed reasonable in comparison to the be nefits and against recognized norms (“value for money”) ; and that (ii) the objectives of the Projec t were fully rele v ant to th e Hait i an education system , a s we ll as the World Bank ’s strateg y . In particular, the costs were reasonable compared to market pric es and similar activities ’ costs under other investment projects in the sector. Additionally, the P roject’s objectiv e s to stre ngthen data collection and th e