Projet d'Apprentissage, d'Évaluation et d'Analyse - II (LEAP-II) Rapport Final

DISCLAIMER This publication was produced for review by the United States Agency for International Development. It was prepared by International Development Group, LLC. LEARNING, EVALUATION, AND ANALYSIS PROJECT-II (LEAP-II) FINAL REPORT PHOTO CREDIT: MELISSA STONE LEARNING, EVALUATION, AND ANALYSIS PROJECT-II (LEAP-II) Final Report Contract No. AID-OAA-I-12-00042, Task Order No. AID-OAA-TO-14-00046 Cover photo: Entrepreneur in Sierra Leone DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States government. CONTENTS EXECUTIVE SUMMARY 1 INTRODUCTION 1 ACTIVITIES 1 PERFORMANCE EVALUATIONS 1 #1 – LIBERIA MUNICIPAL WATER PROJECT (LMWP) MID-TERM EVALUATION 1 #2 – PARTNERSHIP FOR GROWTH (PFG) MID-TERM EVALUATION 3 #3 – FINAL PERFORMANCE EVALUATION OF THE LIBERIA ENERGY SECTOR SUPPORT PROGRAM (LESSP) 4 #5.A EVALUATION OF THE BEYOND ADVOCACY FUND (BAF) IN SOUTH AFRICA 5 #7 – FINAL EVALUATION OF THE USAID ME/TS IMPROVING WATER AND SANITATION SERVICES IN THE MIDDLE EAST AND NORTH AFRICA REGION (IWSMR) ACTIVITY 7 #10 – MID-TERM PERFORMANCE EVALUATION: LIBERIA INVESTING FOR BUSINESS EXPANSION PROJECT (IBEX) AND SUSTAINABLE MARKETS INITIATIVE – LIBERIA (SMI-L) 8 #17 – FINAL PERFORMANCE EVALUATION OF THE “TACKLING YOUTH EMPLOYMENT IN TUNISIA” PROJECT (MASHROU3I) 9 #28 – MID-TERM PERFORMANCE EVALUATION: USAID/INDIA COUNTRY DEVELOPMENT AND COOPERATION STRATEGY DEVELOPMENT OBJECTIVE 4 11 #29 – FINAL PERFORMANCE EVALUATION OF THE FURTHER ADVANCING THE BLUE REVOLUTION INITIATIVE (FABRI) 19 #30 – MID-TERM PERFORMANCE EVALUATION OF THE BIZ+ PROJECT 20 IMPACT EVALUATIONS/LARGE HOUSEHOLD SURVEYS 23 #4 AND #14– SENEGAL YAAJEENDE MIDTERM EVALUATION AND KOLDA REGION BASELINE IMPACT EVALUATION 24 #11 – LIBERIA FEED THE FUTURE ZONE OF INFLUENCE INTERIM ASSESSMENT 27 #15 SENEGAL FEED THE FUTURE ZONE OF INFLUENCE INTERIM ASSESSMENT 29 #25 LIBERIA FOOD AND ENTERPRISE DEVELOPMENT (FED) IMPACT SURVEY 31 ECONOMIC ANALYSES 35 #0.A – CARGO PREFERENCE ANALYSIS 35 #5.A – ANALYSIS TO DEVELOP A TRADE INTERVENTION AGENDA: PRIORITIZING COUNTRIES AND ACTIVITIES IN SOUTHERN AFRICA 35 #25.B – LIBERIA CROSS BORDER TRADE ASSESSMENT 37 #27.A – TIMOR-LESTE CUSTOMS ASSESSMENT AND DESIGN 39 #27.B – TIMOR-LESTE IMPORT AND EXPORT REGULATIONS ADMINISTERED BY GOVERNMENT AGENCIES 41 #13 AND #16 – AGRICULTURAL COMMERCIAL, LEGAL, AND INSTITUTIONAL REFORM (AGCLIR) DIAGNOSTICS IN SIERRA LEONE AND GUINEA 43 #33 – ANALYSIS OF THE USAID-CILSS INTERREGIONAL TRADE MONITORING DATA 49 COST-BENEFIT ANALYSIS (CBA) 53 CBA METHODOLOGY 53 #18 – COST-BENEFIT ANALYSIS OF USAID/SENEGAL’S RICE VALUE CHAIN 55 #19 – COST BENEFIT ANALYSIS OF USAID/RWANDA’S DAIRY VALUE CHAIN INTERVENTION 56 #20 – COST BENEFIT ANALYSIS OF USAID/LIBERIA’S RICE AND GOAT VALUE CHAIN INTERVENTIONS 58 #21 – COST BENEFIT ANALYSIS OF USAID/MALI’S SORGHUM AND MILLET VALUE CHAIN INTERVENTIONS 60 #23 – FEED THE FUTURE COST-BENEFIT ANALYSIS SYNTHESIS REPORT 63 #26 – INTEGRATING GENDER IN COST-BENEFIT AND COST-EFFECTIVENESS ANALYSIS 67 #6 – HAITI: INCREASING THE EFFICIENCY FOR PUBLIC INVESTMENT PROJECT 70 #12 – KENYA PROGRAM ON INTEGRATED INVE STMENT APPRAISAL AND RISK ANALYSIS71 #31 – COST-BENEFIT ANALYSIS-COST-EFFECTIVENESS ANALYSIS OF USAID ACTIVITIES IN NIGERIA AND ZIMBABWE 73 #32 – CBA OF USAID RESILIENCE IN THE SAHEL ENHANCED 84 OTHER ACTIVITIES 90 #0.B – LOGISTICAL SUPPORT TO THE SECOND ECONOMISTS REGIONAL WORKSHOP 90 #8 AND #24 – ADVISORY SERVICES TO THE POWER AFRICA COORDINATOR’S OFFICE 90 LIST OF FIGURES FIGURE 1: ACCELERATING INNOVATION, TRANSFER, AND DIFFUSION IN DEVELOPING COUNTRIES ..................................................................................................................................................................... 14 FIGURE 2: INNOVATION PROXIMITY ................................................................................................................... 19 FIGURE 3: ECONOMIC AND FINANCIAL ANALYSIS ....................................................................................... 53 FIGURE 4: MAP OF FTF PROJECTS FOR CBA SYNTHESIS (2015 VC ONLY) ............................................. 63 FIGURE 5: INCREMENTAL ECONOMIC NET PRESENT VALUE (ENPV) ..................................................... 64 FIGURE 6: INCREMENTAL ECONOMIC NET PRESENT VALUE (ENPV) (USAID PERSPECTIVE) BY VALUE CHAIN ................................................................................................................................................................ 65 FIGURE 7: POVERTY ALLEVIATION OF REGIS-ER’S AND REGIS-AG’S INTERVENTIONS ................... 87 LIST OF TABLES TABLE 1: LEAP-II ACTIVITIES BY TYPE ..................................................................................................................... 1 TABLE 2: COUNTRIES, MARKETS, AND COMMODITIES COVERED BY THE USAID-CILSS TRADE DATA. ................................................................................................................................................................................ 50 TABLE 3: PCE CBA IMPACT FIGURES ..................................................................................................................... 55 TABLE 4: RDCP II CBA IMPACT FIGURES ............................................................................................................. 57 TABLE 5: INCREMENTAL ANALYSIS OF FED PROJECT (USAID PERSPECTIVE) ...................................... 58 TABLE 6: ARDT_SMS CBA IMPACT FIGURES ...................................................................................................... 62 TABLE 7: CBA SYNTHESIS HIGH-LEVEL FIGURES .............................................................................................. 64 TABLE 8: INCREMENTAL ECONOMIC NET PRESENT VALUE (USD) ......................................................... 74 TABLE 9: INCREMENTAL ECONOMIC NET PRESENT VALUE (USD) ......................................................... 82 LIST OF ACRONYMS ACWUA Arab Countries Water Utility Association AEO Authorized Economic Operators AgBEE Agriculture business enabling environment AgCLIR Agricultural Commercial, Legal, And Institutional Reform AGUIPEX Agence Guinéenne de Promotion des Exportations AI Artificial insemination ASYCUDA Automated System for Customs Data ATE Average treatment effect BAF Beyond Advocacy Fund BCR Benefit-cost ratios BFS Bureau for Food Security BLSA Business Leadership South Africa BSO Business services organization CA Constraints analysis CBA Cost-benefit analysis CADS Cluster Agricultural Development Services CAPI Computer Assisted Personal Interviewing CAQDAS Computer assisted qualitative data analysis CBL Central Bank of Liberia CBSPs Community-Based Solution Providers CBT Cross-border trade CD Customs Directorate CDCS Country Development Cooperation Strategy CEA Cost-effective analysis CF Conservation Farming CIEPEX Centre International d’Echa nges et de Promotion des Exportations CILSS Permanent Interstate Committ ee for Drought Control in the Sahel CIP Capital improvement project COR Contracting Officer Representative CRDH Centre de Recherche pour le Developpement Humain CRI Cambridge Resources International, Inc. CRP Customs Reform Project CTPEA Centre de Techniques de Pl anification et d'Économie Appliquée CWG Citizen Work Group DALY Disability adjusted life year DCA Development Credit Authority DD Difference-in-difference DEC Development Experience Clearinghouse DO Development Objective E3 Economic Growth, Education and Environment ECOWAS Economic Community of West African States ENAF Ecole Nationale d’ Administration Financiere ENPV Economic net present value EP Office of Economic Policy ERR Economic Rate of Return EVD Ebola Virus Disease FABRI Further Advancing the Blue Revolution Initiative FCT Federal Capital Territory FED Food and Enterprise Development FGDs Focus group discussions FNPV Financial Net Present Value FRC Fiscal Reform Commission FTCC Fast Track Commercial Court FTF Feed the Future FTFZ-CD Feed the Future Zimbabwe Crop Development FTFZ-LD Feed the Future Zimbabwe Livestock Development GAHP Good Animal Husbandry Practices GAP Good Agricultural Practices GDA Global Development Alliance GOH Government of Haiti GOI Government of India GOG Government of Ghana GOK Government of Kenya GOM Government of Mali GON Government of Nigeria GOS Government of Senegal GOSL Government of Sri Lanka GOT Government of Tanzania IBEX Investing for Business Expansion Project ICBT Informal cross-border trade ICER Interventions Combined In cremental Cost-Effectiveness Ratios IDG International Development Group LLC (IDG) IRB Institutional Review Board IRR Internal Rate of Return JCAP Joint Country Action Plan HH Household HP Hewlett Packard IWSMR Improving Water and Sanitation Serv ices in the Middle East and North Africa Region KII Key informant interview KSG Kenya School of Government LEAD Linkages for Economic Advancement of the Disadvantaged LEAP-II Learning Evaluation and Analysis Project-II LESSP Liberia Energy Sector Support Project LGA Local Government Areas LMWP Liberia Municipal Water Project LSA Liberia Strategic Analysis M&E Monitoring and evaluation MARAD US Maritime Administration MCC Milk Collection Centers ME/TS Middle East Bureau/Technical Services MENA-NWC Middle East and North Africa Network of Water Centers of Excellence MIE Midterm impact evaluation MOF Ministry of Finance MPEC Ministry of Planning and External Cooperation MRU Mano River Union MUAC Mid-upper arm circumference NRVCC Nutrient-rich value chain commodity NRW National Single Window NUPAS Non-US Organization Pre-Award Survey O&M Operation and maintenance OAA Office of Acquisition and Assistance ODK Open Data Kit OGAs Other Government Agencies OHADA Organisation pour l'Harmonisat ion en Afrique du Droit des Affaires PBS Population-based surveys PCE Economic Growth Project PFG Partnership for Growth PICS Purdue Improved Crop Storage PIO Public International Organization PIP Public Investment Program PPA Public private alliance PPP Public-private partnership PSM Propensity score matching QED Quasi-experimental designs QPI Qualitative Personal Interviewing RAF Risk assessment framework RD Regression discontinuity RDCP Rwanda Dairy Competitiveness Program II REGIS-AG Resilience and Economic Growth in the Sahel – Accelerated Growth REGIS-ER Resilience and Economic Gro wth in the Sahel - Enhanced Resilience RISE Resilience in the Sahel Enhanced ROCTP Regional Operator Certification and Training Program SBS Business support organizations SEforALL Sustainable Energy for All SILC Savings and Internal Lending Communities SME Small and medium enterprise SMI-L Sustainable Markets Initiative – Liberia TCLP Total Consumption Poverty Line TFA WTO Trade Facilitation Agreement TLTIP Timor-Leste Trade Information Portal TOT Training of trainers TVET Technical Vocational and Education Training UEMOA West African Economic and Monetary Union UNIDO United Nations Industrial Development Organization USAID United States Agency for International Development VC Value chain VE/VA Value engineering/value analysis WASH Water, sanitation and hygiene WCO World Customs Organization WEAI Women’s Empowerment in Agriculture Index WCO World Customs Organization WCS Warrantage credit scheme ZIMSTAT Zimbabwe National Statistics Agency ZOI Zone of Influence 1 | LEAP-II FINAL REPORT USAID.GOV EXECUTIVE SUMMARY PROJECT OVERVIEW The Learning, Evaluation, and Analysis Project-II (LEAP-II) provided economic analysis and monitoring and evaluation support services to satisfy the overall operational objectives of USAID’s Bureau for Economic Growth, Education and Environment (E3). LEAP-II was a $14m project with a period of performance of October 2014-September 2018. The objective of the project was to provide a mechanism for the Office of Economic Policy in E3 (E3/EP) and for Missions and Bureaus to increase access to rigorous, independent technical evaluation and analysis wi thin and beyond USAID. LEAP-II provided a vehicle for USAID Washington and USAID Missions to contract for analysis services to be executed for projects within their economic growth or other portfolios. LEAP-II was structured as a contract mechanism implemented through buy-ins.1 Services under LEAP-II’s scope of work included:  Cost-benefit/cost-effectiveness analysis  Macroeconomic and financial analysis  Single- and multi-country evaluations  Randomized controlled trials (RCTs)  Policy Analysis Matrix/Domestic Resource Cost Analysis  Constraints Analysis/Inclusive Growth Diagnostics  Other Analytical and Project Support  Integrating Gender Equality and Female Empowerment In addition to the activities listed above, LEAP-II provided for the presentation and dissemination of analysis findings and lessons to help guide the design, implementation, and subsequent evaluation of economic growth and other sectors’ activities. LEAP-II also provided a vehicle for USAID Missions to support capacity building of partner government and civil society counterparts. Mission and Bureaus interested in engaging LEAP-II services submitted a SOW to the Contracting Officer Representative (COR). The COR determined if the proposed activities were within the SOW of 1 A buy-in is defined as a funds-transfer to LEAP-II during a fiscal year where the buy-in objectives are consistent with LEAP-II objectives and Mission use of the mechanism will advance common strategic objectives. USAID.GOV LEAP-II FINAL REPORT | 2 the contract, and if so, provided the SOW to IDG. IDG would then prepare and submit to the designated Activity Manager at the Mission or Bureau and COR a business proposal including a brief technical approach, work plan, budget, and the CVs for the proposed team. Business proposals were submitted to the Activity Manager, COR, and CO for approval. Once the approvals were in place, IDG would quickly mobilize the team and launch the activity, often within just two weeks of receiving approval. PROJECT RESULTS LEAP-II was immensely popular and by year 3 could no longer accommodate new buy-ins. Thirty-six activities in 30 countries were implemented over the four years of the project including 15 performance and impact evaluations and 21 economic analyses covering a wide range of technical areas: water, energy, trade policy and trade facilitation, agriculture and agribusiness, nutrition, access to finance, SME development, youth, gender, infrastructure, workforce development, education, and health. (See table 1 below for LEAP-II work orders by activity time.) LEAP-II’s key achievement was generating evidence for development. LEAP-II activities contributed to improving the effectiveness of USAID programming through generating evidence and actionable recommendations for its use to inform policy and programming decisions for maximum impact. Learning was at center of LEAP-II activities and the majority included learning events such as presentations and workshops so results yielded change. LEAP-II services helped Bureaus and Missions apply USAID Program Cycle Principles and increased compliance with the USAID Evaluation Policy (January 2011, updated in October 2016) and ADS 201 (revised in September 2016). LEAP-II also helped Bureaus and Mission to manage resources effectively by providing the data and information required to align resources against country strategies, make difficult trade-offs, and re-allocate resources toward activities that are demonstrating meaningful results. LEAP-II deliverables presented findings, conclusions, and recommenda tions that are based on facts, evidence, and data, and supported by quantitative and qualitative information that is reliable, valid and generalizable. Deliverables included: an introduction; a full description of the methodology used; findings and results; limitations; conclusions and inferences; and options or recommendations, as appropriate. Recommendations were specific, action-oriented, and scalable and with defined responsibility for the action allowing USAID to integrate key findings into strategic plans and program management decisions. LEAP-II deliverables can be found on the Development Experience Clearinghouse (DEC) under contract no. AID-OAA-I-12-00042/AID-OAA-TO-14-00046. 1 | LEAP-II FINAL REPORT USAID.GOV TABLE 1: LEAP-II ACTIVITIES BY TYPE PERFORMANCE EVALUATIONS IMPACT EVALUATIONS/ LARGE HOUSEHOLD SURVEYS ECONOMIC AND TRADE ANALYSES COST-BENEFIT ANALYSIS OTHER ACTIVITIES #1 – Liberia Municipal Water Project (LMWP) Mid-term Performance Evaluation #4 – Senegal Yaajeende Midterm Impact Evaluation #0.b – Cargo Preference Analysis #6 – Haiti: Increasing the Efficiency for Public Investment Project #0.a – Logistical Support to The Second Economists Regional Worksho p #2 – Partnership for Growth (PFG) Mid- term Evaluations #11 – Liberia Feed the Future Zone of Influence Interim Assessment #5 – Analysis to Develop a Trade Invention Agenda in Southern Africa #12 – Kenya Program on Integrated Investment A pp raisal and Risk Anal y sis #8 – Advisory Services to the Power Africa Coordinator’s Office #3 – Final Performance Evaluation of the Liberia Energy Sector Support Program (LESSP) #14 – Senegal Yaajeende Kolda Impact Evaluation Baseline #13 – Agricultural Commercial, Legal, And Institutional Reform (AgCLIR) Diagnostics in Sierra Leone #18 – Cost-Benefit Analysis of USAID/Senegal’s Rice Value Chain Intervention #24 – Advisory Services to the Power Africa Coordinator’s Office II #5.a – South Africa – Evaluation of the Beyond Advocacy Fund (BAF) #15 – Senegal Feed the Future Zone of Influence Interim Assessment #16 – Agricultural Commercial, Legal, And Institutional Reform ( A g CLIR ) Dia g nostics in Guinea #19 – Cost-Benefit Analysis of USAID/Rwanda’s Dairy Value Chain Intervention #7 – Final Evaluation of the USAID ME/TS Improving Water and Sanitation Services in the Middle East and North Africa Re g ion ( IWSMR ) Activit y #25 – Liberia Food and Enterprise Development (FED) Impact Survey #25.a – Liberia Cross-Border Trade Assessment #20 – Cost-Benefit Analysis of USAID/Liberia’s Rice and Goat Value Chain Interventions #10 – Mid-term Performance Evaluation: Liberia Investing for Business Expansion Project (IBEX) and Sustainable Markets Initiative – Liberia ( SMI-L ) #27.a – Timor-Leste Customs Assessment and Design #21 – Cost-Benefit Analysis of USAID/Mali’s Sorghum and Millet Value Chain Interventions #17 – Final Performance Evaluation of “Tackling Youth Employment in Tunisia” #27.b – Timor-Leste Import and Export Regulations Administered by Government A g encies #23 – Feed the Future Cost-Benefit Analysis Synthesis Report #28 – USAID/India Country Development and Cooperation Strategy Development Objective 4 Mid-term Performance Evaluation #33 – Analysis of the USAID- CILSS Inter-Regional Trade Monitoring Data #26 – Integrating Gender in Cost-Benefit and Cost- Effectiveness Analysis #29 – Final Performance Evaluation of the Further Advancing the Blue Revolution Initiative (FABRI) #31 – Cost-Benefit Analysis-Cost-Effectiveness Analysis of USAID Activities in Ni g eria and Zimbabwe USAID.GOV LEAP-II FINAL REPORT | 2 TABLE 1: LEAP-II ACTIVITIES BY TYPE PERFORMANCE EVALUATIONS IMPACT EVALUATIONS/ LARGE HOUSEHOLD SURVEYS ECONOMIC AND TRADE ANALYSES COST-BENEFIT ANALYSIS OTHER ACTIVITIES #30 – Mid-term Performance Evaluation of the BIZ+ Project #32 – Cost-Benefit Analysis of USAID Resilience in the Sahel Imitative #9 – Sierra Leone and Guinea Cost Benefit Analysis Activity was canceled. #22 – Cost-Benefit Analysis of USAID/Malawi’s Value Chain Interventions was canceled. 1 | LEAP-II FINAL REPORT USAID.GOV INTRODUCTION This report presents summaries of the 36 activities implemented under LEAP-II. The report is divided into the following sections: Performance Evaluations, Impact Evaluations, Economic and Trade Analyses, Cost-benefit Analysis, and Other Activities. Each section begins with a brief description of LEAP-II’s approach. Each activity summary includes the following sub-sections: Introduction, Methodology, Key Results, and Learning. ACTIVITIES PERFORMANCE EVALUATIONS LEAP-II conducted 10 performance evaluations oriented toward methodological rigor, utility to USAID, and cost effectiveness. LEAP-II applied a set of six quality standards – utilizing a set of checklists and rating/review tools – to ensure close attention to quality at each step of an evaluation: 1) ensuring a clear statement of purpose regarding a given evaluation; 2) defining or refining evaluation questions directly linked to the evaluation purpose; 3) developing an evaluation design matrix that ensures evaluation questions are effectively addressed; 4) developing high-quality data collection instruments that are linked to outcomes and indicators of interest; 5) employing a range of approaches to data analysis and data visualization to allow for the identification of important trends and patterns in the collected data, thereby ensuring that evaluation findings drive conclusions and conclusions inform recommendations; 6) completing all evaluation products – reports, debriefs, recommendation work plans – with a special focus on quality and utilization. The majority of performance evaluations utilized mixed methods research with data collection involving a desk review , key informant interviews (KIIs), focus group discussions (FGDs), online surveys, and site visits. #1 – LIBERIA MUNICIPAL WATER PROJECT (LMWP) MID-TERM EVALUATION INTRODUCTION The primary objective of LMWP was to support the design, tendering, execution and operation of water supply infrastructure improvements in the three Liberian county capitals – Robertsport (Grand Cape Mount County), Sanniquellie (Nimba County), and Voinjama (Lofa County). The LMWP implementing partner, Tetra Tech, was tasked with assisting local and national authorities in developing plans for water supply and sanitation improvements, overseeing construction of this capital improvement project (CIP), supporting initial operations of water supply infrastructure improvements, and establishing local capability to sustainably operate and maintain constructed water supply systems. LEAP-II conducted a mid-term evaluation to determine whether LMWP was meeting its objectives, and if adjustments were needed as USAID advanced plans to begin capital works in line with the plans developed by LMWP for the three cities. The evaluation addressed the following three key areas: (1) Effectiveness of infrastructure planning/construction oversight approach, (2) Effectiveness of institutional framework and capacity building, and (3) Overall project positioning and strategy for phase-out of USAID assistance. METHODOLOGY LEAP-II’s approach to the mid-term evaluation included a desk review, key informant interviews, and a field visit to Liberia from November 24 to December 15, 2014. The evaluation team reviewed LMWP’s deliverables and cross referenced them against the expected deliverables as outlined USAID.GOV LEAP-II FINAL REPORT | 2 in the Objectives and Scope of Work. The team conducted over 20 key informant interviews and consulted over 15 organizational stakeholders. Field activities included visits to the cities of Robertsport, and Kakata. The evaluation was conducted during the Ebola outbreak in 2014 and required proper training for the team and extensive operational support from the IDG home office. LEAP-II utilized a risk assessment framework (RAF) to evaluate potential risks and impacts of recommended CIP improvements including analysis of the operation and maintenance (O&M) cost recovery and sustainability, capacity building and institutional strengthening. The RAF was used as a strategy for prioritizing and sharing information about the risks to infrastructure investment. The evaluation team analyzed the risks and rewards of a decision using data collected during the evaluation. KEY FINDINGS The evaluation team found that while LMWP employed key tools and concepts in designing the CIP, it did not apply a value engineering/value analysis (VE/VA) for the design (which is suggested by international best practice). As a result, the O&M cost recovery period of one to three years which is projected by LMWP is not realistic. The evaluation team conducted a sensitivity analysis and found that the O&M cost recovery period was closer to seven years. Another key finding was that the institutional framework and capacity building provided by LMWP was effective and one of the project’s key achievements. LEARNING The final report presented the RAF and detailed findings supported by evidence and data to the evaluation questions. The report included three levels of priority recommendations to reflect the degrees and levels (1-3) of importance (high-low) of the proposed recommendations to allow timely and concurrent, yet priority-driven improvements to the project. Priority recommendations included conducting VE/VA prior to and during construction; re-evaluating the economic and financial analysis including identifying trade -offs through sensitivity analysis; and utilizing performance-based contracting with provisions for incentives; and providing support to the government to overcome external and internal institutional challenges. To facilitate learning, the Key informant interview in Robertsport, Liberia PHOTO CREDIT: SAM KODUAH 3 | LEAP-II FINAL REPORT USAID.GOV evaluation team presented the results to USAID/Liberia and discussed in detail the recommendations and how to implement them. #2 – PARTNERSHIP FOR GROWTH (PFG) MID-TERM EVALUATION INTRODUCTION The Partnership for Growth (PFG) initiative aims to achieve accelerated, sustained, and broad-based economic growth in selected partner countries through bilateral agreements between the USG and the partnering countries’ national governments. PFG requires rigorous, joint analyses of countries’ individual constraints to growth, joint action plans to address the most pressing of these constraints, and high-level mutual accountability for the goals and activities selected to alleviate them. In February 2011, the Governments of Ghana and Tanzania committed to work with the USG to accelerate and sustain broad-based and inclusive growth in Ghana and Tanzania through the PFG initiative. This included a commitment to jointly prepare a Constraints Analysis (CA) using the Growth Constraints approach of Hausmann, Rodrik, and Velasco (2006). The CA’s for Ghana and Tanzania were finalized in August and September 2011 respectively. These were used as the basis for the development of the five-year Joint Country Action Plans (JCAP) for Tanzania 2012-2016, issued in April 2012 and for Ghana which was finalized in March 2013 with the five-year program running to February 2018. LEAP-II carried out the mid-term evaluations in Ghana and Tanzania included examining PFG’s effectiveness in addressing binding constraints identified in each country (energy and access to finance in Ghana, and energy and rural roads in Tanzania). The mid-term evaluations sought to determine whether the PFG process represent an improvement over the pre-PFG assistance approach. The evaluations also examined whether PFG analyses and activities are sufficient for addressing the identified constraints, realizing the desired outcome and attributing the impact of PFG interventions on reducing the constraints. METHODOLOGY The evaluation team used three data collection methods: 1) A desk review based on the program documentation received primarily from USAID for the PFG initiatives as a whole, as well as documents that refer exclusively to the core focal areas addressed under the PFG in each country. 2) Semi-structured interviews with key PFG stakeholders, including high-level leadership, leadership, architects, goal leads, implementers, and independent experts in the US, Ghana, and Tanzania. The team spoke with 83 people in Ghana and 129 people in Tanzania. 3) A web-based survey of officials and technical specialists. KEY FINDINGS Key findings from the evaluation include:  At the midterm, PFG is regarded as a successful initiative by GOG, GOT and USG leaders, designers and goal leads.  All parties involved (interviewed) stated their belief that the PFG process had contributed to improved dialogue as partners in a shared commitment to development.  The processes with the CA, JCAP, log frames and a developing a commitment to M&E advanced understanding and encouraged deeper thinking on causal linkages. USAID.GOV LEAP-II FINAL REPORT | 4  The JCAP in both Ghana and Tanzania were ambitious and optimistic taking into account the difficult development challenges.  PFG approach adopted and owned in the power sector, but more challenges were experienced in access to finance in Ghana and rural roads in Tanzania.  Many of the PFG interventions are on target and substantial progress has been made on a range of challenging policy and regulatory issues in a relatively short period of time.  PFG is partially on track in Ghana and Tanzania for the power sector, which has received additional financial resources.  There has been limited progress in Ghana on the access to finance constraint, and modest progress in Tanzania or rural roads connectivity. LEARNING LEAP-II produced three evaluation reports: Ghana, Tanzania, and a crosscutting report. The reports addressed country specific questions for Ghana and Tanzania, and crosscutting questions. They included detailed recommendations on the PFG process and targeted sectors. The evaluation team also produced detailed case studies of the power sector in each country that addressed: public-private partnerships, policy, regulatory and institutional challenges; energy mix and power generation sources; and power usage and transmission and distribution system coverage as well as case studies on the roads sector in Tanzania and access to finance in Ghana. The team presented the results and recommendations to USAID/Ghana, USAID/Tanzania, and USAID/Washington. #3 – FINAL PERFORMANCE EVALUATION OF THE LIBERIA ENERGY SECTOR SUPPORT PROGRAM (LESSP) INTRODUCTION LESSP (implemented by Winrock International) aimed to achieve the following objectives: 1) Increased, sustainable access and affordability of electricity within selected urban and rural poor communities; 2) Improved performance of local governments, civil society and the private sector in monitoring, regulating and managing the use of renewable energy; 3) An increase in the percentage of households and businesses utilizing clean energy, and a corresponding increase in economic activity; and 4) Policy changes that improve the investment climate for the energy sector. In 2014, the effects of the Ebola epidemic began to impact project performance. Winrock International requested a no-cost extension through March 2015. However, given the uncertainties with the project performance, contract compliance, and expected construction end dates, USAID did not grant an extension to the contract, and let the project end in October 2014 as originally scheduled. The construction of the pilot projects at Sorlumba, Kwendin, and Gbarnway were incomplete at project close out. The primary purpose of the evaluation was to determine whether the assistance provided by LESSP met the stated development objectives, and identify lessons learned. The evaluation provided a detailed picture of the major accomplishments and weaknesses of the project since inception, including management issues, and indicating the recommended changes in implementation approach to future energy programs, and USAID’s project design/programming approach to assure successful completion of the project objectives. The evaluation questions were organized into three categories: 1) Effectiveness of infrastructure planning / construction oversight approach; 2) Effectiveness of institutional framework and capacity building; and 3) Overall project positioning and strategy for phase-out of USAID assistance. 5 | LEAP-II FINAL REPORT USAID.GOV METHODOLOGY Data collection methods included a comprehensive desk review of relevant documents and reports and KIIs of key actors and stakeholders both in the U.S. and in Liberia (the field work took place April 17 – May 12, 2015). The evaluation team conducted KIIs with 27 key stakeholder organizations and approximately 60 participants in the US and Liberia. In-country interviews were supplemented by visits to the three LESSP pilot sites located in Gbarnway (solar in Lofa County), Sorlumba (biomass in Lofa County) and Kwendin (biomass in Nimba County) as well as two visits to the Booker Washington Institute to assess the situation on-the-ground with respect to sustainability of activities. KEY FINDINGS 1) The program scope was too broad with too many initiatives each of which required time and resources. The lack of focus created implementation challenges. Ultimately by overreaching, the program was unable to accomplish its objectives. 2) The program design was not strategic and stakeholder driven making it difficult for stakeholders to claim ownership to facilitate sustainability. 3) At the time of the evaluation, there was little evidence that LESSP built any meaningful relationship within the GOL. Most parties outside of GOL had positive interactions with LESSP but had high expectations of what would still be delivered based on their interactions with LESSP. 4) Capacity building did not emphasize organizational performance improvement sufficiently and did not have a rigorous evaluation mechanism to determine long term impacts. 5) The pilots initiated by LESSP did not achieve their original purpose – the demonstration of sustainable mechanisms to provide modern energy service in rural poor communities. Putting pilots in remote areas (in a post-conflict country) where there are technological and logistical challenges requires more resources to facilitate effective implementation which should have been identified during the due diligence. LEARNING The evaluation report included specific, action-oriented recommendations including improving and expanding stakeholder partnerships, aligning initiatives with national goals and objectives, strengthening M&E, and conducting a sensitivity analysis. The report also identified priority areas that should be the focus of possible future programming in the Liberian energy sector, including the renewable energy market and rural development strategies. The evaluation team presented the results of the evaluation to USAID/Liberia. #5.A EVALUATION OF THE BEYOND ADVOCACY FUND (BAF) IN SOUTH AFRICA INTRODUCTION Beyond Advocacy Fund (BAF) was established through a Memorandum of Collaboration signed between USAID and Business Leadership South Africa (BLSA) in November 2013. BAF, which was co-financed by matching commitments of US$1.5m each from USAID (in cash) and BLSA (in cash or in-kind contributions), was scheduled to operate for an initial period of three years, which was extendable depending on performance and the availability of funds. BAF was conceived as a tool to help address South Africa’s major development challenges by catalyzing BLSA’s members (private sector businesses) to work with the Government of South Africa on collaborative projects that address SA’s most pressing development challenges. The Fund aims to promote partnership-based approaches between government and business to identify, test and replicate innovative and lasting solutions to major development challenges in South Africa. USAID/South Africa requested that LEAP-II conduct an evaluation to determine how effective and efficient BAF is in achieving its goals. It also assessed BAF’s efforts to promote better government- private sector relations through funding small projects. The findings were designed to inform adjustments and improvements to current activities and a second three-year funding phase. USAID.GOV LEAP-II FINAL REPORT | 6 METHODOLOGY Data collection methods included:  Desk review of project documentation. The evaluation team reviewed documents relating to BLSA and BAF strategy and management, quarterly and annual reports, selection criteria, project proposals, and background reports.  Key informant interviews (KIIs) with key project stakeholders and grant recipients.  Field observations. For two projects the evaluators visited sites where the projects are active in order to better understand project issues and assess potential project effects. For other projects, there were no physical locations to visit or work had not yet begun. KEY RESULTS Selection process. BAF selection procedures, driven by informal connections and discussions, can be characterized as relationship-driven. By not advertising widely or marketing BAF, the program does not attract a large pool of applicants. Projects. The evaluation team’s review of the projects found that each indeed focuses on facilitating systemic change, freeing blockages, and acting as a catalyst for further growth and development. In this respect, they align clearly with a core BAF objective. The team also found that the selection process and focus of BAF has evolved over time, with later projects showing a clearer and stronger case for private- public sector collaboration. Closer engagement between government and private sector. While the facilitator role BAF plays is clearly important, it is not a guarantor of ultimate success. The question over the longer-term is whether there is enough support to hold the various initiatives together and keep them moving forward – especially once/if the facilitator leaves. The grantees are usually the facilitators, but in some projects their long-term engagement is not assured. Evidence of program outcomes. BAF funded projects rate highly in terms of relevance. Although the focus of projects has evolved, most have demonstrated, or show the potential to demonstrate, ways in which partnering between the public and private sector has addressed complex barriers which have to date curtailed South Africa’s development. Private sector interest and awareness in BAF. BAF has been able to show that there are private sector players willing to engage in a new approach to South Africa’s development challenges. They realize their contribution cannot simply be a donation and photo opportunity arranged by the marketing department. To date, a number of partners have also been willing to put substantial amounts of money and effort behind this. LEARNING The evaluation team produced a comprehensive evaluation report providing evidence and data to address the evaluation questions. The report included a summary of BAF’s projects’ key features and performance, and case studies of six projects that examined in detail the elements that contributed to their successes or lost opportunities. The team presented the results to USAID/South Africa. The results were used by USAID/South Africa to inform the extension of BAF and to improve the operational efficiency and the management of the activity. [... middle sections omitted for long document ...] 87 | LEAP-II FINAL REPORT USAID.GOV justify these costs. The ENPV and ERR from REGIS-AG’s perspective in Burkina Faso are US$ -3.15 million and -10.49% respectively. In Niger the ENPV is US$ - 4.22 million and the ERR is -13.72%. Impacts on Poverty Alleviation: One of the main objectives REGIS-ER and REGIS-AG is to increase the incomes of the projects’ beneficiaries. This objective is tied to the overall objective of RISE, which is to lift the recipients of USG’s assistance out of extreme poverty. Given an extreme poverty threshold of US$ 694 per capita, in which case it is assumed that an individual spends on average US$ 1.90 daily over the course of a year, Figure 7 illustrates how REGIS-ER’s CF and goat and chicken Habbanayé interventions, and REGIS-AG’s WCS are expected to contribute to alleviating poverty through the agriculture, chicken, and goat VCs. 17 Chicken vs. Guinea Fowl Rearing: The comparative CBA showed that with an investment of US$ 70 for chicken (which is enough to purchase parent stock of chicken of 10 layers and one cockerel) and US$ 151 for guinea fowls (which is enough to purchase 8 guinea hens and 4 cockerels), guinea fowl rearing is more profitable in both Burkina Faso and Niger with annualized incomes of US$ 249 and US$ 267, respectively. While guinea fowl rearing is more profitable than chicken rearing, it requires higher investment costs. In 17 Extreme poverty often refers to living on less than $1.90 per day. (USAID, September 2015). https://www.usaid.gov/ending- extreme-poverty 635  635  317  317  497  497  248  248  221  236  158  276  220  228  166  247  694   ‐  100  200  300  400  500  600  700  800  900 Agriculture Without Warrantage Agriculture With Warrantage Chicken Habbanaye Goat Habbanaye Agriculture Without Warrantage Agriculture With Warrantage Chicken Habbanaye Goat Habbanaye USD/per capita Value  Chain Income - Without Project Incremental Income - With Project Extreme Poverty Threshold BurkinaFaso Niger FIGURE 7: POVERTY ALLEVIATION OF REGIS-ER’S AND REGIS-AG’S INTERVENTIONS USAID.GOV LEAP-II FINAL REPORT | 88 addition, while Habbanayé chicken targeted women from poor households, the CBA team observed that guinea fowl rearing was mostly done by men. Goats vs. Sheep Rearing: The analysis revealed that while both goat and sheep rearing is financially and economically feasible, goat rearing results in higher financial returns due the lower investment cost requirements. Sheep rearing, in turn, generates higher annual income when compared to goats. Also, farmers that are engaged in sheep rearing have the opportunity to profit from sheep fattening which generates additional income. It is recommended to advise farmers to invest their annual proceeds from the sale of crops into goat farming. Once farmers reach a financial position where they can afford the purchase of the first sheep herd, they should diversify into sheep rearing as well. The only difference between goats/sheep rearing and Habbanayé from the farmers’ perspective is the source of funding for the initial herd purchase. Therefore, most of the observations and conclusions of the Habbanayé analysis are also directly applicable to the goats and sheep rearing. Sheep fattening: Sheep fattening allows farmers to obtain a profit margin of about US$ 25/year assuming they fatten one animal twice a year. However, the sheep fattening intervention may not be appropriate for the most vulnerable beneficiaries since vulnerable households have competing cash needs and may find it financially challenging to feed sheep over a six-month period. Field visits revealed that farmers that engage in the sheep fattening are not necessarily the same farmers that rear small ruminants. Sheep fattening is done by women as the investment of cowpea proceeds after the harvest or by those able to identify the commercial opportunity (high demand for lamb during festivals) and with the technical knowledge required to fatten sheep. Supporting sheep fattening will help farmers diversify their diversification, mitigate inflation, and create assets. The marginal cost of adding sheep fattening to other training conducted by REGIS-ER and REGIS-AG is likely minimal and should be explored. LEARNING Integrating Purdue Improved Crop Storage (PICS) bags outside of the cowpea VC should be explored. Though spoilage occurs with all three crops, farmers use PICS bags for only cowpeas, and only in rare instances did we find that farmers use PICS bags to store millet and sorghum. An analysis on the use of PICS bags shows that the benefits far outweigh the cost of purchasing the bags. The benefit- cost ratios (BCRs) show that farmers in all three VCs would benefit from the adoption of PICS bags. In Burkina Faso, the BCRs are estimated at 7.35, 6.70 and 4.48, and in Niger, at 7.26, 13.68 and 5.11, for the cowpea, millet, and sorghum VCs respectively. This analysis highlights that the cost of PICS bags is not the only constraint to farmers using them. Improve access to inputs. Some targeted areas are remote and lack access to input markets, which makes it difficult and expensive for farmers to purchase recommended inputs such as fertilizers. As a result, some farmers cannot fully apply the recommended set of inputs, which negatively impacts the benefits of CF with regards to increased crop productivity. Though REGIS-ER has promoted Community-Based Solution Providers (CBSPs) to fill this gap, work should continue and be scaled up if possible. Diversify household livelihoods. Burkina Faso and Niger have only one cropping season over the course of the year. In order to lift agricultural HHs out of extreme poverty and increase their resilience, HHs need to diversify their livelihoods. Farming HHs should be trained and encouraged to engage in alternative 89 | LEAP-II FINAL REPORT USAID.GOV income generating activities during the agricultural off-season. Though REGIS-AG has conducted training in the processing of cowpeas into various products such as snacks, flour, couscous, and baby foods, future projects should explore scaling up capacity building in processing. Adding value to the raw cowpeas would increase farmers’ incomes and provide a source of income other than that from crop cultivation. Consider future regional economic trade. Both Burkina Faso and Niger export significant amounts of their cowpeas in the West African region. However, the primary destination for Burkinabe and Nigerien cowpea is Nigeria. Nigeria, which is the largest consumer of cowpeas in Africa, has in the past had to import cowpeas to supplement its huge demand. However, the country's production is growing and is nearing self-sufficiency. Burkina Faso and Niger’s cowpea export markets may need to be diversified to include other regional and international destinations. Though countries like Ghana, Benin, Togo, and Mali can absorb the output from Burkina Faso and Niger, their demand for the crop is not as strong as Nigeria’s. It should be noted that there is a need to conduct a study on the comparative advantage of Burkinabe and Nigerian cowpeas versus other producers in the regional export market before any measures are taken to promote the crop. Habbanayé intervention: The results indicate that the benefits of the Habbanayé intervention outweigh the costs from USAID’s perspective. One of the frequent complaints about the Habbanayé intervention is that farmers do not transfer the first offspring to other vulnerable households. While it may be both challenging and costly to monitor the transfer, this should not prevent USAID from investing in Habbanayé, as the economic returns presented in this study are estimated assuming no transfers are carried out. Therefore, the Habbanayé intervention when coupled with the animal husbandry training is an efficient way to assist the most vulnerable households. USAID.GOV LEAP-II FINAL REPORT | 90 OTHER ACTIVITIES #0.B – LOGISTICAL SUPPORT TO THE SECOND ECONOMISTS REGIONAL WORKSHOP LEAP-II provided logistic support to the Second Economists Regional Workshop (the Workshop) to be held in Pretoria, South Africa, November 9-13, 2015. The objective of the Workshop was to bring together USAID economists to discuss broad subjects including monitoring and evaluation; economic growth and jobs; macroeconomics and domestic resource mobilization; poverty and its measurement; and project/program cycle. Approximately fifty-one (51) people participated, including forty (40) USAID staff, four (4) USAID-funded outside speakers, six (6) non-USAID-funded local participants. LEAP-II received the request for assistance just over a month before the Workshop and successfully provided administrative and logistical support in preparation for the Workshop as well as during the workshop. In preparation for the Workshop, LEAP-II: identified and booked accommodations for external speakers; wrote and printed agendas, speaker bios, contact lists, and presentation materials; coordinated lunch, coffee/tea, and snacks; coordinated ground transportation and airport pickups for external speakers; set up training facility rooms for presentations, including projectors/laptops, etc.; pretested equipment to ensure it was working properly for presentations; and gathered all presentation materials. During the Workshop LEAP-II: registered guests; supported IT functions, including live- streaming and PowerPoint presentations; paid per diems to speakers and coordinate payment for services; set up lunch and snack breaks; straightened up Workshop rooms at the end of each day; and provided logistic support to the Workshop organizers, as needed. #8 AND #24 – ADVISORY SERVICES TO THE POWER AFRICA COORDINATOR’S OFFICE Power Africa represents a whole of U.S. Government, transaction-based development approach to increase access to electricity in sub-Saharan Africa. Power Africa works with African governments, the private sector, and other partners such as the World Bank and African Development Bank in six focus countries — Ethiopia, Ghana, Kenya, Liberia, Nigeria and Tanzania — to add more than 30,000 megawatts (MW) of cleaner, more efficient electricity generation capacity. By expanding mini-grid and off-grid solutions and building out power generation, transmission, and distribution structures, Power Africa will make electricity access available for 60 million household and business connections. At the same time, Power Africa will enhance energy resource management capabilities, allowing partner countries to meet their critical energy needs and achieve sustainable, long-term energy security. The Sustainable Energy for All (SEforALL), a non-profit and global initiative launched by Former UN Secretary-General Ban Ki-moon that promotes universal access to modern and efficient energy sources in support of the SDG7’s core objectives. On September 22, 2014, Power Africa signed an Aide Memoire for Cooperation Understanding in furthering energy access, electric power generation and market development, cross-border transmission network development, clean energy investments, off- grid energy generation, and energy project financing in sub-Saharan Africa. The purpose of this activity was to provide support to both Power Africa and SEforALL in the design and review of investment Prospectuses, with specific focus in the five countries of overlap: Ethiopia, Ghana, Kenya, Liberia, and Tanzania. IDG advised SEforALL and the USAID Power Africa Coordinator’s office in the design and review of transaction Investment Prospectuses, with specific focus in five countries: Ethiopia, Ghana, Kenya, Liberia, and Tanzania. IDG supported the Finance Sub-Committee of 91 | LEAP-II FINAL REPORT USAID.GOV SEforALL in operationalizing its strategy to enhance financial opportunities and products that further the goals of SEforALL both regionally and globally. Specifically, IDG assisted in elaborating a standardized structure and use-case for the transaction Investment Prospectus; supports the design of the transaction Investment Prospectus; identifies the primary content items needed to satisfy financing requirements, identifies areas for private and public-private interventions; supports coordination; helps to identify sources of capital and financing instruments; identifies sources of debt (and associated credit enhancements), equity, and fixed income capital for investment and financing instruments; and assists with the development of structures, such as financing platforms.