Évaluation de la performance du programme USAID/Haïti pour l'accession à la propriété et l'expansion des prêts hypothécaires (HOME)

USAID/HAITI HOME PERFORMANCE EVALUATION REPORT HOME OWNERSHIP AND MORTAGE EXPANSION Haiti Evaluation and Survey Services AUGUST 2020 This publication was prepared independently by Social Impact, Inc., at the request of the United States Agency for International Development. USAID/HAITI HOME OWNERSHIP AND MORTGAGE EXPANSION (HOME) PERFORMANCE EVALUATION Final Report USAID/Haiti Evaluation Mechanism No: AID-521-A-15-00009 Cover Photo Credit: USAID/Haiti HOME This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Sandra Wark (Team Leader) and Phanol Philippe (Assistant Team Leader) on behalf of Social Impact, Inc. Contact: Jennifer Mandel, Chief of Party 2300 Clarendon Blvd, Suite 1000 Arlington, VA 22201, USA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | USAID/HAITI HOME EVALUATION REPORT USAID.GOV ABSTRACT Social Impact’s Haiti Evaluation and Survey Services conducted an independent final evaluation of the United States Agency for International Development (USAID)/Haiti’s Home Ownership and Mortgage Expansion (HOME) activity. The World Council of Credit Unions (WOCCU) implemented HOME in partnership with Habitat for Humanity International (HFHI) and the Affordable Housing Institute (AHI). Using a mix of pay for performance (P4P) incentives and technical assistance, HOME aimed to increase access to safe and durable housing by low-to-middle income Haitians by catalyzing private sector investment. The activity was largely successful in motivating three local property developers to build eco-friendly commercial housing developments affordable to middle-class (but not lower-income) households. Even though HOME’s results in terms of the quantity of houses build fell short of its end-of- activity targets, most key informants broadly agreed that one property developer’s success showed that a viable market exists, mainly among the middle class, for affordable, standard housing financed through bank and credit union mortgages. HOME also facilitated the provision of mortgages for the sales of some of these units and supported the expansion of housing microfinance by three credit unions. In addition, the technical assistance provided to credit union partners to improve their housing loan terms and conditions for owner-driven housing improvements contributed to the issuance of the first mortgage by a credit union in Haiti. In the short-term, however, any significant housing development expansion will suffer from Haiti’s current economic conditions and persistent governance challenges. USAID.GOV USAID/HAITI HOME EVALUATION REPORT | ii CONTENTS Tables and Figures ii Abbreviations and Acronyms iii Executive Summary i I. Introduction 1 Project Background 1 HOME Activity Goal and Objectives 2 HOME’s Theory of Change 3 Evaluation Purpose and Audience 4 Evaluation Questions 4 II. Evaluation Design 5 Sampling 5 Data Collection Methods 5 Analysis 8 Limitations and Mitigation Strategies 9 III. Findings and Conclusions 10 EQ1: To what extent and in what ways has the HOME activity promoted successful models to support affordable housing development in Haiti? 10 EQ2: To what extent and in what ways was the HOME-promoted “pay for performance” methodology effective in generating results and will be replicable and sustainable? 22 EQ3: To what extent and in what ways has HOME’s facilitation approach helped to create local real estate developers, reinforce housing financing, improve the ecosystem, and create an enabling environment? 33 IV. Recommendations 38 Annex A: Evaluation Statement of Work 40 Annex B: References 47 Annex C: Evaluation Team Members 49 Annex D: Evaluation Design Matrix 50 Annex E: List of Persons Interviewed 56 Annex F: Informed Consent Scripts & Data Collection Protocols 58 Annex G: Disclosure of Conflict of Interest 81 TABLES AND FIGURES Table 1: Key informant interviews by stakeholder group and sex ................................................................ ........ 7 Table 2: Focus Group Discussion by Stakeholder Group and Sex ................................................................ ........ 7 Table 3: Overview of HOME property developer projects’ progress-to-date................................................. 11 Figure 1: HOME results framework ................................................................................................ .............................. 3 Figure 2: HOME property developers’ units planned, built, and sold ................................................................. 15 Figure 3: Number of credit union housing loans before and after award.......................................................... 18 Figure 4: Gender distribution of credit union partner loans at end of HOME................................................. 27 Figure 5: Evolution of credit union housing portfolio before, immediately after award, and 18–24 months later ..................................................................................................................................................................................... 29 iii | USAID/HAITI HOME EVALUATION REPORT USAID.GOV ABBREVIATIONS AND ACRONYMS AHI Affordable Housing Institute CPF Caisse Populaire Fraternité EDGE Excellence in Design for Greater Efficiency EPPLS Public Enterprise for the Promotion of Social Housing ESS Evaluation and Survey Services ET Evaluation Team EQ Evaluation Question FGD Focus Group Discussion GI Group Interview HTG Haitian Gourd HFHI Habitat for Humanity International IFC International Finance Corporation GOH Government of Haiti HOME Home Ownership and Mortgage Expansion KII Key Informant Interview M&E Monitoring and Evaluation MFI Microfinance Institution P4P Pay for Performance PAP Port-Au-Prince PAR Portfolio at Risk PFI Partner Financial Institutions SI Social Impact SME Small and Medium-Sized Enterprise UNOPS United Nations Office for Public Service USAID United States Agency for International Development USD United States Dollar i | USAID/HAITI HOME EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY The United States Agency for International Development (USAID) in Haiti requested that Evaluation and Survey Services (ESS) of Social Impact, Inc. (SI), design and conduct an independent final evaluation of the Home Ownership and Mortgage Expansion (HOME) activity. HOME is implemented by the World Council of Credit Unions (WOCCU) in partnership with Habitat for Humanity International (HFHI) and the Affordable Housing Institute (AHI). The purpose of this report is to present the evaluation team’s (ET’s) main findings, conclusions, and recommendations. EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this final evaluation is to inform USAID’s future programming that might use HOME’s “pay for performance” (P4P) development model in Haiti and elsewhere. The evaluation also assesses the extent to which HOME achieved intended results, identifies any lessons learned, and recommends possible follow-up actions. The primary audience for this final evaluation includes USAID/Washington, USAID/Haiti, the grantee and subgrantees (WOCCU, HFHI, and AHI), other donors, Haitian private sector entities, and the Government of Haiti (GOH). This report addresses the following evaluation questions (EQs): 1. To what extent and in what ways has the HOME activity promoted successful models to support affordable housing development in Haiti? The ET should consider both the real estate developing and the financing aspects of the model and the relationship between the two, highlighting what worked, what did not, and why. 2. To what extent and in what ways was the HOME-promoted ‘pay for performance’ methodology effective in generating results and will be replicable and sustainable? The evaluation team should identify any specific elements of the approach that are likely to be replicable/sustainable and those that will not be. 3. To what extent and in what ways has HOME’s facilitation approach helped create local real estate developers, reinforce housing financing, improve the ecosystem, and create an enabling environment (notaries, related associations, Central Bank, Public Private Unit at the Ministry of Finances)? In this context, sustainable local real estate developers and financial institutions should be understood as those that will continue the work started under HOME. Among other empowerment criteria, the evaluation should consider the level of leadership taken by local entities in the undertaken activities and success in terms of housing projects and housing finance. ACTIVITY BACKGROUND In 2015, USAID/Haiti developed the HOME activity to facilitate access to safe and durable housing through market-based solutions in the Port-Au-Prince (PAP), Cap-Haitian, and Saint-Marc development corridors. HOME set out to achieve two results: build and improve houses and provide housing finance to low- and medium-income Haitian households. To achieve private-sector-led solutions to Haiti’s housing deficit, HOME provided a mix of performance-based financial incentives and technical assistance to local property developers to create housing developments for underserved low-to-middle-income Haitians. HOME also assisted and incentivized financial institutions to develop home finance products and increase their home loan and mortgage portfolios serving low-to-medium-income Haitians, USAID.GOV USAID/HAITI HOME EVALUATION REPORT | ii particularly households headed by women. In 2015, WOCCU and its sub-grantees, HFHI and AHI, launched HOME, initially through a three-year, USD 6,999,683 grant. In 2018, USAID/Haiti extended HOME for two additional years and increased its budget to USD 10,669,683. The activity planned to end in June 2020 but was recently extended to January 2021. EVALUATION DESIGN, METHODS, AND LIMITATIONS The ET used a primarily qualitative evaluation design, with data collection methods consisting of a document review, key informant interviews (KIIs), group interviews (GIs), focus group discussions (FGDs), and loan portfolio and HOME monitoring and evaluation (M&E) data analysis. The ET interviewed 53 individuals (23 women and 30 men) from various stakeholder groups. The ET also organized six FGDs with housing microfinance loan recipients, two sex-segregated groups from each of HOME’s three credit union partners, reaching 39 participants in total (16 women, 23 men). The ET used purposeful sampling to select informants, including HOME personnel, property developer and financial institution partners, other value chain entities, other stakeholders such as USAID officials, and homebuyers. The ET conducted all data collection remotely because of COVID-19 public health restrictions. Evaluation limitations included recall, response, selection, and gender bias, as well as communication challenges. The ET mitigated potential bias and limitations by selecting informants who were well informed about the activity and represented diverse stakeholder groups, preparing and using detailed data collection protocols, triangulating information from multiple sources, and using various data collection methods, as well as identifying and using various communication technologies. FINDINGS AND CONCLUSIONS EQ1: TO WHAT EXTENT AND IN WHAT WAYS HAS THE HOME ACTIVITY PROMOTED SUCCESSFUL MODELS TO SUPPORT AFFORDABLE HOUSING DEVELOPMENT IN HAITI? To support the emergence of an affordable housing market in Haiti, HOME provided technical assistance and financial incentives to five partner financial institutions (PFIs) and three property developers. In collaboration with its partners, HOME sought to support affordable housing development in Haiti through two models: • Model 1: Mortgage-financed, commercially driven affordable housing developments and • Model 2: Credit union housing microfinance loan-funded, owner-driven affordable housing development. HOME was largely successful in making a business case for commercially driven housing developments for a middle-class market segment (Model 1). Through Model 1, HOME’s three property developer partners initiated six affordable housing developments, with a total of 257 homes planned, all located in PAP. HOME leveraged USD 9.6 million from private sector developers and USD 627,000 from PFI for the housing developments. As of mid-August 2020, HOME property developer partners reported completing 47 housing units. To date, one developer has sold a total of 22 units, 17 of which were financed through mortgages. Even though HOME’s results in terms of the quantity of houses built fell short of its end-of-activity targets, most key informants broadly agreed that the one property iii | USAID/HAITI HOME EVALUATION REPORT USAID.GOV developer’s success showed that a viable market exists, mainly among the middle class, for affordable, standard housing financed through bank and credit union mortgages. This positive demonstration effect, although tangible, was nevertheless diminished by the challenges and delays affecting other HOME property developer partners. One HOME partner’s project is currently halted because of financing issues and may also suffer from the development location’s insecurity and access issues. A third HOME partner reported good progress and is confident about completing the first phase of its development once COVID-19 related delays are overcome. However, evidence that homes in the development will sell is still pending. Rigorous market testing and HOME’s activities demonstrated that offering commercially driven housing developments for lower-income Haitians was not a viable business case in current market conditions. Although one development featured six low-priced houses, all of which have been sold, the conditions under which the development was built appear challenging for others to replicate at scale. A dozen or more key informants agreed that, in the absence of government incentives such as increased investment in basic infrastructure and services, re-zoning publicly owned land for commercial housing developments, and/or other fiscal incentives for developers, commercially driven housing developments are unlikely to be profitable at prices that are affordable to low-income Haitians because of high construction costs. HOME property development partner experiences produced useful lessons learned that may help guide future investments in commercially driven housing developments in Haiti: • Construction finance is as important as buyer access to mortgages; • Smaller developments and building in phases appear less risky and easier to finance than larger developments; • Pre-construction financing appears largely unviable due to trust issues; • Property development location and access to basic services are important success factors; and • Increasing the supply of commercial, affordable housing is an important condition for increasing mortgage lending to lower- and middle-income homebuyers. Model 2 (credit union housing microfinance loans) effectively helped low-income Haitians access housing finance to improve their homes under more favorable conditions. Overall, HOME leveraged USD 7.2 million from credit unions for housing finance activities. HOME was successful in strengthening the housing finance products and services of its credit union partners, who exceeded activity portfolio expansion targets and whose mainly lower-income members enjoyed lower interest rates, longer loan reimbursement periods, and improved credit agent outreach and support during the HOME award period. Model 2 successfully contributed to the first credit union mortgage product by one credit union partner, a milestone toward the democratization of mortgage lending in Haiti. Model 2’s success is mitigated by the ubiquity of housing microfinance in Haiti and its association with the spread of unregulated, informal, and unsafe housing. USAID.GOV USAID/HAITI HOME EVALUATION REPORT | iv EQ2: TO WHAT EXTENT AND IN WHAT WAYS WAS THE HOME-PROMOTED ‘PAY FOR PERFORMANCE’ METHODOLOGY EFFECTIVE IN GENERATING RESULTS AND WILL BE REPLICABLE AND SUSTAINABLE? HOME used P4P incentives effectively, flexibly adapting the modality to the activity’s objectives and context and producing several positive outcomes. By moderating real and perceived risks, HOME P4P incentives effectively motivated property development partners to invest in affordable housing and improve their building practices. P4P incentives and concessional loans were also largely effective at expanding housing microfinance lending by HOME’s credit union partners. P4P also incited significant cost-sharing from the Haitian private sector. Shared risk and the promise of tangible rewards effectively focused HOME’s partners’ efforts to achieve contract benchmarks. Even though HOME’s award structure evolved to reward incremental progress, increasing USAID/Haiti’s risk that its investment could fall short of achieving desired outcomes, the change was justified by Haiti’s worsening economic context, the high cost of construction finance in Haiti, and the limited time HOME had to achieve results. HOME’s use of P4P also had some shortcomings. The modality, in conjunction with the time allocated for implementation, did not effectively facilitate access to awards by builders and construction firms lacking large amounts of seed capital, which limited the scale of interventions and opportunities afforded to less well-off entrepreneurs. Although women-headed households represent a large segment of potential homebuyers, HOME’s interventions to overcome gender bias were limited. In addition, HOME’s allocations for mortgage subsidies were not effective at facilitating access to mortgages for a significant number of homebuyers to date (July 2020). The limited supply of finished houses in affordable housing developments and banks’ reluctance to allocate subsides both appear to be reasonable explanations for this intervention strategy’s lack of success. Expanded credit union housing lending will probably be sustained after HOME, but with modifications to terms and conditions. Two of HOME’s three partner credit unions have managed to continue expanding their housing finance portfolio 18–24 months after the end of HOME subsidies, suggesting continued commitment to the segment. However, information provided by credit union key informants shows that some of the qualitative benefits of HOME’s subsidies to borrowers have not been replicated post-award since credit unions did not maintain lower interest rates and decreased the size and length of housing loans. Credit unions’ aspirations to help their members access housing through mortgage lending, shown in their deal-making with property developers, is also a promising sign that credit union mortgage lending will be sustained and possibly replicated after HOME. HOME’s success catalyzing a market for affordable housing developments will probably be sustained and replicated after the end of subsidies but may face short term setbacks. Property developer and financial institutions’ actions and stated interest in continuing and potentially increasing their investments in mortgage-driven affordable housing developments are promising. Property developers’ perceptions that new building practices HOME introduced saved them money and added value to their developments also portend well for sustainability. The likelihood that property developers will finish current developments and expand into new developments in the short term is negatively affected by Haiti’s current political and economic troubles, and, for at least one developer, depends on the firm overcoming its current financing challenges. HOME’s efforts to mobilize new sources of financing and v | USAID/HAITI HOME EVALUATION REPORT USAID.GOV technical assistance are positive for all three property developer partners, but short-term results remain uncertain in light of Haiti’s currently poor investment climate. EQ3: TO WHAT EXTENT AND IN WHAT WAYS HAS HOME’S FACILITATION APPROACH HELPED TO CREATE LOCAL REAL ESTATE DEVELOPERS, REINFORCE HOUSING FINANCING, IMPROVE THE ECOSYSTEM, AND CREATE AN ENABLING ENVIRONMENT? HOME was largely effective in facilitating the emergence of local real estate developers. By changing builders’ and construction firms’ financing and development paradigm from individual client- driven to developer-driven, the activity contributed to the emergence of local property developers. HOME’s technical assistance to improve partners’ management and financial structures, although initiated somewhat late, appears to have spurred changes that might facilitate access to needed construction financing. The activity’s learning-by-doing approach was effective in building partner self- reliance while keeping HOME focused on achieving its results. In addition to facilitating change within individual enterprises, support for the property developers’ association enabled needed collective solutions and experience sharing, potentially strengthening the voice and advocacy power of the nascent developer community. HOME’s facilitation was moderately effective at reinforcing housing financing. Encouraging credit union mortgage product development and interest in partnerships with property developers both contributed to more equitable access to mortgages. However, credit unions’ concerns about expanding longer-term lending too rapidly suggest limited growth potential without additional capital infusions. Based on one property developer’s success obtaining commercial bank mortgages for its homebuyers, HOME also contributed to PFIs’ willingness to accord mortgages to middle-class borrowers. However, based on commercial bankers’ own as well as others’ testimony, HOME could not moderate bank mortgage lending terms and conditions, and, as a result, qualifying for a bank mortgage remains difficult and expensive. HOME’s contribution to expanding mortgage lending overall was minimal, largely because it did not significantly increase the affordable housing supply. HOME’s interventions also had a relatively small but positive effect on the housing sector ecosystem and enabling environment. By establishing the good practice of subdividing land and registering titles before construction, HOME reinforced ecosystem actors’ capacity and facilitated mortgage access. At the end of HOME, property developers still cannot apply Haiti’s Condominium Law, leaving the promise of “building up” to alleviate Haiti’s housing deficit unrealized. However, HOME has contributed to clarifying what is needed to resolve the impasse and taken steps that others, including property developers’ association members, can build on as they move toward solutions. RECOMMENDATIONS 1. In future activities, USAID grantees should focus earlier and more overall on interventions to facilitate property developers’ access to construction financing and credit unions’ access to affordable capital loans. 2. In future similar activities, USAID grantees should focus more on increasing the supply of affordable houses in the initial implementation phases, especially in areas where it has PFI partners. USAID.GOV USAID/HAITI HOME EVALUATION REPORT | vi 3. In future activities, USAID grantees should strengthen incentives and capacity building for partners to encourage single women-headed households’ access to affordable housing. More broadly, USAID/HAITI should ensure that future programming is informed by gender analysis that enables it to respond to specific barriers and biases that may affect women’s equal access and ability to benefit from housing finance. 4. USAID/Haiti should provide technical support to overcome regulatory barriers to building apartment buildings. 5. USAID/HAITI should maintain contact with the property developers’ association and provide guidance when needed. 1 | USAID/HAITI HOME EVALUATION REPORT USAID.GOV I. INTRODUCTION The United States Agency for International Development (USAID) in Haiti requested that Evaluation and Survey Services (ESS) of Social Impact, Inc. (SI), design and conduct an independent final evaluation of USAID/Haiti’s Home Ownership and Mortgage Expansion (HOME) activity, implemented by the World Council of Credit Unions (WOCCU) in partnership with Habitat for Humanity International (HFHI) and the Affordable Housing Institute (AHI) (see Annex A for the evaluation Statement of Work). The evaluation team (ET) reported initial findings based on a document review (see Annex B for list of documents consulted and Annex C for the team members’ profiles) and proposed the evaluation design in an Inception Report/Evaluation Design, which USAID/Haiti approved before the team started fieldwork (see the summary of evaluation design in the Evaluation Design Matrix in Annex D ). This report presents the evaluation’s principal findings, conclusions, and recommendations. PROJECT BACKGROUND Haiti has a housing deficit estimated at half a million to a million units in a country of only 11 million people. A 2010 earthquake destroyed approximately 250,000 homes, adding to an existing shortfall of at least 300,000 homes. Despite several donor interventions to rebuild Haiti’s housing stock, enduring shortages, high population growth, and urban migration continue to fuel demand for urban housing. As a result, most urban residents still live in precarious shelters and continue to be vulnerable to climatic and seismic hazards ten years after the earthquake. 1 Both the demand and supply sides are driving Haiti’s housing deficit. 2 On the demand side, most low-to- medium-income Haitians cannot afford to buy a home and may also face challenges financing home rehabilitation. Despite relatively high levels of liquidity, Haitian banks are risk-averse to mortgage lending. In 2015, there were only about 1,000 mortgages in Haiti, and the average value of the mortgaged homes was over USD 200,000, a price tag well out of reach for low-to-middle-income households. Also, few low-to-middle-income households are eligible for mortgage financing because they earn too little or their income stems from informal employment, which is difficult to underwrite. 3 Weak land tenure also hinders banks from providing loans, since homeowners cannot offer their property as collateral without proof-of-ownership. 4 Women are especially affected, since uneven power dynamics between men and women make it difficult for women to gain needed land ownership and the accompanying deeds required to access housing credit. Moreover, because many women have limited 1 Amnesty International, “15 Minutes to Leave: Denial of the Right FN Adequate Housing in Post-Quake Haiti,” 2015. 2 The demand side represents people’s ability to borrow money to finance the purchase of a new home or repair an existing home. The supply side of the housing value chain looks at the number of homes being produced. 3 Adapted from Caldwell and Clodomir, “Haiti HOME: Catalyzing Haiti’s Affordable Housing Market.” 4 Haiti does not have an effective national cadastre and lacks a comprehensive, functional system for recording land ownership. Before the earthquake, only 40 percent of landowners in Haiti possessed ownership documentation, such as a legal title or transaction receipt. See P. Giampaoli and M. Freudenberger, “USAID Issue Brief: Land Tenure and Property Rights in Haiti,” (USAID, January 2010), https://www.land-links.org/wp- content/uploads/2016/09/Land-Tenure-and-Property-Rights-in-Haiti-1.pdf . USAID.GOV USAID/HAITI HOME EVALUATION REPORT | 2 disposable income and savings, they remain dependent on male providers to cover housing costs, heightening their vulnerability to domestic violence. 5 Non-bank financial institutions (credit unions and microfinance institutions [MFIs]) typically offer unsecured, short-term housing microloans at rates upwards of 24 percent per year. Clients frequently take costly, successive short-term loans to finance a home over the long term. Constraints affecting non-bank financial institutions’ capacity to offer larger, longer-term home loans include lack of know- how to develop and market new financial products sustainably, as well as lack of capital. 6 On the supply side, Haiti’s formal real estate market offers few affordable homes for sale. The average home price in Port-au-Price (PAP) in 2018 was approximately USD 300,000. With constraints like limited appropriate land availability, high-cost construction materials, unclear, costly and time-consuming regulations associated with building and selling real estate, and limited access to financing both for construction and mortgages, property developers have unsurprisingly been afraid to invest in what they perceive to be a high-risk and low-reward affordable housing market. HOME ACTIVITY GOAL AND OBJECTIVES In 2015, USAID/Haiti developed the HOME activity to facilitate access to safe and durable housing for low-to-middle-income Haitian households, particularly women-headed households, through market- based solutions. HOME sought to catalyze the affordable housing market in Haiti by working with local private sector developers and financial institutions in the PAP, Cap-Haitian, and Saint-Marc development corridors. HOME used a mix of pay-for-performance incentives and technical assistance to address both supply- and demand-side constraints affecting the affordable housing market. In 2015, WOCCU and its sub-grantees HFHI and AHI launched HOME, initially through a three-year, USD 6,999,683 grant. In 2018, USAID/Haiti extended HOME for two additional years and increased its budget to USD 10,669,683. In July 2020, USAID/Haiti awarded WOCCU a no-cost extension that delayed HOME’s closure until January 2021. HOME’s overall goal was to increase low-to-middle-income Haitian households’ access to longer-term financing, enabling them to construct, rehabilitate, and purchase safe, durable, and affordable housing. To contribute to this goal, HOME aimed to achieve two results, each with two intermediate results, as described in Figure 1 below. 5 “The Haiti Gender Report,” A Coalition Gender Shadow Report of the 2010 Haiti Post-Disaster Needs Assessment (PDNA), p. 27. 6 Caldwell and Clodomir, “Haiti HOME: Catalyzing Haiti’s Affordable Housing Market.” 3 | USAID/HAITI HOME EVALUATION REPORT USAID.GOV FIGURE 1: HOME RESULTS FRAMEWORK HOME’S THEORY OF CHANGE USAID/Haiti designed HOME to catalyze and create an enabling environment for Haiti’s private sector to meet low- and medium-income Haitians’ demand for safe, durable, and affordable homes. WOCCU and its subgrantees HFHI and AHI hypothesized that the private sector in Haiti is capable of “working around” or directly overcoming many of the challenges affecting affordable home ownership, so long as housing value chain stakeholders have the right incentives and capacities. 7 By providing financial incentives to mitigate the real and perceived risks facing financial institutions and real estate developers, as well as technical expertise to build local capacity to apply good home finance and real estate development practices, HOME theorized that it could unlock sustainable, private-sector-led solutions to Haiti’s affordable housing deficit. On the demand side, HOME assumed that it could improve access to financing to build, rehabilitate, and purchase homes by providing technical assistance and incentives to microfinance institutions, credit unions, and banks. HOME’s interventions included pay for performance (P4P) incentives to motivate lending to female-headed households, effectively motivating and capacitating lending institutions to offer more home loans to low-to-medium-income households. HOME aimed to shrink the home financing gap facing low-to-medium-income Haitians by incentivizing banks to lower minimum mortgage values, while also building the capacity of and providing financial incentives to credit unions to increase their housing 7 HOME assumed that Haitian companies are familiar with the country’s land issues, know which land is under dispute, and can navigate Haiti’s complex land registration system. USAID.GOV USAID/HAITI HOME EVALUATION REPORT | 4 loan values, lengthen payback periods, and lower interest rates. By overcoming perceived risks and capacity constraints, HOME assumed that banks and credit unions would realize the business potential of providing home loans and mortgages to low-to-medium-income households and would continue to serve this market segment without subsidies or other assistance. On the supply side, instead of using donor funds to build housing projects, HOME postulated that it would be more effective and efficient to provide technical support and financial incentives to Haitian developers to motivate them to expand “down market” into affordable housing segments. HOME calibrated its financial incentives for hitting construction and sales benchmarks to cover the amount needed to make up the “profitability deficit” of building and offering housing at prices affordable to low- and medium-income buyers, while using green building techniques and satisfying international building safety standards. HOME further assumed that once developers saw the business potential of supplying green, safe, affordable housing, they would continue to build and expand supply even without subsidies. To ensure that HOME’s technical assistance and incentives would benefit low-to-middle-income Haitians, HOME limited its supply-side support to homes sold to income-qualifying households through a mortgage (which HOME ensured would be available through its demand-side interventions, such as down-payment subsidies). EVALUATION PURPOSE AND AUDIENCE This final evaluation aims to inform future USAID programming that uses HOME’s P4P development model in Haiti and elsewhere. The evaluation also assesses the extent to which HOME achieved its intended results, identifies lessons learned, and recommends possible follow-up actions. The primary audience for this final evaluation includes USAID/Washington, USAID/Haiti, HOME’s grantee and subgrantees (WOCCU, HFHI, and AHI), other donors, Haitian private sector entities, and the Government of Haiti (GOH). EVALUATION QUESTIONS When answering these evaluation questions (EQs), the ET considered stakeholder satisfaction, unintended results, and lessons learned. 1. To what extent and in what ways has the HOME activity promoted successful models to support affordable housing 8 development in Haiti? To answer this question, the ET will consider both the real estate development and finance aspects of the model and the relationship between the two, highlighting what worked, what did not, and why. 2. To what extent and in what ways was the HOME-promoted ‘pay for performance’ 9 methodology effective in generating results and will be replicable and sustainable? 8 According to HOME’s Performance Indicator Reference Sheet, the target income range of Haitian households for home loan loans and mortgages was between HTG 10,000 and HTG 250,000 per month (or the USD equivalent, USD 90–2,230 at time of writing this report). 9 Pay for results or performance is an umbrella term for initiatives that pay upon accomplishment of results rather than efforts to accomplish those results. For example, a funder or a project implementer sets aside financial incentives for an entity to deliver predefined results and pays these incentives after verifying the results. 5 | USAID/HAITI HOME EVALUATION REPORT USAID.GOV To answer this question, the ET will identify any specific elements of the approach that are likely to be replicable/sustainable and those that will not. 3. To what extent and in what ways has HOME’s facilitation approach helped to create local real estate developers, reinforce housing financing, improve the ecosystem, and create an enabling environment (notaries, related associations, Central Bank, Public Private Unit at the Ministry of Finances)? The ET understands sustainable local real estate developers and financial institutions as those that will continue the work they initiated under HOME. Among other empowerment criteria, the ET will consider the level of leadership local entities have demonstrated in undertaking activities and their success in terms of housing projects and finance. II. EVALUATION DESIGN SAMPLING The ET collected data on the perspectives and experiences of key stakeholders involved in HOME activity using a purposeful sampling technique (see list of persons consulted in Annex E ). Key stakeholder groups included HOME personnel, property developer and financial institution partners, other value chain entities, other stakeholders (including USAID officials), and homebuyers and credit union microfinance loan recipients. Evaluation respondents were from all three geographic zones in which HOME operated: the PAP, Saint Marc, and Cap Haitian development corridors. The ET supplemented purposeful sampling with snowball sampling to identify additional key informants and fill gaps in its initial list of key stakeholders. For example, the ET used snowball sampling to identify key informants such as: notaries, land surveyors and lawyers who had worked with HOME partners; bank representatives who supplied mortgages without a formal partnership agreement with HOME; credit union credit agents who participated in HOME training; government officials with whom HOME personnel interacted; and HOME partner housing development unit homebuyers. During initial interviews, HOME personnel and property developer and financial institution partners provided additional names and contact information for missing individuals. Overall, the final informant sampling allowed the ET to compare the perspectives of different key stakeholder groups and cross-check and fill gaps in HOME’s reporting. Although there were more male than female respondents (58 percent male respondents versus 42 percent female), the proportion reflected the roles played by men and women in HOME. DATA COLLECTION METHODS The ET used a predominantly qualitative design. The ET extracted quantitative data from HOME’s monitoring and evaluation (M&E) systems and data on credit union partners’ loan portfolios. The ET compared and complemented project-supplied data with qualitative data collected through various methods, consisting of document review, key informant interviews (KIIs), group interviews (GIs), and focus group discussions (FGDs). The ET reviewed activity documents and produced an inception report in May and June 2020. The ET carried out data collection in two phases. From June 1 to 12, the ET interviewed members of HOME’s implementation team. From June 29 to July 13, the ET interviewed all USAID.GOV USAID/HAITI HOME EVALUATION REPORT | 6 other stakeholder groups and carried out FGDs with credit union loan recipients. Due to COVID-19- related public health restrictions, the ET conducted all data collection remotely by telephone and various online conferencing technologies. The ET developed data collection protocols (see Annex F ) to: (1) ensure all key issues were covered during data collection, (2) elicit rich, sometimes unanticipated, information from respondents, (3) help organize information in a form that could be usefully and efficiently analyzed, and (4) ensure that the ET collected sex-disaggregated data. The protocols consisted of questions that address and derive from the EQs, as well as from the ET’s document review, its initial discussions with USAID, and its knowledge of evaluation design. DOCUMENT REVIEW The ET reviewed HOME documents to understand HOME’s design and implementation, extract findings relevant to the EQs, and inform data collection protocol development so that instruments appropriately supplemented background documents. The ET reviewed the following categories of documents (see Annex B for a full listing): • HOME grant proposal, • HOME quarterly and annual reports, • HOME annual work plans, • HOME M&E plans, • Activity descriptions and modifications, and • Journals and other publications. The ET also analyzed data on loan numbers, housing loan volume, loans made to women borrowers, and portfolio at risk (PAR, the percentage of loans with payments in arrears) provided by HOME for its three credit union partners’ housing microcredit loan portfolios. For most of these, HOME provided three data points: baseline values, values at the end of the award contract, and in December 2019, 18–24 months after the end of HOME’s credit union housing microfinance interventions. KEY INFORMANT INTERVIEWS The ET interviewed 53 individuals (23 women and 30 men) from various stakeholder groups. The ET interviewed most key informants individually, except for GIs with property developer partners teams (Tecina, Panamera), credit union credit agents, and members of the property developers’ association. Because of the small sample size, the ET interviewed all possible key informants within the HOME personnel, property developer partners, and partner financial institution (PFI) stakeholder categories. Findings from KIIs contributed to the ET’s responses to all EQs. Table 1 shows the distribution of interviews by key stakeholder group and sex. Each KII followed an interview protocol adjusted for different respondent types of respondents and was conducted in French, English, or Haitian Creole, depending on the key informant’s preference. 7 | USAID/HAITI HOME EVALUATION REPORT USAID.GOV TABLE 1: KEY INFORMANT INTERVIEWS BY STAKEHOLDER GROUP AND SEX Tthe evaluation team conducted individual and group interviews remotely, mainly using Skype, Skype Meet nNow, Zoom, or Whatsapp. Bbecause of absent or poor internet connections, the team leader (based outside Hhaiti) used Skype credit to call some key informants on their mobile phones on several occasions as well. Bbased on the team’s experience carrying out remote data collection, carrying out successful remote data collection the approach requires use of multiple communication tools and the ability to quickly switch from one technology to another when one technology fails. Using these tools, the team was able to successfully able to complete interviews with all identified respondents. FOCUS GROUP DISCUSSIONS The ET organized six FGDs with housing microfinance loan recipients and two sex-segregated FGDs from each of HOME’s three credit union partners, reaching a total of 39 participants (16 women and 23 men). TABLE 2: FOCUS GROUP DISCUSSION BY STAKEHOLDER GROUP AND SEX STAKEHOLDER FEMALE MALE Socolavim Loan Recipients Group 1 6 Socolavim Loan Recipients Group 2 4 Kotelam Loan Recipients Group 3 9 Kotelam Loan Recipients Group 4 7 CPF Loan Recipients Group 5 8 CPF Loan Recipients Group 6 5 Total 16 23 The evaluation team used SKYPE Meet Now and mobile phones to carry out FGDs with participants. The team experience some technological challenges and challenges achieving the planned number of particpants per group. For example, during the CPF focus groups (participants based in the North, in STAKEHOLDER INTERVIEWS FEMALES MALES Grantees and sub-grantee personnel 11 7 4 HOME PFI & property developer personnel 19 6 13 Other value chain entities 10 4 6 Other stakeholders 7 2 5 Homebuyers 3 3 0 USAID 3 1 2 Total 53 23 30 USAID.GOV USAID/HAITI HOME EVALUATION REPORT | 8 Cap Haitien), the facilitator experienced connection problems and so switched to a regular mobile phone line conference call, which required dividing the group into two and carrying out two smaller group discussions. The team also needed to make other adjustments to ensure successful remote FGDs. For example, after challenges getting the expected number of participants with the first FGDs, the team decided to call back scheduled participants individually and remind them of the planned discussion the day of the FGD, a strategy that ensured an overall higher participation rate for the remaining FGDs. ANALYSIS ET members took detailed notes from KIIs and FGDs and cleaned and shared electronic summaries on a rolling basis throughout fieldwork. Team members conducted internal debriefings regularly throughout fieldwork to discuss progress and adjust the evaluation schedule as needed. The ET also discussed evidence collected to help answer the EQs and identify any discrepancies. The ET employed several data analysis methods to identify key findings from the data, draw conclusions, and make recommendations for activity follow-up or future potential USAID programming similar to HOME. The ET triangulated HOME’s M&E data and loan portfolio analysis with qualitative data collected through its document review, KIIs, and GIs. Several ET members independently reviewed evaluation notes and tallied occurrences of key themes. The ET compared data from the varied sources described above to determine whether findings were divergent or convergent. The extent to which multiple informational streams provide consistent findings informed the certitude and internal validity of the conclusions. 1. Content Analysis: The ET reviewed and coded KII and FGD data to identify and highlight notable examples of HOME’s successes (or lack of success) that contributed to (or inhibited) achievement of its objectives. 2. Gap Analysis: The ET analyzed where HOME fell short of anticipated performance targets and the likely factors contributing to these gaps. 3. Comparative Analysis: The ET compared different stakeholder groups’ perspectives to assess convergence and divergence. The ET identified findings that arose from the interviews and developed a tally sheet that cross-checked findings by EQ, respondent type, interview type (KII or FGD), and sex. This method allowed the ET to look for trends within and across subgroups. 4. Gender Analysis: The ET disaggregated all KII and FGD data by sex and analyzed for possible differences in respondents’ perspectives associated with gender. The Team Leader captured preliminary findings, conclusions, and recommendations in a matrix categorizing analysis by EQ. The matrix (1) ensured that the ET prepared a systematic and thorough response to each EQ, (2) verified that preliminary analysis accounted for gender and social dimensions, (3) identified any gaps needing additional clarification or analysis, and (4) served as the basis for developing the evaluation report. 9 | USAID/HAITI HOME EVALUATION REPORT USAID.GOV LIMITATIONS AND MITIGATION STRATEGIES BIAS The ET was aware of several bias risks in data collection and analysis and developed mitigation strategies to overcome these: 1. Recall Bias: Recall bias is a common challenge in final evaluations, especially evaluations that take place months after an activity’s close. For example, HOME’s interventions in credit union housing microfinance ended in 2018. Some key personnel were no longer part of the HOME team and may not have been aware of or remembered details of HOME’s interventions or their outcomes. To mitigate this, to the extent possible, the ET interviewed team members (including former team members) and recipients who participated directly in HOME’s interventions and compared stakeholder responses, such as between the HOME manager and the credit union manager recipient, to increase the validity of the evaluation findings. The team also probed respondents’ answers to facilitate accurate recollection. 2. Response Bias: Response bias is the risk that key informants may have been motivated to provide responses they considered socially desirable or influential in obtaining continued donor assistance. For example, when the ET began data collection, USAID had not yet responded to HOME’s request for a no-cost extension. Some respondents, especially partners unaware of USAID procedures, may have believed their responses to the ET questions could influence the outcome of the extension request. To mitigate this potential bias, the ET used a clear informed consent protocol that, among other things, highlighted that the team’s independence from USAID/Haiti. Once USAID/Haiti accepted HOME’s extension request, the risk of this particular response bias was substantially reduced. 3. Selection Bias: Selection bias is an inherent risk that implementers orient the ET to stakeholders with primarily positive views of an activity. This risk was quite small in the case of HOME because there were a limited number of stakeholders directly involved in the activity in most stakeholder categories, all of whom the ET interviewed. In the few categories with larger numbers of potential respondents, such as housing loan recipients and homebuyers, the evaluation randomly selected the respondents from lists provided by HOME partners. The ET arranged meetings directly, without going through the grantee, in nearly all cases. 4. Gender Bias: Gender bias is a risk because most individuals have a subconscious sense of appropriate