Rapport de fin d'exécution - Premier projet de santé

Docurnent of The World Bank FOR OFFICIAL USE ONLY Report No: 22720 IMPLEMENTATION COMPLETION REPORT (IDA-20850) ON A CREDIT IN THE AMOUNT OF SDR 22.2 MILLION US$ 28.2 MILLION EQUIVALENT TO THE REPUBLIC OF HAITI FOR HT- FIRST HEALTH PROJECT 09/06/2001 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized CURRENCY EQUIVALENTS (Exchange Rate Effective ) Currency Unit = Gourdes I G = US$ 0.04 US$ 1.00 = 23.35 G FISCAL YEAR October 1 September 30 ABBREVIATIONS AND ACRONYMS AIDS Acquired Immunodeficiency Syndrome CIDA Canadian International Development Agency DOTS Directly Observed Treatment, Short Course Strategy HIV Human Immunodeficiency Virus ICC International Child Care IDA International Development Association IEC Information Education and Communication Material IMR International Mortality Rate MSPP Ministry of Health and Population (Ministere de la Sante Publique et de la Population) MEF Ministry of Economy and Finance (Ministere de l'Economie et des Finances) NGO Non-governmental Organization PAHO Pan American Health Organization PMU Project Management Unit PNLT National Tuberculosis Program (Programme National de Lutte contre la Tuberculose) SAR Staff Appraisal Report STD Sexually Transmitted Disease STI Sexually Transmitted Infection TB Tuberculosis UNAIDS Joint United Nations Programm on HIV/AIDS WDI Word Development Indicator WHO World Health Organization Vice President: David de Ferranti Country Manager/Director: Orsalia Kalantzopoulos Sector Manager/Director: Charles Griffin Task Team Leader/Task Manager: Girindre Beeharry FOR OFFICIAL USE ONLY HAITI HT- FIRST HEALTH PROJECT CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 1 4. Achievement of Objective and Outputs 6 5. Major Factors Affecting Implementation and Outcome 10 6. Sustainability 11 7. Bank and Borrower Performance 11 8. Lessons Leamed 8 9. Partner Comments 12 10. Additional Information 12 Annex 1. Key Performance Indicators/Log Frame Matrix 13 Annex 2. Project Costs and Financing 14 Annex 3. Economic Costs and Benefits 15 Annex 4. Bank Inputs 16 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 18 Annex 6. Ratings of Bank and Borrower Performance 19 Annex 7. List of Supporting Documents 20 Annex 8. Borrower's Report 23 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Project ID: P007311 Project Name: HT- FIRST HEALTH PROJECT Team Leader: Ruth E. Levine TL Unit: LCSHH ICR Type: Core ICR Report Date: January 31, 2002 1. Project Data Name: HT- FIRST HEALTH PROJECT L/C/TF Number: IDA-20850 Country/Department: HAITI Region: Latin America and Caribbean Region Sector/subsector: HC - Primary Health, Including Reproductive Health, Chi KEY DATES Original Revised/Actual PCD: 10/15/1987 Effective: 06/20/1990 08/02/1990 Appraisal: 11/28/1989 MTR: 02/20/1998 02/20/1998 Approval: 01/16/1990 Closing: 06/30/1996 03/31/2001 Borrower/Implementing Agency: GOVERNMENT OF HAITI/MINISTRY OF HEALTH Other Partners: CANADLIN INTERNATIONAL DEVELOPMENT AGENCY (CIDA) STAFF Current At Appraisal Vice President: David De Ferranti Shahid Husein Country Manager: Orsalia Kalantzopoulos Sector Manager: Charles Griffin Team Leader at ICR: Ruth Levine Xavier E. Coll ICR Primary Author: Ruth Levine; Joelle Dehasse; Girindre K. Beeharry 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U-Unsatisfactory, HL-Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: UJN Institutional Development Impact: M Bank Perfornance: U Borrower Performance: U QAG (if available) ICR Quality at Entry: S Project at Risk at Any Time: Yes 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The original objectives of the US$33.7 million project were to (a) reduce central administration expenditures of the Ministry of Health (Ministere de la Sant6 Publique et de la Population, or MSPP); (b) reduce the current ratio of salaries to operating expenditures; (c) improve collection of patient contribution to the cost of medical services and drugs; (d) strengthen key management and technical functions of the MSPP; (e) reduce maternal and child mortality rates; and (f) counter the potentially devastating effects of AIDS and tuberculosis epidemics (Development Credit Agreement, February 2, 1990). The project's objectives-and particularly those that emphasized structural and budgetary changes at the MSPP (objectives a, b, c and d)-were based on a 1987 IDA sector review. That review revealed that the MSPP had grown to be one of the largest government ministries, accounting for 16 percent of government expenditures by the latter half of the 1980s. Central administration expenditures, including salaries, absorbed nearly one-quarter of the budget; overall, salaries accounted for up to 90 percent of the MSPP budget, crowding out virtually all complementary inputs. Ministry staffing was top-heavy, with a critical shortage of nurses, technicians and auxiliaries. Health personnel were concentrated in Port-au-Prince and other urban areas, while in rural areas there was less than 1 physician per 10,000 inhabitants. The sector review concluded that the deteriorating health conditions in Haiti could be countered most effectively by addressing underlying financial, organizational and managerial imbalances that characterized the health system. The review called for reallocating public resources in favor of primary health care through increased financing of non-salary operating expenditures; decentralization of management, including delegation of operational responsibility to the district level; and greater participation of non-governmental organizations in primary health care. Additional objectives that were more directly related to health outcomes (objectives e and f) reflected a recognition that Haiti's health conditions were by far the worst in the hemisphere. It was decided that the Bank had a role to play in improving basic MCH services by providing badly needed financing for non-salary inputs (both capital and recurrent), and in strengthening the government's ability to address the threat of AIDS and TB. The objectives were based on the analysis of sector priorities, and were consistent with the Bank's overall country strategy. At the same time, the objectives reflected optimistic assessments of the quality of governance, readiness for organizational change, future political stability, and institutional capacity. They were overly ambitious, given Haiti's weak track record in implementation. 3.2 Revised Objective: The project objectives and closing date were revised three times. Overall, the changes resulted in a major increase in the duration of the project, and a shift in emphasis away from large-scale institutional change toward a strengthening of specific, high-impact elements of public health and basic health service delivery. In September 1991, about a year after implementation began, Haiti experienced a coup d'etat, a subsequent three-year international embargo and suspension of all IDA activities. When the project resumed in March 1995, the Credit Agreement was amended (effective November 1995) to streamline project implementation, set up a Special Account, and update provisions for disbursement and procurement. The closing date was extended by two years, to June 30, 1998. In February 1998, after three consecutive years of operation, a midterm review concluded that the - 2 - development objectives were unrealistic, given the Haitian context. The closing date was extended by 18 months, to December 31, 1999. The objectives and targets were simplified to reflect what had been attained to date, and what-in the view of the Bank-could be achieved during the remaining life of the project. The modifications also took into account the activities of other development organizations, including U.S., Canadian and other bilateral donors. The revised objectives de-emphasized large-scale changes in Ministry organization and budgeting, and were to: (a) contribute to strengthening of the MSPP's cost recovery mechanisms and service delivery strategies and standards; (b) improve access to and utilization of basic health services, with particular emphasis on maternal and child health care; and (c) support interventions to increase utilization of tuberculosis treatment services and expand awareness of the AIDS epidemic. In December 1999, within days of the project closing, the objectives and closing date were once again revised. At the time, Bank staff and management recognized that it would be desirable to close the project and initiate design of a new operation. However, the political context of the time-and particularly the absence of a functioning Parliament-made it impossible to foresee negotiation and ratification of a new loan. Therefore, the Bank decided to extend the project to maintain a presence in the sector, and to address critical health needs-particularly in the area of TB-that required some bridge financing until new donor relationships could be established. Bank management extended the project for 15 months, to March 31, 2001, on the understanding that the remaining funds (roughly US$4 million) would be used for essential drugs, maternal and child health, and preventive and treatment activities related to tuberculosis, HIV/AIDS and other sexually-transmitted infections (STIs). It was agreed that the project would no long finance civil works. Reflecting areas in which the project had demonstrated some successes, revised objectives were to: (a) improve maternal and child health; (b) reduce infections related to AIDS/STIs and TB; and (c) improve the administration and supply of essential drugs. 3.3 Original Components: Originally, the project consisted of three components: (a) Institutional Development; (b) Health Services Delivery; and (b) Epidemic Prevention and Control. Project activities were to be implemented by a Project Management Unit (PMU), which operated outside of the rules and procedures of the Ministry, and was physically separate from the Ministry. Institutional Development (estimated at US$4.1 million). This component sought to provide support for restructuring the MSPP to rationalize and decentralize staff. It promoted the development of simple budget allocation indicators to monitor and improve the effectiveness of the use of public resources. Health personnel-most of whom were in Port-au-Prince-were to be redeployed to rural areas in the Western Health Region, covering 2.1 million people in four districts (Port-au-Prince, Croix des Bouquets, Jacmel and Petit Goave). This component also was designed to strengthen the Ministry's capacities to recover costs for medical services ard drugs, manage civil works, and procure and distribute drugs. Most of the inputs financed under this component fell into the category of technical assistance. Services Delivery (estimated at US$21.2 million). This component was designed to improve the delivery of health services at the local level, under a decentralized regime in the Western Health Region. The four-part strategy included: (a) creating 30 commune-based local health systems; (b) supporting 7 priority service delivery areas (maternal and child health care, family planning, nutrition, immunizations, control of diarrheal diseases, TB prevention and treatment, malaria control, and HIV/AIDS prevention); (c) promoting the delivery of standard, basic health services in periodic community gatherings; and (d) integrating the private sector in basic health service delivery. Inputs financed included upgrading of the - 3 - health center/dispensary network, medical equipment and supplies, vehicles, training, essential drugs, per diem for sanitary officers and technical assistance. Epidemic Prevention and Control (estimated at US$8.5 Million). This component, implemented on a nationwide basis, focused exclusively on the development of an integrated tuberculosis control program, and support for HIV/AIDS prevention activities. The TB control program comprised: (a) conversion of the treatment regime to a multiple-drug short course; (b) expansion of treatment coverage; (c) maintenance of BCG immunization at a 60 percent rate; (d) expansion of profesional and para-professional training; and (e) provision of expert supervision and evaluation. Major inputs included drugs, training, and diagnostic materials and equipment Inputs financed to sUppott to HIV/AIDS prevention included laboratory testing equipment; information, education and communication materials; protective equipment and supplies for health personnel; and an AIDS-TB conference. Many of the aims of the project were in line with the then-government's goals and capacities, and thus had a reasonable chance for success, given the initial knowledge base. The largest share of project investments-for improvement of basic health services-closely followed the stated government programmatic intentions, supporting the basic services that were identified by the Ministry as high pnority. In contrast, the institutional strengthening activities, although seen by the Bank as essential to long-term effectiveness and sustainability of public health work in Haiti, did not fit well with the underlying capacity or political will. Many of the fiscal reforms, for example, depeded not on the Ministry of Health but also on the Ministry of Economy and Finance, which did not have the commitment to follow through. 3.4 Revised Components: Revisions across all components genermally represented a scaling-down of the project's ambitions, in light of political turbulence and weak governance; an increasing recogition of the institutional limitations of the Ministry of Health; and a reorientation toward core public health activities. In November 1995, the first amendment to the Credit Agreement specified the following revisions: The Institutional Development component was narrowed to reinforce a set of specific capacities within the MSPP, including: (a) developing and implementing policies for public health, public safety, health emergencies and health technology; (b) setting priorities for health spending and research; (c) determining standards for service delivery and ensuring compliance; (d) maintaining a national health information system; and (e) setting norms for the training and licensing of health workers (Credit Agreement, Amendment 2). In November 1998, the second amendment to the Credit Agreement specified the following revisions: In Institutional Development, the component was further narrowed to strengthen the MSPP's capabilities in the design and implementation of (a) cost recovery stategies, stating with basic analytic work on health service financing and costs; and (b) service delivery strategies and standards for essential drugs, quality control of pharmaceutical products, medical emergencies and control of TB and AIDS/STIs. Inputs include technical assistance, staff training and establishrment of norms and protocols. In Services Deltvery, support remained similar to the original conception, but was more highly specified under six objectives and several sub-objectives. For most of hese, the objectives were expressed in terms of the inputs to be provided rather than as objectives in the traditional sense (e.g., the provision of medical equipment, radio communications systems and related training to health personnel and community representatives in the South-Eastern Health Region). -4 - Under Epidemic Disease Prevention and Control, the component was modified to take advantage of up-to-date global knowledge, and support the MSPP's (a) Integrated National Tuberculosis Control Program, including the introduction of the Direct Observed Therapy Short-term (DOTS) strategy, following the WHO protocol; and (b) AIDS/STIs Program through implementation of social marketing and information, education and communication (IE&C) strategies. In December 1999, the third amendment was issued, with the agreement that, as of January 2000, the remaining project funds (roughly US$4 million) would be used for public health programs of the highest priority for the sector: essential drugs, maternal and child health (including immunization campaigns), TB and AIDS/STIs. No civil works were to be financed. The closing date was extended to March 2001. 3.5 Quality at Entry: Mardnallv satisfactory. While the project objectives fit well with the prevailing understanding of issues in the Haitian health sector, and the Bank's emphasis on social sector investments in very poor countries, several shortcomings in the design can be identified. First, as noted earlier, there was insufficient national commitment to institutional restructuring. The Ministry of Economy and Finance, whose cooperation was essential for reorganizing the budget, did not have a sense of ownership of the program. Second, the design assumed that certain changes in personnel changes could be made within the sector, when they in fact required a government-wide transformation. Third, the Ministry's capacity to implement multiple activities was overestimated. Fourth, the design reflected a conceptual confusion between inputs and objectives. Fifth, judged by current standards, the project was excessively supply-oriented, with little attention to dimensions of demand and community participation. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: Unsatisfactory. The project failed to achieve most of its major relevant objectives. Overall, public health in Haiti today cannot be said to have improved over the course of the past ten years, and there is no evidence that the project ameliorated a significant share of the health problems associated with general political, economic and social upheaval. If measured by the original health outcomes that the project sought to achieve (per the Staff Appraisal Report)-reducing mnaternal, infant and child mortality by 40 percent by 1995-then the project was unsatisfactory. In addition, virtually all the institutional development objectives were abandoned during implementation. However, the project did have some important successes after design revisions were made to strengthen core public health functions, particularly in provision of essential drugs and in certain elements of Epidemic Prevention and Control (see details in Section 4.2). 4.2 Outputs by components: Institutional Development: Unsattsfac.orv. Failures are seen in each of the major elements of this component, as it was originally conceptualized. First, at the start the project sought to promote the reallocation of the MSPP budget away from salaries, and toward complementary inputs-correcting a distortion of long-standing. Soon after the project's launch, political events in fact led to an increase in spending on salaries. By 1998, the likelihood of achieving a better balance between spending among administration, personnel and other inputs within the health sector was seen as so remote that a covenant calling for reduced salary ratios and central administration expenses was dropped from the Credit Agreement. It is interesting to note, however, that in 1999, staff salaries were brought down to 79 percent of the budget (from 90 percent in 1987), when the government launched a national public sector reform program. This implies that the transformation required government-wide action, and could not be realized - 5- within the health sector alone. Second, although a study was conducted on cost recovery options, the recommendations were found to be overly ambitious. Several additional minor steps were taken to strengthen cost recovery, such as some consultancies, but there is no evidence of impact. Third, redeployment of personnel to rural areas was never stimulated by the project, despite the original intentions. Recognizing the low probability of success, the covenant related to salary expenditures for staff at health centers outside of Port-au-Prince was later dropped. Although the institutional restructuring component was reduced in scale and scope during the multiple project revisions, the reorientation toward core public health functions did have positive spill-over effects at the institutional level. For example, progress was made in applying standardized therapeutic protocols in the national tuberculosis program, and in introducing drug treatment protocols for the essential drugs program. The public-private partnerships developed under the TB program increased institutional capacity to participate in contractual relationships. Modest improvements were also ultimately made in the Ministry's introduction of cost recovery measures. The essential drugs program relied on revenues generated at the health facility level to finance the availability of drugs at the peripheral level. Services Delivery. Unsatisfactory. Crudely, the project can be judged by comparing its original outcome targets with current health conditions reported in the 2001 World Development Indicators and elsewhere. While changes in indicators such as infant mortality are the result of many factors outside the health system, and cannot be attributed to the successes or failures of the project, it is clear that health conditions have not consistently improved (and that measurement problems continue to exist). For infant mortality, the SAR target was a decrease by 15 percent by the end of the third year of the project; and 35 percent by the end of the sixth year (i.e., from 119 per 1,000 live births in 1987 to 78 per 1,000 by 1995). Haiti's IMR was estimated at 70 per 1,000 in 1999, suggesting that improvements have been made. For child mortality, the story is quite different. The project was to contribute to a reduction by 15 percent by the end of the third year; and 40 percent by the end of the sixth year (from 22 per 1,000 in 1987 to 14 per 1,000). Haiti's under-five mortality was estimated in 1999 at 118 per 1,000, indicating either a very rapid deterioration in health conditions or very poor measurement, or both. For maternal mortality, the project was designed to contribute to a reduction of maternal mortality by 15 percent by the end of the third year; and 40 percent by the end of the sixth year (from 267 per 10,000 live births to 220 per 10,000). The rate of maternal mortality in 2000 is estimated by the MSPP at 523 per 10,000. Again, this implies some combination of worsening health conditions and measurement error. Looking at output measures, which can be more directly attributed to project implementation, results are disappointing. Of the 28 health facilities to be constructed (5 health centers and 6 dispensaries) and rehabilitated (16 health centers and 1 dispensary), 3 health centers were constructed and 2 were rehabilitated. Three of the seven planned shelters/housing were completed. Half of the 10,000 planned latrines were built; however, 3,250 additional latrines were built in areas for which they had not originally been planned. Thus, including both planned and unplanned works, construction of latrines reached an 83 percent completion level relative to the original figures. The rehabilitation of 10 drug warehouses and 4 community centers not originally planned brings the total of completed civil works to 8,272 out of the 10,036 originally planned (or 82 percent of the works). The quantity of civil works fell short of expectations primarily because per-facility costs were higher than expected. Facilities were found to be in worse shape than anticipated at appraisal (about six years earlier), and in several cases had to be replaced instead of rehabilitated. -6 - Whether the physical improvements supported by the project had a health impact remains an open question. A 1999 evaluation study found that upgraded facilities were not utilized more than other facilities. In spite of commitments made to the Bank, the MSPP had difficulties recruiting and retaining staff to work in remote areas, and thus the facilities often operated below capacity. Bank supervision missions attempted to persuade Ministry authorities to delegate responsibility for the management of these facilities to non-governmental organizations, which have a better track record in operating health centers, but these efforts were unsuccessful. Procurement and distribution of drugs and medical supplies was unsatisfactory during much of the project's implementation. Chronically, Hlaiti has suffered (and continues to suffer) fiom a lack of coordination and poor assessment of health centers' needs, resulting in inadequate distribution and wastage. In some instances, inexperienced health staff simply do not know how to use the materials; in other instances, a lack of storage space and/or missing installation equipment has meant that drugs or equipment have not been put into service. The project did not successfully address these systemic problems. The project can take credit for some notable improvements in the essential drug supply after the design revisions, however. There is documentary evidence of: (i) increased availability of drugs at affordable prices; (ii) improved management of patient care at the peripheral level; and (iii) introduction of basic cost recovery measures. An external evaluation conducted in 1999 found that essential drugs were available in a high percentage of health facilities (95 percent of dispensaries; 80-90 percent of health centers; and all hospitals). Field visits conducted during supervision missions confirmed that drugs were available even in remote areas. Similarly, spot checks confirmed that prices of essential drugs distributed through the national program (PROMESS) were dramatically lower than in private pharmnacies (i.e., two- to seven-fold differentials). The achievements in the essential drugs program, which was essentially managed as a project of PAHO, were found nationwide, and were not limited to the project areas. At least in quantitative terms, training and technical assistance was satisfactory, with more than 800 health care staff taking training in medical and administrative topics. Epidemic Prevention and Control. Satisfactorv. The success of this component is due to the partnerships between the public sector and non-governmental organizations (NGOs) that implemented the activities, even under difficult circumstances. In addition, this component benefited from constructive coordination with PAHO/WHO, CIDA and USAID, all of whom were involved in various ways and agreed to support the interventions after project closing. The TB Control component had a very satisfactory outcome. As planned in the SAR, the project contributed to the establisit .,ent of a renewed National TB Control Program, based on the WHO-recommended DOTE ,iodel. The program successfully involved various partners and operated within the primary health c a infrastructure. Detection of smear-positive cases increased from 5,493 cases in 1997 to 6,828 in 1999. fie proportion of smear-positive cases treated under DOTS rose from 0 in 1996 to more than 30 perce-. in 2000. Treatment success for smear-positive cases detected in 1998 reached 79 percent for those treated under DOTS, compared to 63 percent in non-DOTS areas. Even in areas where the DOTS strategy is not yet used, success rates have improved. In addition, a quality control system has been established for laboratories that diagnose TB. Despite progress, the TB control results are relatively fragile. More than 50 percent of estimated TB cases are not detected, reported and treated, and the epidemic is worsening due to its association with HIV/AIDS. The success to date has resulted in large part from the formal collaboration among MSPP staff, facilities - 7 - and NGOs. This relationship was complicated by the short contract periods provided under the project between the NGOs and the MSPP, the delays frequently experienced in renewing the contracts during the course of the project, and long delays in obtaining reimbursement for expenses. Due to limited access to vehicles provided to departmental-level authorities during the project, adequate field supervision was inhibited. On the positive side, the objectives, work products and indicators utilized in the contracts improved over time, and technical staff in the Ministry and PMU staff worked closely in communicating with NGO partners. The public-private collaboration supported by the project offers a useful model for future public health interventions. Under Haiti's difficult conditions, developing the necessary administrative and personal relationships to generate successful collaboration was and will continue to be time- and resource-intensive. With respect to AIDS/STI prevention activities, results are more difficult to quantify and have not been formnally evaluated, but are generally viewed positively. NGOs conducted wide-ranging awareness-raising activities (conferences, seminars, workshops, outreach to youth, caravans, community meetings, condom distribution and so forth), testing and counseling. Observers note that the project contributed to enhanced HIV screening capabilities, with the establishment of five departmental screening centers; and improved access of vulnerable groups to condoms through the social marketing program. 4.3 Net Present Value/Economic rate of return: N/A 4.4 Financial rate of return: N/A 4.5 Institutional development impact: Modest. While some 800 health care personmel received project-funded training, there is little evidence that the new skills were applied systematically. The strong management skills of the PMU, though invaluable for the implementation of the project, were not formally transferred to the MSPP. NGOs participating in the TB control and HIV/AIDS activities under the project were able to increase their capacity as contractors, in both technical and administrative terms. Although NGOs have for many years played a critical role in operation of health facilities-and often have outperformed the government in this area-the project further strengthened their ability to contribute to sectoral goals. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: This project-as the whole Haiti portfolio-was strongly affected by instability, civil unrest and political crises. Project implementation was brought to a halt about a year after implementation began, following the September 1991 coup d'etat, which was followed by an international embargo and a three-year suspension of the IDA program. The coup and embargo devastated the economy, with grave repercussions throughout society-from households to government agencies. In 1997, two years after the project resumed, the resignation of the Prime Minister further strained the political climate and distracted the administration's attention from development issues. The political crisis became still more pronounced after the May 2000 elections, when procedures for establishing winners of legislative and local elections were seen by many as unconstitutional and fraudulent. Presidential elections went ahead in November 2000, despite an opposition boycott; a new administration came into power in February 2001, headed by President Jean Bertrand Aristide and Prime Minister Jean Marie Cherestal. The political crisis persisted and new international assistance to Haiti remained on hold. - 8 - Throughout these events, the government has proven unable to fulfill core public health functions, not adequately financing the provision of public goods (e.g., vector control) and goods with externalities (e.g., immunizations). The breakdown of the health system, coupled with the high level of poverty, has made the Haitian population extremely vulnerable to the emergence, re-emergence and rapid spread of infectious diseases. Haiti has become an incubator of diseases for the region; the situation is especially worrisome for vaccine-preventable diseases, HIV/AIDS, and multi-drug-resistant TB. 5.2 Factors generally subject to government control: A lack of commitment on the part of the Government, in particular the successive Ministers of Health and Ministers of Finance, undermined the project from the outset. This factor, coupled with the high turnover of administrations and ministers (3 administrations, 4 Ministers of Health), meant that priorities were frequently being reevaluated. The First Health Project was unable to gamer full ownership of successive administrations. During project implementation, the culture and institutional incentives present in the MSPP were rarely conducive to efficient execution. Ministry officials faced a series of natural and human-made emergencies (e.g., measles epidemics, drug shortages, hurricanes, strikes), and viewed the project as a pot of money to be used to resolve some of these problems. In such a context, insufficient attention was devoted to systemic issues affecting implementation, and to the long-term goals of the project. This situation was exacerbated by a lack of a tradition of programming and monitoring project activities. On several occasions, the Government of Haiti failed to fulfill basic responsibilities. For example, during calendar year 2000, the government frequently fell into arrears with IDA repayment, resulting in several suspensions of IDA disbursements. These portfolio-wide suspensions permitted exceptions for the procurement of drugs for maternal and child health, TB and AIDS/STIs. From November 2000 through February 2001, an overdue audit report-largely the result of lack of performance by a private auditing firm-led to the suspension of disbursements for activities under the project, causing several activities to stall. As a result, many of the potential benefits of the final extension were not realized. 5.3 Factors generally subject to implementing agency control: In the midst of the political turmoil affecting Haiti, the PMU was able to maintain a relatively high degree of efficiency and effectiveness. Strong management skills and remarkable continuity in key personnel insulated the project, to some degree, from a general envirornment of chaos during several periods. The PMU Coordinator proved able to work across political divisions, which benefited implementation tremendously. And, by working with NGOs, the PMU was able to achieve impressive successes in TB prevention and control activities. The PMU was isolated physically and institutionally from the Ministry-a situation that simultaneously helped and hindered its effectiveness--and therefore had limited capacity to ensure that the project was integrated into the Ministry's core functions. 5.4 Costs andfinancing: The total cost of the project was estimated to be US$33.7 million, of which US$28.2 million equivalent (SDRs 22.2 million) would be financed by the IDA credit; the govemnment was to allocate the equivalent of US$3.1 million (in salaries). CIDA was to co-finance on a parallel basis the equivalent of US$2.4 million. The base costs to be supported by the project were estimated at US$28.7 million, with contingencies of US$5 million. The foreign exchange component (for imported materials) were estimated at US$22.4 million, or 66 percent of total project cost. The costs also included the refinancing of a project preparation -9- facility of US$560,000, approved in October 1987; and increases of US$190,000, approved in April 1989, and of US$150,000, approved in November 1989. The original allocation of project resources was modified along with the changes in project content. The Institutional Development component, originally programmed to absorb US$4.1 million, eventually used about US$1 million. The Services Delivery component was originally budgeted at US$21.1 million, and ultimately used about US$17 million. For the Epidemic Prevention and Control component, which was budgeted at US$8.5 million, approximately US$10 million was expended by the close of the project. Final disbursements under the project amounted to US$29.1 million equivalent (SDRs 21.1 million), resulting in a cancellation of US$1.3 million undisbursed. (Due to a chance in the US$/SDR exchange rate, the total disbursed amount in US$ exceeds the commitment amount.) Thus, in the end, 11 years after the start of the project, the project had disbursed 95.2 percent of the original credit amount. 6. Sustainability 6.1 Rationale for sustainability rating: Unlikely. The many activities that were initiated under the project-and particularly the institutional development interventions, civil works, stocking of health centers-are not sustainable without government support. That support is unlikely due to both the government's on-going fiscal crisis, and lack of institutional commitment. 6.2 Transition arrangement to regular operations: Several of the project activities constituted one-time investments that can be appropriated by the government with no further outlays. Others, which require maintenance, additional salary support, and other recurrent expenditures, are currently on shaky ground. There are no plans for project activities to be part of the Ministry's regular operation. However, certain project components-in particular, the TB and AIDS work-are expected to continue at some level with external support. IDA currently is disengaged from Haiti and will likely remain so until the pending issues related to governance and arrears have been resolved. The Bank is processing a number of grants that will allow it to capitalize on the achievements of the First Health Project despite the current IDA disengagement. A first grant of US$50,000 from the Japanese government has been approved. This grant will serve to assist the preparation of the National HIV/AIDS Strategic Plan, which will provide the framework for all future donor operations in HIV/AIDS in Haiti. A second grant of US$250,000 from the World Bank's Post-Conflict Fund also has been approved. This grant will assist the efforts of several agencies to contain the recent polio outbreak in Haiti. A third grant request (for an approximate amount of US$1 million) in support of HIV/AIDS prevention and related activities that will be implemented by NGOs will be submitted shortly to the Post-Conflict Fund. In addition to these grants, the Bank has sought to maximize the benefits of the First Health Project by seeking alternative sources of financing to maintain the key activities. In collaboration with WHO, the Bank has thus successfully negotiated relay financing from other donor agencies to maintain support to the National Tuberculosis Program . The U.S. Agency for International Development and the Canadian Government have developed projects to support the Tuberculosis Program beginning this year. All fundamental functions of the Program that were formerly financed from the First Health Project, including drug supply, will be maintained and/or further developed. It is expected that the Tuberculosis Program will continue to expand its tuberculosis case detection and its improvements in cure rates. 7. Bank and Borrower Performance - 10- Bank 7.1 Lending: Satisfactory. After more than 11 years, in the light of sweeping changes in Haiti's political landscape, it is extremely difficult to judge the Bank's lending performance in health. It met the minimum criteria of identifying priority issues for the sector. However, the Bank overestimated institutional capacity for financial management and procurement. In addition, in retrospect it is clear that the political risks were far greater than had been anticipated during project design. 7.2 Supervision: Unsatisfactor. Projects in high-risk countries such as Haiti require intense supervision, yet unrest or political upheavals frequently prevent timely supervision missions and/or field visits. As shown in Annex 4, there were several significant gaps in supervision due to security concems; those supervision missions that did occur often were restricted to Port-au-Prince. Thus, judging supervision performance against a "gold standard" for timeliness is difficult, but it is clear that the Bank supervision teams made every effort to conduct missions as regularly as possible.. Supervision missions adequately reported on project implementation progress, made impressive efforts to mobilize technical collaboration from other agencies (WHO, UNAIDS and PAHO), and quickly pointed out design flaws and implementation problems. On occasion, follow-up on recommendations was weak, due in part to frequent changes in ministerial leadership, as well as the lack of integration of the PMU into the Ministry. In addition, there were several changes in project task managers (six), sector managers, resident representatives and country directors-as well as the Bank's project management structure and processes-over the life of the project. Despite the supervision teams' satisfactory performance in the face of very difficult country conditions, questions can be raised about the wisdom of the decisions taken by the Bank regarding the orientation and extension of the project. Specifically, several observers question whether the decision to focus project resources on public health activities, and to minimize attention to institutional reforms, was a correct one. If it was clear that the political will for institutional change was absent, then even the most pro-poor and urgently needed public health activities-such as those supported by the project in its later phases-could not be expected to be sustained. In addition, given that Bank management was aware of the lack of Borrower commitment and the low probability of sustainability, it is reasonable to ask whether an extension to the closing date should have been granted. 7.3 OverallBankperformance: Unsatisfactory. The Bank made a concerted effort to design and supervise the project well, in the face of severe limitations. However, the Bank demonstrated questionable judgment the change in orientation of the project and the extension of the closing date. Borrower 7.4 Preparation: Satisfactory. Documents indicate a relatively high degree of Borrower involvement during the design phase. 7.5 Government implementation performance: Unsatisfactorv. Throughout the project, political turbulence, lack of continuity among key personnel within the Ministry, uneven commitment to project aims, and isolation of the PMU from the Ministry's line operations contributed to the operation's poor performance. 7.6 Implementing Agency: Hi0hlv Satisfacton. The PMU possessed strong management capabilities, had a strong commitment to the aims of the project, and was responsive to Bank rules and regulations. Within the Haitian context, the PMU stood out as a remarkably effective management team. While compliance with substantive covenants was poor-and, in fact, many of the key covenants related to institutional strengthening were dropped as the project was revised-compliance with most routine administrative covenants was adequate. 7.7 Overall Borrowerperformnance: Unsatisfactory. Many positive aspects of the implementing agency's performance were dwarfed by the overwhelming constraints associated with the political and economic upheaval affecting Haiti during the 1990s. Table 1: Histry of Status of Legal Covenants Agreement Covenant Present Original Revised Description of Covenant Comments ISection Type Status FuMilment FuMfilmlent Date Date 2.03 5 After Delay 06/30196 03131101 Closing Date Requested for Haig's Compiled with Pubic Health needs 2.06 2 Partially 01115191 To be Commitment and Services Charges Lack of payment and Complied wfith doterminded arriers occasioned during projecrs implementation 3.01 (b) 5 After Delay 09120/2000 06/08/2001 Project Management Unit Completion Delay due to data Complied with Report colebtion within the PMU 3.01(c) (i) 10 Complied with 08/02/1990 CoordInating unit 3.01 (e) 3 Complied with 6/20/1990 Maitbin an account in Gourdes and make adequate deposits In the Project Account, so as to maintain an average quaterly balance not less than the equivalent of $250,000 or such other balance as agreed upon between the Borrower and the Association 3.03 (a) 12 After Delay 04/30/1991 12/31/1997 Undertake a study for purposes of Due to the 1991 Complied with Pert A.2 of the Project and ensure that suspension of all IDA!s such study is completed in a timely activities in Hait fashion 3.03 (b) 12 Not Complied 09/30/1991 01/31/1999 Put into effect recommendations of Due to 1991 suspenslon with the above study as agreed upon with of all IDA's activities In the Association Haiti, and subsequente change in project focus 3.04 11 Partially 06/1/1991 07/18/1997 SubmHttotho Associaton the public Dueto the 1991 Complied with sector health budget and the plan of suspension of all IDA's Project expendiures for the following actities in Hait . year 4.01 (a) 01 Complied with 09/01/1991 The Borower shal have the Records and Accounts Onduding Special Accounts) for each fiscal year audited by hidependent auditors acceptable to the Assodation 4.01 (b) ('') 01 Complied wIth 03101/1991 The Borrower shall fumish to the Assodation no later than 6 months after the end of each fiscal year, a certfled copy of the audit by said auditors 6.02 5 After Delay 06/20/1990 08/02/1990 Effective Date Due to delays In Complied with establishing Special Account and because of change of authorities due to charnge of Governmenit -12 - Covenant Type: 1: Accounts/audft; 2: Financial perfbrmance/generate revenue from benefidaries; 3: Flow and utilization of Project Funds; 4:Conterpart Funding; 5: Management aspects of the Project or of its executing agency; 6: Environmental covenants; 7: Involuntary resettiement; 8: Indigenous people; 9: Montiing, review and reportng; 10: Implementation; 11: Sectoral or cross-sectoral budgetary or other resource allocation; 12: Sectoral or cross-sectoral regulatorymsutional action; 13: Other 8. Lessons Learned 8.1 Under the right conditions. public-private partnerships can help to mitigate institutional constraints. The TB program owes much of its singular success to the structured partnership between the public sector and committed non-governmental organizations, which were contracted to carry out well specified tasks. Through partnership with the NGO sector, the program was able to engender rapid change and scale-up to a more effective model of TB prevention and control. Key elements of this success were (a) the development of contracts with outputs that could be monitored well, and verified; (b) recognition and reinforcement of the role of the Ministry of Health in managing national programs; and (c) continuity and stability of funding for NGO activities. 8.2 Supply-driven public sector projects cannot overcome systemic constraints. Time and again, investments made under this project failed to yield the expected benefits because of the limited management and financing capacity of the public sector. Specifically, the physical improvements in the primary health facilities were never matched by additional or better trained personnel, and thus the ultimate benefits on health conditions had little chance of being realized. In retrospect, it might have made more sense to promote demand-side financing, which would permit use of health care providers outside of the public system. 8.3 Basic implementation capacity is essential. Throughout its implementation, the project suffered from the lack of fundamental management skills within the public sector. Baseline data were not collected; targets often were not set, or not set appropriately; activities were not programmed in a consistent manner, and sound monitoring systems were not established. Development and maintenance of these skills, which requires time, financial resources and institutional commitment, is a basic prerequisite to successful lending. 8.4 Governance of acceptable quality and basic implementation capacitv are essential for the success of a Bank oNeration. Throughout its implementation, the project suffered from the lack of fundamental management skills and rules of accountability within the public sector. Baseline data were not collected; targets often were not set, or not set appropriately; activities were not programmed in a consistent manner; and sound monitoring systems were not established. Prerequisites for future lending should include: (a) at least a minimally acceptable quality of governance; and (b) a commitment to d