Determinants of Health Productivity in Haiti

Public Disclosure Authorized DETERMINANTS OF HEALTH PRODUCTIVITY IN HAITI Public Disclosure Authorized Health, Nutrition and Population Public Disclosure Authorized Global Practice October6, 2022 Public Disclosure Authorized The World Bank 1818H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org 0 I. Context Understanding the key factors that underlie the efficiency of specific health programs is key to improve population health. This is especially true for Haiti, where utilization of health services (in the formal sector) on average is very low, even when compared to other low-income countries, and despite a relatively high density of health facilities per unit area (meaning that distance is not the main barrier to access). There is a particular need to evaluate policies that may enhance primary health care utilization and outcomes, since primary care has proven to be an effective platform for strengthening health systems in several countries. Potential interventions to be evaluated include community health workers as possible key determinants of health care utilization in Haiti. Limited healthcare resources in low- and middle-income countries (LMICs) have led policy-makers to improve healthcare productivity.(1) A simple indicator of efficiency, such as productivity of health facilities, measured using routine health facility data and demographic health surveys, as well as other data sources such as health facility surveys, is an approach that can be replicated and compared across different contexts (2). More specifically, productivity is defined as the ratio of the output to the inputs of any system. Therefore, a productive system is one which achieves higher levels of performance (outcomes, outputs) relative to the inputs (e.g. resources, time) consumed (3). For this study, we measure outcomes or outputs as total provision of services – either outpatient visits or vaccine provision – while the number of clinical health workers is the input measure for this study. The latter is a viable measure since health care workers account for the largest share of the total cost of functioning of health facility. Hence the productivity measure used is the number of patient visits, or the number of pentavalent vaccines provided, per clinical health care worker at a health facility. Health productivity in Haiti In the health sector, technical efficiency consists of achieving a maximum level of consultations or admissions to a health facility with a given level of inputs(4). Of the low-income countries, Haiti displays one of the lowest technical efficiency scores for all health facilities(5). For example, technical efficiency in Haiti was 4% in dispensaries, 9% in health centers without bed (centres de santé sans lit, CSLs), and 30% in health centers with bed (centres de santé avec lit, CALs). Figure 1 below shows the distribution of productivity by type of health facility. Despite productivity being low across health facilities, there is significant variability across health facilities, particularly amongst hospitals. Given the high variance in productivity amongst hospitals, this analysis focuses on primary-care facilities: health center with beds, health center without beds, and dispensaries. 1 Given that the completeness of indicators available on SISNU –Haiti’s National Unified Health Information System (Système d’Information Sanitaire National Unique)– varies significantly, this analysis focusses on two of the most reliable indicators: institutional visits and pentavalent vaccine doses. First, institutional visits per health facility is a measure of the overall volume of patients that accessed medical care at the institutional level. Second, pentavalent vaccine doses is a reliable indicator of children’s access to care during the earlier years of life as it is given at different points in time and protects against respiratory infections, one of the most common causes of consultations in children. Finally, Haiti has four types of health facility managing authorities: government/public (42% of all health facilities), private not for profit (14%), private for profit (23%), and mixed (21%). This analysis excludes private for-profit facilities because there is sufficient on-the-ground evidence that these facilities are significantly different from the others and failing to exclude them would bias the results. II. Data Datasets The following datasets were used for the analysis: - SISNU: Haiti’s National Unified Health Information System (Système d'Information Sanitaire National Unique). This government-led data repository provides information about population health indicators and institutional-level services. Information is available on different health system levels: national, departmental, arrondissement, commune, section communale, and health facilities; in addition to disaggregating by sex and five-year age groups. Finally, the data is disaggregated daily up to yearly 2 basis. We obtained clinic-level monthly data from 2017 to 2020 of institutional visits and pentavalent vaccine. - Haiti’s updated list of health country facilities- (Liste actualisee des institutions sanitaires du pays): This is a governmental database that provides the official —and most current— official name, identifier code (SISNU code), and geographical references of the 1072 active health facilities in the country. - Service Provision Assessment (SPA) Health Facility Survey of 2017. This dataset provides the characterization of health facilities about the infrastructure, resources, systems, and services available. They were used to obtain information on, among others, institutional and contractual health care workers, community health workers, and the type of health facility. - Donor programs: This dataset, incorporating data from the year 2018, comes from a previous effort of an intensive exercise to obtain data from several different donor programs in Haiti – including type of program and health facilities covered by each. This is key given the large number of bilateral and multinational donors in Haiti. Various consultations with different donor partners and various units from MSPP were carried out overall several months to arrive at this final dataset. Merging and matching datasets After identifying the datasets that were needed to fulfill the study objectives, a matching and merging exercise was carried out with the overarching goal of matching the different datasets at the level of the health facility. This exercise was time and resource intensive because the health facilities (HFs) in different datasets often had different names (for the same facility). Furthermore, over time some HFs ceased functioning, while other new ones started to function. This process had three stages: 1. Matching SISNU outputs with the SISNU codes Despite the previously mentioned comprehensiveness of the SISNU database, it has several limitations. For example, these health-facility names are not identical to those on the “Liste actualisee des institutions sanitaires du pays”, and for most cases, a manual matching process had to be carried out. 2. Matching SPA and SISNU databases After completing the SISNU health facility name to SISNU code matching, a separate process to match SPA-SISNU followed. First, given the SISNU code and SPA code are not comparable, the matching process was by the health facility’s name. Then, given that the names were not identical, another manual matching process followed. In the end, around 90% of all the SISNU-SPA were matched. 3. Matching the donor database Finally, once the SPA-SISNU match was finalized, the donor dataset was merged with the master donor dataset using the SPA code. A panel dataset was thereby created and used for the analysis. Note that for some key variables – those from the SPA and donor datasets – data were available only for one point in time (from 2017 for the SPA 3 dataset and from 2018 from the donor dataset). SISNU data were available for each month from 2017 onwards – with gaps for some months, for some facilities. III. Methodology and Equations Mixed effects regression is a generalization of linear regression, and Ordinary Least Squares (OLS) is a special case of a mixed effects regression. Mixed effects regressions contain both fixed (Xβ) and random (Zu) effects. More specifically, the fixed portion is analogous to a linear prediction from a standard Ordinary Least Squares (OLS) regression model with β being the regression coefficients estimated. The random effects u are not directly estimated, but are characterized by the variance components. y = Xβ + Zu + ε (1) Mixed effects regressions are particularly useful when there are significant differences in the relationship between the dependent and independent variables in different groups or clusters. In the present analysis, this is assumed to indeed be so for different clusters of health facilities – where health facilities are grouped in different clusters according to: (i) the Department where each is located, and (ii) the management status of each facility (public or otherwise). A version of the above is estimated for the analysis in this paper – a hierarchical or multistage formulation of mixed-effects models where each level is described by its own set of equations. Specifically, we estimate an equation with two levels, and the following is given as an illustrative example in the specific hypothetical case where there is just one independent variable. yij = γ0j + γ1jxij + єij (2a) γ0j = β00 + u0j (2b-i) γ1j = β10 + u1j (2b-ii) where: • yij is the value of the outcome (dependent) variable, productivity, for health facility i in cluster j • xij is the value of the independent variable for health facility i in cluster j • The equation for the intercept γ0j (2b-i) consists of the overall mean intercept β00 and a cluster specific random intercept u0j • The equation for the slope γ1j (2b-ii) consists of the overall mean slope β10 and a cluster-specific random intercept u1j This regression allows for the slope as well as the intercept to differ in different clusters, hence allowing a high degree of flexibility. In the case of OLS, the slope and the intercept are constrained to be the same in all clusters. The above equation (2) illustrates the special case where there is just one independent variable. In the equations estimated for this paper, a range of independent variables were introduced, as described and reported below. It was not possible to introduce random intercepts and slopes for all independent 4 variables in each equation since this would have been it computationally impossible (with difficulties with convergence) and would result in a less parsimonious model. Hence the focus – for introducing random slopes and intercepts – was on the variables that were more relevant from a policy perspective. Some equations were also estimated using OLS. This allowed for a large range of independent variables – or variations of independent variables that had previously been estimated using mixed-effect regressions – to be introduced and estimated, without computational difficulties (difficulties with convergence etc.) The clusters were defined as follows: Health facilities were divided according to the Department in which each was located (ten sub-groups), and by ownership status – public or not (two sub-groups). Facilities that were categorized as non-profit private or “mixed” were classified as non-public, while for-profit facilities were excluded from the analysis altogether. Hence, twenty clusters were created – based on the division by Department and by public/private – and incorporated into the mixed-effect regressions. The outcome variables (general visits and pentavalent vaccine doses, based on SISNU data) were log transformed to account for their skewness (long tails). And outliers (excessively high values) were also excluded from the analysis1. In addition, data were included from any one year only if the number of months with available data from SISNU exceeded six for that particular year. Finally, SISNU data from 2020 onwards were not included in the analysis, since this was the start of a different phase for health facilities in Haiti – due to COVID, and also due to the worsening security situation. Data from this phase were considered not to be compatible with data from the phase before 2020. 1 Outliers were defined as any health facility that has more than 100 daily institutional visits and reported less than four pentavalent vaccine doses per day. 5 IV. Summary Means Table 1. Daily total visits, 2018 Government public Private not for profit Private for profit Mixed Mean n Mean n Mean n Mean NA 111 8 NA NA 131 94 NA 121 21 195 NA 201 37 220 40 277 84 245 211 30 314 46 152 108 182 128 93 222 211 166 38 548 125 68 84 211 32 Level of facility n Department hospital Community reference hospital 35 Other hospitals 34 Health center with bed 116 Health center without bed 263 Dispensary/community health center 237Table 2. Daily efficiency by type of health facility*, 2018 Government public Private not for profit Private for profit Mixed Mean n Mean n Mean n Mean NA 2 8 NA NA 3 94 NA 4 21 6 NA 3 15 3 9 7 2 224 8 29 25 35 3 3 145 4 65 18 151 13 4 475 12 62 11 202 6 Type of facility n Department hospital Community reference hospital 31 Other hospitals 34 Health center with bed 88 Health center without bed 266 Dispensary/community health center 189 *efficiency= total visits/all health workers Table 3. Daily total pentavalent vaccine doses, 2018 Government public Private not for profit Private for profit Mean n Mean n Mean n Mean NA 78 10 NA 79 92 NA 120 27 120 NA 159 32 49 44 49 57 255 68 30 44 35 44 41 200 54 102 44 220 44 22 570 30 78 25 234 25 Mixed Type of facility n Department hospital NA Community reference hospital 27 Other hospitals 44 Health center with bed 34 Health center without bed 220 Dispensary/community health center 234 6 Table 4. Daily total deliveries, 2018 Government public Private not for profit Private for profit Mixed Mean n Mean n Mean n Mean NA NA NA NA NA 19 12 NA NA 95 96 NA 67 27 67 149 49 41 49 41 35 259 164 33 51 47 51 26 172 26 93 69 145 69 13 470 13 72 15 173 15 Type of facility n University hospital Department hospital Community reference hospital 27 Other hospitals 49 Health center with bed 47 Health center without bed 145 Dispensary/community health center 173 Table 5. Service Availability Readiness Assessment Score (SARA) Government public Private not for profit Private for profit Mixed Mean n Mean n Mean n Mean 66 5 NA 63 1 72 7 79 1 NA 67 25 NA 79 6 79 63 9 70 9 66 20 64 61 51 62 16 60 32 65 51 62 56 47 54 59 55 44 164 46 31 43 58 47 Type of facility n University hospital NA Department hospital NA Community reference hospital 8 Other hospitals 6 Health center with bed 25 Health center without bed 64 Dispensary/community health center 56 7 V. Vaccines - Results A. Core Regressions We fitted different versions of a multilevel mixed effects model to understand the facility-specific factors that predict a higher number of daily doses per health worker. Table 6 below provides a comparison of the three different model parameters described in more detail in Section III. Table 6. Basic Vaccine Models Outcome: daily doses of pentavalent vaccine (log-transformed) Model VA-1 Model VA-2 0.973*** (0.04) Model VA-3 One clinical healthcare worker -- 0.864*** (0.041) Two clinical healthcare workers -- 0.44*** (0.039) 0.363*** (0.041) Three clinical healthcare workers -- 0.261*** (0.04) 0.201*** Four clinical healthcare workers -- 0.094** (0.042) - 0.035 (0.046) Five clinical healthcare workers -- 0.073 (0.045) 0.064 (0.052) Six clinical healthcare workers -- - 0.329*** (0.051) - 0.318*** (0.055) Seven clinical healthcare workers -- 0.193*** (0.052) - 0.151** (0.062) Number of clinical healthcare workers -0.061** (0.022) -- -- Proportion of CHWs out of total number of health workers 1.021** (0.356) 0.84*** (0.26) 0.713*** (0.188) Facility has a microplan for vaccine program 0.1 (0.088) 0.102 (0.084) 0.078 (0.073) Facility has a strategy for communication of vaccines 0.629** (0.218) 0.571** (0.249) 0.31 (0.248) Score amenities 0.0001639 (0.003) 0.004 (0.003) -- Score equipment 0.001 (0.003) 0.003 (0.003) -- Total observations Number of clusters (Average observations per cluster) Mixed vs OLS p-value AIC 9,734 20 (486) 0.000 23659 9,734 20 (486) 0.000 23247 8,424 20 (421) 0.000 15477 8 Model VA-1: Initial Core Regression In this initial regression, we introduce several independent variables on the right-hand side that are potential predictors of vaccine productivity (number of doses given per clinical health worker).2 These independent variables, and the key findings of the regressions, are listed in Table 6 above. A few points to note here: • The independent variables include the Service Availability and Readiness Assessment (SARA) scores for facility readiness in terms of basic amenities and equipment, following the WHO methodology(6) for calculating SARA scores (and using data from the 2017 SPA survey). • The variables for which random intercepts and slopes were included (as part of the mixed-effects model) are: 1) the proportion of community health workers vs all clinical workers, 2) if the facility has a microplan for vaccine delivery, 3) if the facility has a strategy for communication of vaccines, 4) the SARA score for amenities, and 5) the SARA score for equipment. • The analysis only included outpatient facilities in which pentavalent vaccine and a refrigerator for cold storage were stated as available – a necessary condition to carry out vaccination programs – and facilities that offered only inpatient services were excluded from the analysis. (The latter was done because inpatient services are much more HR-intensive than outpatient services, and so the results may be adversely affected by including facilities that offer inpatient services only.) • Donor dummies were introduced for each major donor program, to correct for the fact that some of the right-hand side variables could otherwise be picking up the effect of donor programs. Key findings from the model VA-1 regression are: • Size of health facility (measured via the number of clinical health workers) is strongly negatively associated with health worker vaccine productivity. • The proportion of Community Health Workers (CHWs) out of the total number of health workers (CHWs + clinical health care workers or HCWs) is strongly positively associated with health worker productivity (for the penta vaccine). We estimate that an increase of one standard deviation of the CHWs variable results in a 30% increase in total vaccine provision, starting from the median value. • Having a strategy for communication of vaccines is strongly associated with health worker productivity in terms of pentavalent vaccine. We estimate that having such strategy results in a xx% increase in total vaccine provision, starting from the median value. (Sunil to calculate this value.) • Most of the donor dummies are highly significant. But due to a high degree of collinearity as well as these not being included in the random slopesin the mixed effects model, these results are not considered reliable and are reported. Rather, these donor program dummies should be seen as 2 Note that various other variables were also tried out in these regressions, which were not statistically significant. They were hence not included in the final regression reported. 9 important are for control purposes, to ensure that some of the right-hand side variables do not pick up effects of donor programs. Model VA-2: Variation for Clinical Health Care Workers Variable Here, instead of having a single variable for the number of clinical HCWs, we realize that this relationship is not linear. Hence, we introduce dummies for 1 clinical HCW, 2 clinical HCWs etc. But these dummies are not introduced as random slopes in the mixed effects model to maintain parsimony and allowing convergence in the mixed effects regression. Key findings for the model VA-2 regression are: • Results for CHWs and vaccine strategy are similar here as before. • All the clinical community health worker number dummies are highly significant, and there is a clear negative relationship between size of facility and productivity in terms of vaccines. Model VA-3: Including Only Health Facilities Offering Outpatient Services In this model, we generate a set of results where we exclude any health facility that offers inpatient services on a routine basis. This is because it is possible that the previously generated negative relationship between number of clinical health workers and productivity (in terms of vaccines) was due to the larger clinics offering inpatient services – which are more HR-intensive. Thereby we correct for this by including only facilities that offer outpatient services (as well as inpatient services if not on a routine basis). But we are forced to exclude the SARA facility readiness variables for amenities and equipment in these regressions, to ensure convergence. We adapt model VA-2 here, dropping the amenities and equipment variables and excluding the facilities that offer inpatient services on a routine basis. Key findings from model V-A-3 are that the results are not much different from before for the dummies for the numbers of clinical health workers, overall. But more specifically: • We see here that the negative relationship between number of clinical health workers and vaccine productivity is only there for the range of 1 to 3 clinical health workers. The most productive are the facilities with just one clinical health worker where each worker produces 0.86 more vaccines per day than the missing category of facilities (facilities with more than 7 clinical health workers) – which is almost one standard deviation (since standard deviation for the log vaccine variable is 1.01). This implies that vaccine productivity (vaccines per person) for 1-clinical-health-worker facilities is 2.36 times that of the missing category of facilities (facilities with more than 7 clinical health workers) (i.e. 136% higher). • Facilities with just 2 clinical health workers feature each worker producing 0.36 more vaccines per day than the missing category of facilities (facilities with more than 7 clinical health workers). This implies vaccine productivity (vaccines per person) being 43% times higher for 2-clinical-health worker facilities than for the missing category of facilities (those with more than 7 clinical health workers). 10 B. Digging deeper into the CHWs-related factors (for the vaccine regressions) Since the CHWs variable turned out to be highly and consistently significant (unlike the vaccines strategy which ceased to be significant when facilities offering inpatient services routinely were excluded), we dig deeper into this. The modeling outputs are shown in Table 7 below, and are described in more detail on the text below: TABLE 7. COMMUNITY HEALTH WORKERS AND VACCINE PRODUCTIVITY Outcome: daily doses of pentavalent vaccine (log transformed) Model VB-1 Model VB-2 Model VB-3 Number of clinical workers - 0.006 (-0.16) -- -- Proportion of ASCPs out of all workers 1.702*** (3.61) -- -- Proportion of ASC out of all workers 2.099*** (3.61) -- -- Proportion of supervisors our of all workers 0.96 (1.01) -- -- One clinical worker -- 0.781*** (0.042) -- Two clinical workers -- 0.526*** (0.038) -- Three clinical workers -- 0.496*** (0.044) -- Four clinical workers -- - 0.12** (0.053) -- Five clinical workers -- 0.188** (0.061) -- Six clinical workers -- - 0.201*** (0.053) -- Seven clinical workers -- 0.202*** (0.053) -- Table 7 continues in the next page. 11 TABLE 7. COMMUNITY HEALTH WORKERS AND VACCINE PRODUCTIVITY (CONT) Outcome: daily doses of pentavalent vaccine (log transformed) VB-1 VB-2 VB-3 Proportion of community health workers vs all workers: <25th percentile -- - 0.077 (0.088) -- Between 25th and 49th percentile -- 0.263** (0.088) -- Between 50th and 74th percentile -- 0.392*** (0.086) -- Between 75th and 89th percentile -- 0.665*** (0.09) -- Between 90th and 100th percentile -- 0.743*** (0.09) -- Proportion of CHWs when there is/are One clinical worker -- -- 0.106*** (0.016) Two clinical workers -- -- 0.081*** (0.016) Three clinical workers -- -- 0.059*** (0.016) Four to six clinical workers -- -- 0.053** (0.02) Seven to nine clinical workers -- -- 0.001 (0.016) Ten or more clinical workers -- -- - 0.027 (0.016) Total observations Number of clusters (Average observations per cluster) Mixed vs OLS p-value AIC 7,140 20 (357) 0.000 16332 6,065 OLS model (R2: 0.2453) 15477 8,424 OLS model (R2: 0.202) 20892 12 Model VB-1: Dividing the CHWs variable into sub-categories of CHWs First, we note that the CHWs are subdivided in the source data – the 2017 SPA survey – into three types: (i) Agents de Sante Communautaire Polyvalent (ASCPs) which are CHWs with a range of tasks to be done at the community level; (ii) Agents de Sante Communautaire (ASCs) which are CHWs that are supposed to specialize in specific tasks like malaria or HIV (though they may also perform other tasks in practice); and (iii) CHW supervisors. In model VB-1, we introduce these different sub-categories of CHWs separately – i.e. number of ASCPs, number of ASCs and number of CHW supervisors – instead of as one combined category (i.e. instead of number of CHWs in total). The mixed effects regression method is used as before, and in all cases, the CHW-related variables are included among the variables for which there are random slopes. We include only the clinical HCW and CHW-related variables in the regression (not the others such as the vaccine strategy variable) since convergence was otherwise not being achieved for the mixed effects regression. Key findings here are: • Both the ASCPs and ASCs variables are statistically significant. • However, the number of supervisors variable is not statistically significant. Model VB-2: Exploring a Non-Linear Relationship for the CHWs Variables We now probe if there is a relationship that is other than linear for the CHWs variable. From now on, since we found that the supervisors were not significant statistically, we separate out the CHWs variable into two parts: (a) ASCPs+ASCs, and (b) supervisors (which we do not include in all the regressions). For regression V-B-2 below, we divide the variable ASCPs+ASCs by the total number of CHWs plus clinical health workers. We term this new variable CH_ALLW for now. And, instead of introducing CH_ALLW as a continuous measure in the regressions, we include dummies instead to test for a non-linear relationship: (i) A dummy taking the value 1 if CH_ALLW is greater than 0 but less than its 25th percentile of 0.43 (ii) A dummy taking the value 1 if CH_ALLW is greater than its 25th percentile (0.43) but less than its median value of 0.57 (iii) A dummy taking the value 1 if CH_ALLW is between its median (0.57) and 75th percentile (0.7) (iv) A dummy taking the value 1 if CH_ALLW is between its median 75th percentile (0.7) and 90th percentile (0.8) (v) A dummy taking the value 1 if CH_ALLW is between its 90th percentile (0.8) and 1 We continue to exclude the facilities which offer routine inpatient services on a routine basis, and we only include the right-hand side variables which were significant in Regression VA-3 – i.e., only the variables for numbers of clinical health workers. To ensure convergence, we run this regression now using OLS. 13 Key findings: (i) Starting from a situation of zero CHWs, adding CHWs so that the variable CH_ALLW increases but remains below its 25th percentile (0.43) has no statistically significant impact. (ii) Above its 25th percentile, however, CH_ALLW has increasing impact whereby the higher it goes, the higher is vaccine productivity. This is a very interesting finding that has policy implications –in short: if you add CHWs to a facility, you need to add enough to have an impact. Model VB-3: Allowing for Differential Impacts of CHWs for Different Facility Sizes Next, we allow for differential impacts of CHWs for different facility sizes – i.e., for 1-clinical-health-worker facilities, for 2-clinical-health-worker facilities, for 3-clinical-health-worker facilities, etc. Here, we want to use a CHW variable that allows a clearer comparison between CHWs and vaccines. We create first a variable CH_CLW which consists of number of CHWs (ASCPs and ASCs) divided by number of clinical health workers. The left-hand side variable is now simply the number of vaccines per clinical health worker (without applying the log function). Hence both the left-hand side variable and the right-hand side CHW variable are now comparable in the sense that they are both scaled by the number of clinical health workers. We also use interaction terms to allow for differential impacts of the CH_CLW variable, for different facility size: i. CH_CLW_1 = CH_CLW for 1-clinical-health worker facilities, and zero otherwise ii. CH_CLW_2 = CH_CLW for 2-clinical-health worker facilities, and zero otherwise iii. CH_CLW_3 = CH_CLW for 3-clinical-health worker facilities, and zero otherwise iv. CH_CLW_4to6 = CH_CLW for facilities with 4, 5 or 6 clinical-health worker facilities, and zero otherwise v. CH_CLW_7to9 = CH_CLW for facilities with 4, 5 or 6 clinical-health worker facilities, and zero otherwise vi. CH_CLW_ge10 = CH_CLW for facilities with 10 or more clinical-health worker facilities, and zero otherwise We then run the regression. As before, we continue to exclude the facilities which offer routine inpatient services on a routine basis, and we only include the right-hand side variables which were significant in Regression VA-3 – i.e., only the variables for numbers of clinical health workers. To ensure convergence, we run this regression now using OLS. Key findings: • The results show that there is a clear negative relationship between the additional impact of each ASC/ASCP and facility size from 1-clinical-health-worker facilities to 2-clinical-health-worker facilities etc. until one reaches facilities with 4 to 6 clinical health workers. For larger health facilities, the impact of additional CHWs seems to be negative. 14 • Each additional ASC/ASCP adds 0.105 more vaccines for 1-clinical-health-worker facilities, which amounts to 36% more vaccines than the median of 0.288. Model VB-4: Separate Regressions for Facilities of Different Sizes Now, we run regressions separately first for one-clinical-health-worker facilities, then for two-clinical health-worker facilities, etc. The results are reported in Table 8 below. TABLE 8. FACILITY SIZE EFFECT Outcome: daily doses of pentavalent vaccine (log transformed) VB4-1 facilities with only one clinical worker VB4-2 facilities with two clinical workers VB4-3 facilities with three to five clinical workers VB4-4 facilities with six or more clinical workers Number of community health workers One 0.139 (0.139) - 0.133 (0.101) - 0.224 (0.152) - 0.613*** (0.17) Two 0.206 - 0.429*** (0.085) 0.986*** (0.139) - 0.18 (0.174) Three 0.159 (0.147) 0.183** (0.092) -0.03816 - 0.958*** (0.141) Four 0.821*** (0.142) - 0.371*** (0.097) 0.766*** (0.14) 0.257* (0.144) Five 1.065*** (0.163) -- 1.005*** (0.175) -- Six 0.284 (0.166) 0.64*** (0.092) 0.224 (0.15) - 0.557*** (0.133) Seven 0.852*** (0.138) 0.203** (0.101) 0.785*** (0.142) 0.607*** (0.131) Number of supervisors Total observations (R-squared) AIC 0.153** (0.057( 1,431 OLS model (R2: 0.21) 3487 0.055 (0.057) 1,626 OLS model (R2: 0.23) 3900 - 0.164** (0.053) 1,453 OLS model (R2: 0.40) 3235 0.093 (0.057) 1,555 OLS model (R2: 0.24) 3709 15 Findings: • For facilities with up to 5 clinical health workers, there is no clear positive impact of adding more CHWs until one gets to around 4 to 6 CHWs. • For facilities with more than 5 (6 or more) clinical health workers, the impact of adding more CHWs is rather unclear. • Out of all categories of facilities, only facilities with 1 clinical health worker show clear indication of any positive impact of having a supervisor. These results confirm the above findings (e.g. from Regressions VB-3). 16 VI. Visits to Health Facilities - Results To understand the factors that determine the productivity in terms of daily visits by patients per health facility (for any medical purpose). We fitted a similar multilevel mixed model as in the case of the vaccine’s regressions, and following the methodology described in Section III. As in the case of the vaccine regressions, we used the log-transformed version of the number of visits to account for the skewness of this variable across health facilities. And as before, we use monthly data from 2017 to 2019 for the outcome variable (number of visits per clinical health worker), and we exclude health facilities that provide inpatient services only and that are private-for-profit. In this initial regression model, we introduce several independent variables on the right-hand side that are potential predictors of vaccine productivity (number of doses per health worker).3 These independent variables, and the key findings of the regressions, are listed the table below. Core Regressions TABLE 9. VISITS TO HEALTH FACILITIES Outcome: daily visits (log transformed) VIA-1a VIA2-a VIA2-b VIA2-c Number of health workers One 0.643*** (0.063) 1.008*** (0.07) 0.651*** (0.037) 0.889*** (0.042) Two 0.16** (0.058) 0.464*** (0.063) 0.207*** (0.035) 0.371*** (0.04) Three 0.024 (0.053) 0.332*** (0.058) - 0.024 (0.035) 0.009 (0.039) Four - 0.215*** (0.05) 0.057 (0.055) - 0.085** (0.035) - 0.111** (0.042) Five - 0.247*** (0.05) - 0.045 (0.052) - 0.362*** (0.039) - 0.115** (0.047) Six - 0.256*** (0.045) - 0.162*** (0.047) - 0.22*** (0.042) - 0.177*** (0.05) Seven - 0.41*** (0.047) - 0.371*** (0.05) - 0.486*** (0.044) - 0.765*** (0.061) Proportion of community health workers vs clinical workers - 0.122 (0.192) - 0.014 (0.19) - 0.088 (0.0175) - - Service Availability Readiness Score Charges fees for each service 0.012** (0.004) - 0.462** (0.15) - - - 0.426** (0.155) - - - - - - - - Table 9 continues in the next page. 3 Note that various other variables were also tried out in these regressions, which were not statistically significant. They were hence not included in the final regression reported. 17 TABLE 9. VISITS TO HEALTH FACILITIES (CONT) Service Availability Readiness Score Rural facilities - - 0.013*** (0.004) - - - - Urban facilities - - 0.018*** (0.004) - - - - Overall 0.012** (0.004) Services - - - 0.432 (0.306) 0.101 (0.063) - - Amenities - - - - 0.004 (0.003) 0.004 (0.002) Precautions - - - - 0.001 (0.002) 0.000 (0.003) Equipment - - - - 0.000 (0.002) - 0.001 (0.003) Medicines - - - - 0.004 (0.003) 0.012* (0.005) Diagnostics - - - - 0.002 (0.002) - 0.001 (0.003) Charges fees separately - - - - - 0.468** (0.138) Total observations Number of clusters (Average observations per cluster) Mixed vs OLS p-value AIC 15,603 20 (780) 0.000 38079 6,065 OLS model (R2: 0.2453) 38322 8,424 OLS model (R2: 0.202) 37305 13,637 20 (681) 0.000 33180 Model VIA-1: Initial Core Regression To develop a model that most accurately describes the factors that determine the daily productivity, we first fitted different models other than the ones shown below. For example, including measures of quality such as the frequency of supervisory visits and whether the health facility has a system for measuring quality – and these were not statistically significant. In addition, we decided to exclude these measures because there are a myriad of unmeasured confounders and are not comprehensive measures of quality. Further, other facility characteristics such as cleanliness were not found to be significant and were excluded from the final model. In the initial core regression, we introduce several independent variables on the right-hand side that are potential predictors of productivity as measured by the number of visits per clinical health worker). These independent variables, and the key findings of the regressions, are listed in the table above. The independent variables include a measure of the Service Availability Readiness Assessment (SARA) index 18 developed by the WHO –– and this time, we include one single measure: a simple average of the scores for basic amenities, basic equipment, medicines, precautions, and diagnostics. In all cases, donor dummies are included in the regression matrices so that the other variables included in the regressions do not pick up the correlated effect of donor programs but are not shown in the regression output below. Finally, the dependent variable is the log of the number of visits per clinical health care worker, to account for the skewness of the data. Key findings from these initial core regressions are as follows: • The smallest health facilities are –similarly to the vaccine models– the most productive. Relative to very large facilities with more than seven clinical health care workers, health facilities with one clinical health care worker produce 89% more visits per worker when evaluated at the median (raising the number of visits per health care worker from 3.45 to 6.55). The facilities with two clinical health care workers produce 17% more visits per worker when evaluated at the median (raising the number of visits per health care worker from 3.45 to 4.05). All in all, the smallest facilities with just one clinical health care worker are the most productive. • The SARA facility readiness measure was found to be highly statistically significant. Raising this measure by one standard deviation (14) would raise the number of visits per clinical health care worker by 18.2% when evaluated at the median (raising the number of visits per health care worker from 3.45 to 4.08). • The number of community health workers was found to be insignificant. • Charging fees separately for different items (consultation, medicines, procedures, etc.) reduces the number of visits per clinical health care worker from 3.45 to 2.67 (reduction of 23%) –– this was very significant finding. Hence, clinics that charge a flat fee appear to have more visits per health worker. (Note that virtually all the health facilities in the sample charge fees of some kind.) 19 Model V1A-2: Service Availability and Readiness Assessment (SARA) Variations We now try different variations of the SARA composite score variable. In all cases, the SARA composite variable or others related to it are included in the random intercepts. Results are described in Table 10 below. • Model VIA-2a: First, we separate into rural versus urban. We created a variable that distinguishes from rural and urban facilities based on their SARA scores. We find here that the SARA composite score variable is very significant for both rural and urban areas, but slightly more for urban areas. • Model VIA-2b: Next, we try the regression with the SARA variable split into its different components (basic amenities, basic equipment, medicines, diagnostics, and precautions). Unfortunately, none of these component measures are significant, but this could be due to multicollinearity between these different measures. • Model VIA-2c: Next, we try the regression with the SARA variable split into its different components (basic amenities, basic equipment, medicines, diagnostics, and precautions), and this time we also exclude facilities that offer inpatient services on a routine basis. Here we see results that may be more meaningful than in the previous regressions – we find that basic amenities have an impact that is statistically significant at the 10% level. But even more so, we find that availability of medicines is highly significant. Other SARA measures are not significant. 20 Model VIB-1: Community Health Worker Variations TABLE 10. EFFECT OF COMMUNITY HEALTH WORKERS Outcome: daily visits (log transformed) VIB-1a VIB-1b VIB-1c Number of clinical workers One 0.341*** (0.04) 0.569*** (0.043) 0.451*** (0.035) Two - 0.004 (0.037) 0.154*** (0.039) 0.072** (0.032) Three - 0.091** (0.038) 0.114** (0.04) 0.013 (0.041) Four - 0.389*** (0.042) - 0.327*** (0.044) - 0.274*** (0.034) Five - 0.408*** (0.048) - 0.402*** (0.05) - 0.326*** (0.043) Six - 0.407*** (0.042) - 0.384*** (0.042) - 0.297*** (0.038) Seven - 0.25*** (0.052) - 0.167** (0.062) - 0.316*** (0.041) Proportion of community health workers vs all workers 0.24 (0.309) - - - - Proportion of community health workers vs clinical workers Rural areas - - 0.24 (0.318) - - Urban areas - - 0.898** (0.426) - - Service Availability Readiness Score Overall 0.007 (0.005) 0.01* (0.005) 0.018*** (0.001) Services - 0.13 (0.461) - 0.362 (0.4888) - 0.24*** (0.053) Charges fees separately - 0.391** (0.166) - 0.442** (0.14) - 0.531*** (0.024) Table 10 continues in the next page. 21 TABLE 10. EFFECT OF COMMUNITY HEALTH WORKERS (CONT) Proportion of CHWs when there is/are One clinical worker - - - - 0.032*** (0.003) Two clinical workers - - - - 0.027*** (0.006) Three clinical workers - - - - - 0.024 (0.019) Four to six clinical workers - - - - - 0.01 (0.014) Seven to nine clinical workers - - - - - 0.006 (0.015) Ten or more clinical workers - - - - 0.108** (0.04) Total observations Number of clusters (Average observations per cluster) Mixed vs OLS p-value AIC 10,529 20 (780) 0.000 25060 10,529 20 model (526) 0.000 24597 15,603 OLS model (R2: 0.17) 41311 As mentioned previously, community health workers are key in supporting primary care services in Haiti. The following models aim to understand their role in diverse contexts throughout the country. • Model VIB-1a. First, we include just ASCPs and ASCs without supervisors (i.e., number of ASCPS and ASCs divided by the total of CHWs plus clinical health workers). We find that this variable, again, is not statistically significant. (See Table 10 above.) • Model VIB-1b: Next, we include just ASCPs and ASCs without supervisors, and split this variable into rural versus urban areas. We find that this time, the variable is statistically significant in urban areas, but not in rural areas. (See Table 10 above.) • Model VIB-1c: Next, we do something similar here as for Model VB-3 (for vaccines). As in the case of Regressions VB-3 (for vaccines), we use an Ordinary Least Squares Model and not a multilevel model to assure a parsimonious model. As for Model VB-3, we use interaction terms to allow for differential impacts of the CH_CLW variable, for different facility size: i. CH_CLW_1 = CH_CLW for 1-clinical-health worker facilities, and zero otherwise ii. CH_CLW_2 = CH_CLW for 2-clinical-health worker facilities, and zero otherwise iii. CH_CLW_3 = CH_CLW for 3-clinical-health worker facilities, and zero otherwise. 22 From these regressions, we find that having more ASCs and ASCPs does turn out to significantly affect (positively) the number of visits per health care worker, but this effect is clear only for smaller facilities – those with 1 or with 2 clinical health care workers. Model V1C: Variations with Fees Charged Variable Fees charged per facility varies widely across the health system and are key determinants of the productivity of health facilities. We tried different variables representing different fee modalities. Since different fee variables tend to be correlated, we introduced and tried out fee related variables using a stepwise approach. Key results are shown in Table 11 below. Model VIC-1 a-c: First, we tried introducing individual fee variables, in a stepwise manner – for consultations, medicines etc. We find that out of all these variables, the one only that affects visits to a statistically significant degree (and negatively) is fees for consultations. Fees for medicines may affect visits negatively, but unfortunately, we could not get the regression in this case to converge – but this is probably because only 2.31% of facilities give medicines for free. Most health facilities charge for medicines. 23 TABLE 11. EFFECT OF FEES CHARGED Outcome: daily visits (log transformed) VIC-1a VIC-1b VIC-1c Number of clinical workers One 0.862*** (0.074) 0.555*** (0.037) 0.518*** (0.035) Two 0.362*** (0.066) 0.092** (0.035) 0.005 (0.033) Three 0.121** (0.061) - 0.181*** (0.034) - 0.143*** (0.033) Four - 0.04 (0.057) - 0.223*** (0.035) - 0.261*** (0.034) Five - 0.062 (0.056) - 0.394*** (0.04) - 0.383*** (0.039) Six -0.004743 - 0.378*** (0.041) - 0.378*** (0.041) Seven - 0.339*** (0.058) - 0.64*** (0.047) - 0.648*** (0.049) Proportion of community health workers vs all workers - 0.371 (0.227) - 0.038 (0.195) - 0.063 (0.186) Service Availability Readiness Assessment Score Overall 0.012 * (0.006) 0.008** (0.003) 0.01** (0.004) Services 0.018 (0.402) - 0.022 (0.063) - 0.042 (0.061) Charges fees for consultations - 0.506** (0.205) - - - - Charges fees for tests - - 0.188 (0.111) - - Charges fees for registration - - - - - 0.085 (0.116) Total observations Number of clusters (Average observations per cluster) Mixed vs OLS p-value AIC 13,745 20 (687) 0.0000 13,745 20 (687) 0.000 34302 13,745 20 (687) 0.000 34092 Model V1C-2: Next, we examine what happens if someone comes into a health facility and says they cannot pay for a service. Just 10.6% of people would get exempted in that case, while 26.3% would be asked to pay the fee later. Fortunately, only 2.2% would be denied the service. In the case of facilities where such people are exempted from payment (based on self-reporting), this does not seem to affect the outcome (visits) variable. In the case of facilities where such people are asked to pay the fee later, visits are also not affected. However, in the case of facilities where the services are not provided for those who cannot pay (i.e., fee is mandatory at the time of service), this has a very statistically significant and negative impact on the number of visits per health worker. The results are shown in Table 12 below. 24 TABLE 12. EFFECT OF FEES PAID Outcome: daily visits (log transformed) VIC-2a VIC-2b VIC-2c Number of clinical workers One 0.477*** (0.334) 0.456*** (0.034) 0.469*** (0.032) Two - 0.008 (0.031) 0.001 (0.031) 0.012 (0.031) Three - 0.134*** (0.032) - 0.114*** (0.032) - 0.108*** (0.032) Four - 0.298*** (0.033) - 0.284*** (0.033) - 0.269*** (0.033) Five - 0.485*** (0.038) - 0.498*** (0.038) - 0.47*** (0.037) Six - 0.264*** (0.037) - 0.304*** (0.038) - 0.307*** (0.037) Seven - 0.409*** (0.041) -0.042 - 0.439*** (0.04) Proportion of community health workers vs all workers 0.022 (0.182) - 0.037 (0.174) - 0.027 (0.182) Service availability readiness score 0.012*** (0.003) 0.012*** (0.003) 0.012*** (0.003) Services score - 0.053 (0.057) 0.03 (0.057) 0.046 (0.055) Fees exempted 0.113 (0.15) - - - - Fees paid later - - 0.054 (0.06) - - Fees mandatory - - - - - 0.752** (0.243) Total observations Number of clusters (Average observations per cluster) Mixed vs OLS p-value AIC 15,547 20 (777) 0.000 39131 15,547 20 (777) 0.000 39273 15,547 20 (777) 0.000 39210 25 VII- Key Conclusions Key conclusions from the preceding results are as listed below. Note that these results are for lower-level health facilities and not hospitals (Dispensaries, Health Centers Without Beds and Health Centers with Beds). Furthermore, the results apply to public, non-profit private and “mixed” health facilities. They do not apply to for-profit private health facilities. • There appears to be a tendency among some donors to prefer to support larger health facilities since these are thought to have higher volume and hence to provide a bigger “bang for the buck”. But in fact, the analysis in this paper shows that the smaller health facilities – where size is measured by the number of clinical health care workers – are the most efficient, with efficiency being measured by total vaccine provision or total number of visits by patients, per clinical health worker. (This efficiency measure is used since it is a rough proxy for output per unit cost, since personnel costs account for the largest part of the total cost of health care provision.) Specifically: ➢ The most productive are facilities with just one clinical health care worker, where vaccine provision per clinical health care worker is 2.36 times that of the largest health care facilities (i.e., 136% higher). Facilities with two clinical health care workers have vaccine provision that is 43% higher than that of the largest health facilities. (All evaluated at the mean for the outcome variable.) ➢ The number of visits by patients per clinical health care worker is 89% higher, and 17% higher, for facilities with one and two clinical health care workers respectively, as compared to the largest health facilities. (Evaluated at the mean for the outcome variable.) • Having a strategy for communication of vaccines appears to have a positive impact on health worker productivity in terms of vaccine provision. • Overall, having more Community Health Workers (CHWs) has a strong positive im