Programme de renforcement du secteur de la justice en Haïti : Plan de suivi, d'évaluation et d'apprentissage

DISCLAIM ER The aut hors’ views expressed in t his publicati on do not necess arily reflec t the views of the United States Agency for Internati onal Dev elopment or the United Sta tes governmen t. HAITI JUSTIC HAITI JUSTICE SECTOR STRENGTHENING PROGRAM MONITORING, EVALUATION, AND LEARNING PLAN DISCLAIM ER The aut hors’ views expressed in t his publicati on do not necess arily reflec t the views of the United States Agency for Internati onal Dev elopment or the United Sta tes governmen t. DISCLAIM ER The aut hors’ views expressed in t his publicati on do not necess arily reflec t the views of the United States Agency for Internati onal Dev elopment or the United Sta tes governmen t. HAITI JUSTICE SECTOR STRENGTHENING PROGRAM (JSSP) MONITORING, EVALUATION, AND LEARNING PLAN FEBRUARY 7, 2021 - FEBRUARY 6, 2022 Contract No. AID-OAA-I-I3-00032, Task Order No. AID-521-TO-16-00005 Date of Original AMEP : October 31, 2016 Date of Last Revision : August 14, 2020 Project Name : Justice Sector Strengthening Program Project Address : 11G, Route Montagne Noire, Haiti CONTENTS I. Project Description II. Project Logic III. Monitoring System Description A. MEL Staff Structure B. MEL Approach C. Data Collection and Analysis D. Data Management E. Reporting F. Baseline Data Collection Plan IV. Data Quality Assessment Plan and Procedures V. Evaluation VI. Learning Plan Annex I: Performance Indicator Tracking Table Annex II: Performance Indicator Reference Sheets (PIRS) Annex III: Learning Plan August 14, 2020 This publication was produced for review by the United States Agency for International Development. It was prepared by Chemonics International Inc. 5 ACRONYMS CMIS Case Management Information System COP Chief of Party COR Contract Officer’s Representative CSPJ Conseil Superieur du Pouvoir Judiciaire – High Judicial Council DCOP Deputy Chief of Party DQA Data Quality Assessment GOH Government of Haiti JIU Judicial Inspection Unit JSSP Justice Sector Strengthening Program M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning PIRS Performance Indicator Reference Sheets PITT Performance Indicator Tracking Table PMU Project management unit TOCOR Task Order Contracting Officer´s Representative USG United States government 6 I. PROJECT DESCRIPTION USAID’s Justice Sector Strengthening Program’s (JSSP) goal is to support the Government of Haiti (GOH) and civil society in expanding access to quality justice for Haitian citizens. The project supports the professionalization, independence, and efficiency of the Haitian justice sector, advancing core justice system strengthening while building the foundations of judicial reform, fostering political support, and addressing relevant justice issues in the short to medium term. To accomplish this, JSSP has four overarching objectives: (1) improving the legal, policy, and regulatory framework, (2) strengthening the judiciary as an independent, credible, and effective authority, (3) improving access to justice and protection of rights, and (4) strengthening civil society constituencies for reform. II. PROJECT LOGIC JSSP is contributing to the achievement of the “Post-Earthquake USG Haiti Strategy Toward Renewal and Economic Opportunity,” under Pillar D, Governance and Rule of Law. Pillar D’s specific objectives include Intermediate Result 2, Rule of Law and Human Rights strengthened, with its sub-intermediate results 2.1 “Improve access to and delivery of justice services,” as well as 2.2 “Improve security and strengthen the protection of human rights.” The project contributes to the second objective of the 2015 U.S. government’s (USG) Quadrennial Diplomacy and Development Review by promoting the rule of law, enhancing access to justice, and defending human rights and the inclusion of marginalized and vulnerable groups. It also contributes to the 2016-2019 USG Integrated Country Strategy for Haiti by supporting the draft objective “Efforts to comprehensively modernize the justice system, the civil service, and local governance are increased, so that public institutions can better perform their legally mandated functions and promote civil freedoms and internationally recognized human rights.” The project’s development hypothesis is that if the enabling environment for justice is modernized and improved, justice sector institutions are strengthened, pathways to justice services for protection of human rights are expanded, and civil society groups can mobilize constituencies for reform, then Haitian citizens will be more able to access quality justice services. The project’s theory of change is that by engaging project counterparts in planning and having them identify the resources and commitments they will bring to the effort and providing phased assistance in stages based on counterpart demonstration of participation, project resources will be maximized, and interventions sustained. The project is targeting the following jurisdictions: Port-au-Prince, Croix-des-Bouquets, Saint- Marc, Cap-Haitien, and Fort-Liberte. The CMIS is a nationwide activity that JSSP expanded in FY2019 and FY2020 to 6 other jurisdictions: Grande-Riviere-du-Nord, Hinche, Mirebalais, Gonaives, Petit-Goave and Jacmel. In Y5, JSSP will expand the CMIS into two additional jurisdictions most probably in Aquin and Les Cayes, final decision will be taken by the national Committee on CMIS. The project’s logic is demonstrated through its results framework, Exhibit 1 below. The project goal is directly linked with the USG’s foreign assistance objective of governing 7 justly and democratically. The results framework demonstrates the logical, causal relationships between the different levels of anticipated results. 11 III. MONITORING SYSTEM DESCRIPTION Effective monitoring, evaluation, and learning (MEL) requires that project results and indicators be set early and tracked and analyzed consistently to determine whether goals are being met and, if not, to provide sufficient information to understand why and correct the course of action. The purpose of this MEL plan is therefore not to simply collect data, but to also accurately assess the impact of JSSP and its goal. The JSSP team will continue to use data collected to determine whether the project is meeting its performance requirements, inform decision-making, and improve management and implementation. The MEL plan describes JSSP´s methodology for monitoring and evaluation and will be used to plan and manage the collection of performance data and measure the success of the project in expanding Haitian citizens’ access to quality justice services. The performance indicator tracking table (PITT) includes performance indicators, targets, data sources and collection methods, frequency of data collection, and disaggregation. The MEL plan results will be used to regularly update and collaborate with relevant stakeholders, including USAID, to analyze change and learn from our data and adapt future activities to ensure achievement of results. On August 7, 2020, USAID granted a twelve-month cost extension to JSSP, extending the new project end date to February 6, 2022. Considering this extension, JSSP added Targets for the Extension Year (February 2021 to February 2022) in Annex 1 and 2. LOP Targets were also modified accordingly. 12 A. MEL Staff Structure Deputy Chief of Party. Our Deputy Chief of Party (DCOP), M r . Jacques Miguel Sanon, has the responsibility for the implementation of the MEL plan. He will continue to supervise our project MEL team, review and approve all reports that include data and findings, as well as leads the staff in interpreting data for activity management and learning. Monitoring, evaluation, and learning team. Our project M&E Advisor, Ms. Christelle Safi, will continue to have primary responsibility for overseeing the identification, collection, and analysis of data, tools, and methods. She will lead the technical staff, grantees, and implementing partners in developing the necessary tools and systems and provide training for quality data collection. She will have primary responsibility for the quality of data entered into the JSSP’s data management system, such as checking documentation and data records and performing a data quality assessment (DQA) annually in coordination with home-office staff. Working closely with JSSP technical staff and the DCOP, she will review the MEL plan annually and make necessary changes to the existing indicators and targets with USAID approval. Technical staff members. Data collection is a team effort and will continue to be built into the responsibilities of technical staff. Technical team members – including Port-au-Prince team members and field coordinators – will c o nti n u e t o be responsible for front-line data collection within their respective roles and activities. Project partners . Our project partners, including subcontractors and grantees, will assist in collecting data within their spheres of intervention, such as conducting pre- and post-training testing from training sessions. Home office support . Chemonics’ home office MEL department will support the JSSP MEL staff with assistance as needed, included providing templates, tools, regular training via webinars for all Chemonics MEL specialists, and updates on emerging methods and tools. B. MEL Approach Collaboration : Performance management is most effective when it involves the entire program team and relevant stakeholders. Technical staff will continue to be involved in the finalization of indicators, data collection, analysis, and learning. It is also important to obtain stakeholders’ buy-in into the anticipated program results and critical indicators, and include them as partners in collecting, analyzing, disseminating, and employing information about program results. Capacity Building: M&E is a key management skill for JSSP’s partners. By being involved in M&E, technical team members can transfer M&E skills to our government counterparts and grantees. JSSP will continue to provide technical assistance to grantees and counterparts to strengthen their M&E capabilities by helping them follow up on JSSP M&E requirements, and by strengthening their data collection capacities to measure their efforts, analyze data, and learn from the process. 13 C. Data Collection and Analysis Indicators. Our indicators include 23 at the objective and Intermediate result’s level that measure activity interventions, including 4 U.S. State Department Foreign Assistance Framework (F) indicators, and 1 at the project objective level that measures the higher results of those actions. This two-tiered approach ensures that both activity process and activity design are measured to assess the validity of the activity’s development hypothesis and our compliance with the design in implementation of the design. We have included USAID/Haiti required indicators from the Mission’s performance monitoring plan, F indicators, and our own custom indicators that reflect our interventions and their results. We are using a mix of both quantitative and qualitative indicators; we know that quantitative data can tell us what has changed, but qualitative data is needed to help us understand why changes have occurred and thus better tailor our interventions. Disaggregation of data : All person-level indicators are disaggregated by sex. Other disaggregation includes geographic location, job position, type of infraction, outcome of assistance etc. as appropriate to specific indicators. The disaggregated data provides us with detailed information that allows us to examine the effectiveness of activity interventions and better tailor our plans to ensure equity in activity outcomes across sex, locale and ethnic divides. Baselines and Targets. Baselines are established using these methodologies: structured interviews, surveys, open databases, court registries, grantee’s records, JSSP supported entities’ records, national and regional focus groups. Targets are finalized in collaboration with USAID upon completion of baseline data collection and approval of annual work plan. We will work with USAID to agree on aggressive but realistic annual and life-of-activity targets for all indicators; proposed targets are in Annex I, the PITT. We will continue to review the targets annually to determine if they are realistic, and if not, propose adjustments to them during our annual MEL plan review at work planning in consultation with USAID. Baseline data collection is described in further detail in section F. below. Data sources and collection methods. To the extent possible, we will collaborate to use data from reliable, trusted secondary sources such as other donors, counterparts, and proven GOH data to maximize cost efficiencies. Where possible, we will collaborate with other USAID projects to pool resources for data collection to maximize cost effectiveness. In many instances, however, the activity will collect its own primary data. We plan to use a variety of collection methods such as surveys; interviews and focus groups; and document review of government records; activity records and documents (such as training sign-in sheets, reports, and subcontractor/grantee records). If there is a particularly successful or unsuccessful intervention, we can do an in-depth case study to learn more about the circumstances and outcomes. The MEL advisor will oversee the design and wording of all survey, focus group, and interview questions, and will pretest and adjust all data collection tools before their use. 14 D. Data Management Chemonics used DevResults software for data storage and management, which is designed to accommodate users with low bandwidth and poor connections to the internet. Its web-based interface can be accessed from any internet-enabled device (tablet, mobile phone, desktop). Where internet access is unavailable or inconsistent, data can be captured via formatted Excel spreadsheets and other offline tools and later uploaded. Features include data visualizations, results dashboards, and document and photo storage capabilities. Differing levels of access to data and results can be provided to different users. E. Reporting JSSP will provide USAID with data updates in quarterly and annual reports. These reports will present a table of indicator values and a narrative on notable progress toward each expected result. Annual and quarterly reports will illustrate progress toward targets, challenges, strategies for overcoming challenges, and key successes. Quantitative data will be supplemented with narratives to help the reader understand the numbers’ context. Success stories and other communication materials will blend data with personalized stories to personify and illustrate the program’s activities, outcomes, and overall impact. The final report will contain an analysis of project results, a discussion of best practices and lessons learned, and presentation of success stories as reported over the life of the project. Project data will be used not only for reporting but also to inform project information, education, communication and outreach, such as publications and media campaigns. JSSP will share data regularly on annual basis (in accordance with U.S. government fiscal intervals) or, depending on the data type, on an ad hoc basis. Data analytic results and conclusions will be included in periodic reports to USAID. Depending on the nature of data, it will be shared in standard formats (e.g., reports, maps, tables, graphs, and narrative). ADS 579 Compliance JSSP anticipates the development of datasets that contribute both to baselines and to ongoing indicators. Within 30 days of reporting data, the raw data sets will be presented to USAID in a nonproprietary format and stripped of personally identifiable information in accordance with USAID's ADS 579. F. Baseline Data Collection In FY2017 JSSP completed key program baseline data collection such as the number of days for case processing, the number of GBV, TIP or other vulnerable group cases adjudicated in target jurisdictions, the number of disputes resolved in remote jurisdictions, the percentage of citizens who report greater trust in justice sector institutions and the average CSO scores in the advocacy index. Methodology to collect the baseline is described in detail in the PIRS. IV. DATA QUALITY ASSESSMENT (DQA) PLAN AND PROCEDURES 15 Since USAID requires a DQA once every three years, the home office project management unit (PMU) director conducted in FY2018, the first internal data quality control assessments to ensure that data collected are reliable, precise, and valid. In FY2020 Ms. Safi will continue to conduct the DQA on indicators where she does not have front-line data collection responsibility, and the home office will assist with the DQA for indicators where Ms. Safi has been the person collecting the data. The home office assistance will come from the C h e m o n i c s H o m e O f f i c e M E L D e p a r t m e n t a n d / o r t h e PMU director. In consultation with the technical team and the DCOP, the home office will review the validity, reliability, precision, timeliness, and integrity of JSSP data, identify any data limitations, unexpected trends, outliers, look for errors such as double-counting, and recommend solutions. Ms. Safi, the DCOP, and the technical team will continue revisiting this MEL plan annually to review its appropriateness and adapt indicators and targets as needed. The MEL plan is a living document and necessary modifications can be made at any time. All changes will be well documented and recorded in JSSP MEL files. V. EVALUATION Internal Performance Evaluations: Because of the rapidly changing environment in Haiti and the uncertainty of code reform passage, fraught with uncontrolled and uncontrollable variables, it may be difficult to draw a linear connection between project activities and results. For example, as justice sector institutions begin to learn their roles under the new codes, service delivery may initially be less efficient or effective as entities figure out how to modify or create practices to implement these codes. With support from home office Ms. Safi will lead the technical team in use of low-cost, effective, and scientifically validated methods to provide supplemental data for JSSP. Cooperation with USAID external evaluators. JSSP will continue t o coordinate closely with any third-party external evaluators engaged by USAID. JSSP will help them arrange site visits as appropriate and participate in synchronization meetings. JSSP’s internal data management system will facilitate easy data sharing, cost data, and performance monitoring information sharing with external evaluation groups. VI. LEARNING PLAN At work planning, the JSSP team created its own individual learning plan to ensure that data is used for adaptive management. The Learning Plan is described in further detail in Annex III below The Performance Indicator Tracking Table presented below identifies the USAID F Indicators and custom project indicators, units of measure, percentage formula, disaggregation, data sources, baselines, Actual figures achieved in previous years, Targets from FY2017 to February 2022 and Life of Project (LOP); and Performance Indicator Reference Sheets (PIRS) for each of the project’s F and custom indicators. 16 Annex I: Performance Indicator Tracking Table Ind icato r ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target Project Objective: Access to quality justice for Haitian citi zens improved A. 1 Percent of justice system users who report that they have increased access to quality justice as a result of USG support outco me % Numerat or: Number of users who report increase d access to quality justice Denomin ator: Number of citizens who had service delivery provided with USG support Location and sex Justice system users Survey Annua l N/A N/A N/A 40% 68% 50% 52.3% 55% 55% 55% 55% Objective 1: Legal, policy, and regulatory framework improved 1 Number of new penal code that outpu t Code Code Legislat ive records Docum ent review Quart erly 0 2 0 2 0 2 0 1 1 0 1 17 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target is adopted (Contract) IR 1.1: Through multi - stakeholder advocacy, new and improved criminal and human rights laws and policies passed 2 Number of USG- supported public sessions held regarding changes to the country’s legal framewor k (Contract) outpu t Public session Location Project records Docum ent review Quart erly 0 3 13 2 4 2 2 1 0 3 11 3 Number of civil society advocacy initiatives conducte d to support the code reform process (Contract) outpu t Initiative Location and type of initiative Project records Docum ent review Quart erly 0 3 29 2 3 1 12 1 0 2 9 IR 1.2: Implementa tion of new and existing criminal codes improved in target jurisdictions D Number outpu Individu Sex Sign-in Docum Annua 0 200 290 300 491(385 300 629(47 300 100 250 1,450 18 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target R. 1. 3- 1 of judicial personnel trained with USG assistance t al sheets ent review l (238M, 52F) M, 106F) 9M, 150F) 4 Percent of higher education entities that have integrate d trainings on the revised criminal code into their formal core curriculu m in a sustainabl e manner (Contract) outco me % Numerat or: Number of higher educatio n entities that have integrate d trainings on the revised criminal code into their formal core curriculu m in a sustainab le manner Denomin ator: Total number of higher Location and entity Entity leaders hip Review of curricu lum and Intervi ew Annua l 0 N/A N/A 10% N/A 30% N/A 75% 75% 30% 75% 19 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target educatio n entities Objective 2: The independence, credibility, and effective author i ty o f the judiciary strengthened 5 Percent decrease in case processin g time as a result of USG assistance outco me % Numerat or: Baseline value minus Year actual Denomin ator: Year actual Location Court records Docum ent review Annua l PAP : 265days CDB: 291days SM: 63 days CH: 298days FL: 12days 2% from basel ine PAP : - 161%CDB : -88%SM:- 222%CH:1 9%FL: - 31% 5% from baseline PAP : - 74%CDB : -46%SM:- 75%CH:1 3%FL : - 95% 10%from baseline PAP: N/ACD B: - 72%SM : N/A CH: - 44% FL: - 96% 10%from baseline 10% from baseline 10% from baseline 10% from basel ine IR 2.1: Cou rt case a dministration improved D R. 1. 5- 1 Number of USG- assisted courts with improved case managem ent systems outco me Court Location Project records Site visit and intervi ew Annua l 0 2 4 6(cumula tive) 5 8(cumula tive) 8 11(cumul ative) 11(cumul ative) 13(cumul ative) 13 6 Number of coordinati on mechanis ms at the communit outpu t Coordin ation mechani sm Type of mechani sm Project staff and commu nity person nel Docum ent review Quart erly 0 4 7 5 8 5 7 5 2 2 23 20 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target y level to help improve working relationsh ips, informati on sharing, and case administr ation between judges, prosecuto rs, lawyers, and police (Contract) IR 2.2: Hum an resources effectively developed and managed 7 Number of judicial and court personnel trained to perform their duties effectivel y and efficiently (Contract) outpu t Individu a l Sex Trainin g records and sign-in sheets Docum ent review Quart erly 0 150 270 (189M, 81W) 150 501(387 M, 114F) 150 672(49 4M, 178F) 250 50 250 1,000 21 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target 8 Percent of USG trainees who report that the training has improved their job skills outco me % Numerat or: Number of USG trainees who say that training improve d their job skills Denomin ator: Number of responde nts by jurisdicti on Location and sex USG trainee s Survey Annua l 0 60% Port-au- Prince : 100%Saint -Marc : 97%Cap- Haitien 86%Fort- Liberté 93% 60% 96%(Port- au-Prince ;6M, 2F) 60% 92%63 (56M, 7F) 75% 75% 75% 75% 9 Number of performa nce monitorin g and oversight of court personnel that are performe d according outpu t Inspecti on Location JIU records Docum ent review Annua l 6 8 9 6 11 5 1 3 2 2 26 22 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target to the law / internatio nal standards (Contract) IR 2 .3 : Accountability and oversi ght strengthened 10 Number of judges fully vetted by the CSPJ in accordanc e with procedur es outpu t Judge Location and sex CSPJ records Docum ent review Annua l 0 55 40 100 42 100 39 50 30 30 365 11 Number of oversight missions conducte d by the CPSJ JIU (Contract) outpu t Inspecti on Location and type of oversigh t CSPJ JIU records Docum ent review Annua l 5 4 13 6 9 6 1 4 2 2 24 12 Number of judicial sanctions reported by the CSPJ outpu t Sanction Infractio n and sanction type CSPJ records Docum ent review Annua l 12 10 6 8 8 9 5 9 2 5 43 Objec tive 3: Access to j ustice and protection of rights impr ov ed 23 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target D R. 6. 3- 1 Number of individual s from low income or marginaliz ed communit ies who received legal aid or victim’s assistance with USG support outpu t Individu a l Location and sex Court, Grante e and project support ed entities records Docum ent review Annua l 0 1,200 1,204 (796M, 408F) 1,500 5,180(3,2 69M, 1,911F) 750 2,498 (1,802 M, 696F) 650 0 500 4,100 IR 3.1: Informal and formal legal services to key vulnerable populations increased 13 Percentag e change in the number of GBV, TIP, or other vulnerabl e group cases adjudicat ed in the target jurisdictio ns outco me % Numerat or: Year Actual minus Previous year Denomin ator: Previous year Type of vulnerab ility and location Court records Docum ent review Semi- Annua l PAP : 6 cases CDB: 0 FL: 3 CH: 3 SM : 1 +10% PAP : +116% CDB :N/A FL: -100% CH: -67% SM: +300% +15% PAP : +231% CDB : +600% FL: +833% CH: +625% SM: +240% +20% PAP: N/A CDB: 11case s/ -21% FL: 13 cases/ -53.5% CH: 26 cases/ -10.3% SM: N/A +30% +30% +30% +30% 24 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target (Contract) 14 Number of people in pretrial or illegal detention who are tried, released, convicted, acquitted, or whose cases are otherwise advanced procedura lly with USG assistance (Contract) outpu t Detaine e Sex and outcome of assistanc e (Trial, release, convictio n, acquittal or advance d procedu rally) Court, police, prison and project support ed entities records Docum ent review Quart erly 0 800 388 (363M, 25F) 800 2,352 (1,937M, 415F) 400 1,305 1,146 M, 159F) 400 0 250 2,400 IR 3 .2Adjudication of disputes at the sub - district level increased 15 Number of people visiting USG supported informal or formal justice dispute resolution entities in outpu t Individu a l Location and sex Dispute resoluti on entities records Docum ent review Quart erly 0 N/A 0 200 167 (83M, 84F) 150 471 (384M, 87F) 100 0 50 500 25 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target remote and marginaliz ed communit ies 16 Percent increase in the number of disputes adjudicat ed in targeted remote jurisdictio ns (Contract) outco me % Numerat or: Year Actual minus Baseline value Denomin ator: Baseline value Location and type of dispute Dispute resoluti on entities , court and mecha nism s records Docum ent review Annua l Cap- Haitien: 3 dispute s resolve d Fort- Liberte: 36 dispute s resolve d +15% 0% +15% N/A +15% PAP: 330% SM: 25% CH: 133% FL: 914% +15% 0% +15% +15% Objective 4: Civil society constituencies for reform strengthened D R. 4. 4- 2 Number of civil society organizati ons (CSOs) receiving USG assistance engaged in advocacy outpu t CSO Type of interven tion and location Project records Docum ent review Annua l 0 2 4 2(cumula tive) 3 2(jurisdic tive 3 2 0 3 3 26 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target interventi ons IR 4.1 P ubli c awareness of rights and judicial procedures increased 17 Number of USG- assisted campaign s and programs to enhance public understan ding, NGO support and media coverage of judicial independ ence and accounta bility (Contract) outpu t Campaig n and program Location and type of jurisdicti on Project records Docum ent review Quart erly 0 2 2 2 7 2 3 2 1 2 11 18 Change in level of confidenc e in the justice system (Contract) outco me % Numerat or: Number of citizens who have trust in Location and sex Genera l public Survey Year 3 and Year 4 Average : 75% ; Port-au- Prince 85.71%. Croix- des- Bouque N/A N/A N/A N/A 30% Averag e : 66% ; Port- au- Prince 52.9%. Saint- Marc 30% N/A 30% 30% 27 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target justice sector institutio n s Denomin ator: Number of responde nts by
jurisdicti on ts 100%. Saint- Marc 73.68%. Cap- Haitien 69.23%. Fort- Liberte 25%. 75%. Cap- Haitien 60%. Fort- Liberté 75% IR 4 .2 Civil s ociety networks, monito ring, an d advocac y capacity in creased 19 Percentag e of USG- supported CSOs that show improved scores in the advocacy index outco me % Numerat or: Number of USG- supporte d CSOs that show improved scores in the advocacy index Denomin ator: Number of USG- supporte d CSOs Location USG support ed CSOs Evaluat ion Annua l RENED H 3.1; AFASDA 1.8; MOTDE FAM 0.9; AFATN 0.7; RENDH 3.5; RFJS 1.8; RECID P 2.9; CPD 2.5; Fondati on 40% N/A +50% 84% RERENED H 3.2; AFASDA 4.6; RENDH 2.9; RFJS 4.5; RECIDP 3.6; CPD 3.4; Fondatio n Serovie 4.6 ; JILAP 2.8 ; GREAS 4 ; ACJU 3.1 ; CHCV 3.1 ; FEFBA +50% 79% RENED H 4.2; AFASD A 4.9; RENDH 3.8; RFJS 4.9; RECIDP 5.5; CPD 4.42; Fondati on Serovie 4.5; JILAP 2.9; +50% +50% +50% +50% 28 Indicator ID Indicator (Source) Indicator Type Unit of measure Percentage formula Disaggregation Data Source Data Collection Method Frequency Baseline Target FY2017 (Oct 2016-Sept 2017) Actual FY201 7 Target FY201 8 (Oct 2017-Sept 2018) ActualFY2018 Target FY2019 (Oct 2018-Sept 2019) Actual FY2019 Target FY2020 (Oct 2019-Sept 2020) Target FY2021 (Oct 2020-Feb 2021) Target for Extension Year (Feb 2021-Feb 2022) LOP Target Serovie 5.1 ; JILAP 1.2 ; GREAS 3.6 ; ACJU 1.3 ; CHCV 2.9 ; MOUFH ED 4.9 ; FEFBA 4.5 ; CASODI 2.7 4.5 ; CASODI 4.9 GREAS 3.8; ACJU 3.4; CHCV 1.9 ; FEFBA 4.5 ; CASOD I 5.1 ; OFC 3.8 Annex II: Performance Indicator Reference Sheets (PIRS) USAID Performance Indicator Reference Sheet Indicator ID: A.1 Name of Indicator: Per c ent of justi c e sy s tem users who report t hat they h ave increa sed acc e ss t o quality ju sti ce as a result of USG support Type of Indicator: Outcome Name of Result Measured (DO, IR, sub-IR, Project Purpose, Project Outcome, Project Output, etc.): Project objective DESCR I PTIO N Precise Definition(s ): This indicator measures the percentage of people who report that, as a result of USG-support, they have increased access to and trust in quality justice. Those people should have benefited from at least one USG supported service delivery such as: legal representation at courts from the project’s Legal Aid lawyers, mediation service from the project’s dispute resolution entities or other mechanism, or people who benefitted from improved procedures introduced by the new penal procedure code (such as plea bargaining and electronic bracelet). Quality justice is defined as a combination of access to justice (cost/physical access) and quality of the process and outcome of decision (including trust/confidence in the system). Numerator: Number of users who report increased access to quality justice Denominator: Number of citizens who had service delivery provided with USG support Unit of Measure: Percentage Disaggregated by: Location and sex Rationale for Indicat or (optio nal) : The goal of this indicator is to show how project activities such as support to criminal trials, support to the passage of law on legal aid, support to service delivery entities in target jurisdictions, support to any dispute resolution mechanism, support to legal assistance task force, support to the Judicial inspection unit, support to the code reform etc. have led to an increased access to quality justice for Haitian citizens. Through these activities, the project will increase physical access to justice, availability of justice, trust in the system, and quality of process and outcome of decision. PLAN FOR DA TA COLLEC TI ON Dat a Sou rce: Justice system users Method of Data Collection and Construction: Data will be collected by conducting an annual survey in Year 2 to Year 5 of the project. Each FY, JSSP will conduct a survey of beneficiaries in the target jurisdictions. JSSP will review the legal aid and the mediation forms completed by each lawyer/mediator. The project will also review court records to determine which citizens, after passage of the penal procedure code, potentially benefitted from improved procedures introduced by the new code. JSSP will develop the adequate methodology corresponding to the set of beneficiaries. JSSP will develop a questionnaire that will include questions related to (1) physical proximity to justice (2) cost (time, money) to access justice (3) trust/confidence in the formal justice system, and (4) quality of justice (process and outcome). We anticipate that, in any given year, a total of at least 1,000 people will benefit from USG-supported services. The project will select randomly 20% of beneficiaries for follow-up, with a goal of interviewing at least 10% (i.e., 50% participation), assuming they can be reached by phone directly on their personal phone or through a contact person. Those people selected should have benefitted, during the FY, from at least one USG supported service delivery such as: legal assistance and/or representation at courts from the project’s Legal Aid lawyers, mediation service from the project’s dispute resolution entities or other mechanism, or people who benefitted from improved procedures introduced by the new penal procedure code. Reporting Fr equency: Annual Individual(s) responsible at U SAID: COR PLAN FOR ANA L Y SIS, REVIEW A ND REPORTI NG Data analysis : The M&E advisor will gather the completed questionnaires and enter the related data in a database that will be developed for this indicator. August 14, 2020 This publication was produced for review by the United States Agency for International Development. It was prepared by Chemonics International Inc. 27 Pre sentati on of Data: Data will be presented in the Annual Report Review of Data: M&E advisor and project management will review data to confirm fulfillment of indicator, compared against annual targets R eporting of Dat a : Annual report TARGETS AND BASELINE Basel in e Timeframe: 10/2017 Rationale for Targets: Based on anticipated level of effort for the first year of project implementation, the target is set reasonably DATA QUALITY ISSUES Dates of P r evio us Data Quality A ssessments and Name of Reviewer(s): N/A Da te of Future D ata Qua lity Assessments (optional) : USAID requires DQA at least once every three years Known Data Limitations : Beneficiaries could be reluctant and/or very influenced in answering the questionnaire. Beneficiaries could not be reached by phone easily because the phone number on the form is no longer active or the phone battery is empty. In addition, the denominator includes respondents who could not determine whether they had an increased access to justice thus affecting the total result of the indicator. Actions to address Data Li mitations : The project will explain that the main purpose of the survey is to identify areas for improvement in justice service delivery. The project will add to the “ legal aid form ” the “ phone number of the beneficiary ” in addition to a phone number of a contact person to maximize availability of beneficiary. Despite applying this measure for the survey in FY19, it was still difficult to reach the beneficiaries by phone. Consequently, all beneficiaries who have a phone number were called. CHANGES TO INDICATOR Changes to indicator: Targets for FY20 and FY21 were decreased from 60% and 65% respectively to 55% for both fiscal years. Because of the ongoing socio/political/security crisis which began in 2018, courts were often partially operating or closed. Some related services such as legal assistance are also negatively impacted all together reducing access to justice for citizens. Target for Extension Year (February 2021 to February 2022) was added. PERFOR MANCE I NDICATO R VAL UE Year Ta r get Act ual Note 2017 N/A N/A This is a new indicator proposed by JSSP in its FY2018 MEL plan. Thus, there is no target for FY2017 and targets are set from FY2018 to FY2021. 2018 40% 68% In FY2018 JSSP conducted a perception survey of beneficiaries of legal assistance from the 5 Legal aid offices supported by JSSP (BAL). Sixty individuals (49 men and 11 women) were reached by phone and interviewed on access to justice (cost/physical access) and quality of the process (including trust in the formal justice system). 68% of the respondents reported that they have an increased access to and trust in quality justice. 28 2019 50% 52.3% In FY 2019 JSSP conducted a perception survey of beneficiaries of legal assistance from the 4 Legal aid offices supported by JSSP (BAL). Sixty-five individuals (45 men and 20 women) were reached by phone and interviewed on access to justice (cost/physical access) and quality of the process (including trust in the formal justice system). 52.3% of the respondents reported that they have an increased access to and trust in quality justice. 2020 55% 2021 55% 2022 55% LOP 55% Other Notes (opt ional) : N/A T HIS SHEET LAST UPDATE D ON : 08/14/2020 29 US AID Per fo rmance Indicator Reference Sheet Indicator ID : 1 Na m e of Indicator: Number of new penal code that is adopted Type of Indicator: Output Name of Result Meas ured (DO, IR, sub - IR, Project Pu rpose, P ro ject Outcome, Project Out p ut, etc.): IR 1 DE SC RIPT IO N Precise Definition(s): This indicator measures the number of penal code that is adopted by the Haitian Parliament as a result of USG-supported advocacy efforts. Unit of Measure: Number of codes Di saggre gated by: N/A Ration a le f or Indicator (o pt iona l) : This indicator relates to the success of USG advocacy activities in building consensus on the revised criminal code and facilitating their adoption at the Parliament. Activities may include but are not limited to: providing support to the Penal Reform Commission to complete the draft codes, supporting professional associations (PAPBA, APM, IAWJ, CHAIFEJ) to conduct analysis of the two draft codes, awarding grants to civil society organizations to inform and engage the public in the reform, hosting parliamentary public hearings, and holding working sessions etc. PLAN FOR DATA COLLECTION Data Source: Legislative records Method of Data Collection and Constructio n: Data will be collected by reviewing legislative records. Rep or t ing Frequency: Quarterly Indi vidual(s) responsible a t US AID: COR PLAN FOR ANALYSIS, REVIEW AND R EPORTING Data analysis: The M&E advisor will review Parliament records or the National Gazette ( Le Moniteur) to determine if the code is adopted Presenta ti o n of Data: Data will be presented in the Quarterly Report Revi e w of Data: M&E advisor and project leadership will review data to confirm fulfillment of indicator, compared against annual targets Reporting of Data: Quarterly report TARGETS AND BASELIN E Base line T imeframe: 10/2016 Rationale for Targets: The target is set based on the number of existing criminal code that needs to be adopted by the Parliament. DATA QUALITY ISSUES Dates o f Pr evious Data Qual ity Assessments an d Name of Reviewer(s): N/A Date of F uture Data Q uali ty As sessments (optional) : USAID requires DQA at least once every three years Known Data Limitatio ns: Whether the Parliament meets, considers and passes any bills, the time and pace it would take to vote on the bill, and when this will occur is out of project’s control Actions t o addre ss Dat a Limitatio ns: The project’s mitigation strategy will be to support advocacy and other activities to increase the likelihood of passage. CHANGES TO I NDICATOR Chang es to ind icator: Following contract modification in August 2019, the indicator changed from “ Number of new penal code and criminal procedure code that are adopted” to “Number of new penal code that is adopted”. Targets for FY20 and FY21 were reduced from 2 to 1. Target for Extension Year (February 2021 to February 2022) was added. PERFO RMANCE I NDICA T O R VALUE Year Target Ac tua l Note 30 2017 2 0 In FY2017 JSSP efforts led to significant progress regarding code passage. The codes are now being analyzed in Parliament with JSSP support. Parliamentarians are looking forward to passing them in FY2018. 2018 2 0 In FY2018, the penal code was officially inserted on the agenda of the Senate for passage. 2019 2 0 This fiscal year, the vote on the penal code was postponed in a few occasions due to the civil and political unrest. The MJPS consultant supported by JSSP continued to review the penal code and proposed modifications that will be sent to the Senate’s Justice, Security and Defense Commission along with the list of errata. 2020 1 2021 1 2022 0 LOP 1 Other Notes (opt iona l) : N/A THIS S HEE T LAST UPDATED ON: 08/14/2020 30 USAID Perfo r m ance Indicat or Reference Sheet In dicator ID : 2 Name of Indicator: Number of USG-supported public sessions held regarding changes to the country’s legal framework Type of indicat or: Output Name o f Res u l t Measured (DO, IR, sub - IR , Project Purpose, Pr o j ect Outcome, Project Output, etc.) : IR1.1 DESCRIPTION P recise Definition(s ): This indicator measures the number of public sessions involving parliamentary committees, private sector and professional associations, civil society organizations etc, aiming at analyzing the draft criminal code, draft criminal procedure code and other draft legislations such as Law on the profession of lawyers and providing their inputs. Public sessions refer to hearings, town hall meetings and debates held regarding the draft legislation. Unit of Measure: Number of public sessions Disaggregated by: Location Rationale f or Indicator (optional) : USG support for public sessions helps develop consensus-building and transparency. An increase in the number of public sessions regarding the draft legislation suggests that USG assistance is building an increasingly democratic and consensus-building culture. It also increases the likelihood that when the new legislation is adopted, implementation will occur because a variety of groups came together in the public sessions to openly discuss the draft legislation. P LAN F OR DATA COLLE CTION Data Source: Project records Method o f Da ta Collection an d Construction: Data will be collected by reviewing the project’s mission reports, attendance sheets and any other relevant document. Reporting Fr e que ncy: Quarterly Individual(s) responsibl e at USAID: COR PLAN FOR A N ALYS IS, R EVI EW AN D RE PORTING Data analysis: The M&E advisor will review project records Prese ntation of Data : Data will be presented in the Quarterly Report Revie w of Data: M&E advisor and project management will review data to confirm fulfillment of indicator, compared against annual targets Reporting of Da ta: Quarterly Report TARGETS AND BASE LINE Base line Timeframe : 10/2016 Ratio na l e for Targets: Based on anticipated level of effort for the first year of project implementation, the target is set reasonably DA TA QU AL IT Y ISSUES Dates of Previous Data Qua lity As sessments a nd Name of Reviewer( s): N/A D ate o f Future Data Quali ty Ass es s me nts (optional) : USAID requires DQA at least once every three years Known Data Limitatio ns: Whether the Parliament meets, considers and passes any bills, and when this will occur is out of project’s control Act ions to ad dress Data Lim itations : The project’s mitigation strategy will be to support advocacy and other activities to increase the likelihood of passage CHAN G ES T O IND ICA TOR Ch an ges to indicator: Targets for FY2020 and FY2021 were decreased because public sessions on code reform were conducted in FY2017, FY2018 and FY2019. The draft codes are completed as a result of these public sessions. Target for Extension Year (February 2021 to February 2022) was added and LOP target was modified accordingly. PERFORMANCE INDIC ATOR VALUE 30 Ye ar Target Actual No t e 31 2017 3 13 IN FY2017 JSSP provided logistical support to the Parliament’s Senate Justice and Security Commission, to conduct five public hearings on the draft penal code and draft penal procedure code. The project also supported the Penal Reform Commission in presenting t