National Consultant – Mid Term Evaluation (Promoting Inclusive Access to Justice in Yemen (PIAJY Project).

Overview of Position:

• Yemen is a low income country & the poorest in the Middle East & North Africa regions, with a population of approximately 30.8 million people. The human development index of Yemen in 2019 was 0.470, putting the country 179th out of 189 countries & territories. Yemen ranks 155th of 156 countries in the global gender gap index. Yemeni women remain significantly underrepresented in the public sphere, holding only 4.1% of decision making positions. About 80% of the population need humanitarian assistance. Over 4.3 million people have been displaced, more than 70% of them women & children. Yemen was behind in achieving the sustainable development goals even prior to the conflict. Experts estimate that human development has been set back 21 years & if the conflict persists through 2030, development will be set back nearly 4 decades. The political & military outlook remains uncertain. Yemen’s post Arab spring transition spiralled into a fullblown war in March 2015. Peacemaking efforts led by the Office of Special Envoy of Secretary General to Yemen (OSESGY) have yielded rather uneven & fluid results with geographical variances. In December 2018, the Internationally Recognized Government (IRG) & the De Facto Authority (DFA) signed the “Stockholm agreement” in Al Hodeida. Despite the launch of UN Mission to support the Hodeida Agreement (UNMHA), however, peace in the west coast area remains elusive to date. In August 2019, the secessionist Southern Transitional Council (STC) seized control of Aden, splintering IRG held territories. In 2020, fighting has engulfed Marib, as the Houthis & Saudi Arabia exchange drone & air strikes. In April 2022, President Hadi ceded power to the new Presidential Leadership Council (PLC). Yemen’s governance system has suffered from a fragile central sub national relationship. Patronage networks & a system of pervasive corruption prevented the development of strong state institutions. The politicization & decapacitation of Rule of Law (RoL) institutions is concerning. Impaired public services add a capacity challenge to the political manipulation of the formal institutions. Together with the diminished community protection capacity, the depleted institutional justice capacity has driven vulnerable populations into a greater risk of human rights abuse & violation. Women & juveniles are most vulnerable, suffering from intersecting marginalities. Female detainees risk in prison Gender Based Violence (GBV) & post prison stigmatization & social ostracization for life, including rejection by their own families. Political disputes over the appointment of the Attorney General (AG) in February 2021 have triggered a judicial strike in southern Yemen, particularly Aden. The strike lasted more than a year until the PLC appointed a new 2 AG on 25 May 2022, having caused a large backlog of civil & criminal cases in the formal justice system. The police continued to arrest & detain suspects, creating a heavy burden of pre trial detainees at police cells. Police detainees were further declined transfer to the central prison facilities, which also faced delays in releasing post term prisoners. Such factors have caused extreme overcrowding at the places of detention, with some locations holding 3 times more than their official capacity. Conditions at police cells are particularly inhumane, with detainees spending indefinite time in custody without due process. In parallel to the rapid decrease in formal justice supply, the conflict has caused a significant increase in community justice needs. Economic crisis & worsening living conditions have led to widespread family, civil & criminal cases & neighbourhood disputes over access to resources (land, water) & services (electricity, housing). The shrinking of the RoL institutions has widened the gap in the formal justice, resulting in the culture of impunity even for everyday crimes. To cope with the unsatisfied justice demands, communities are reverting to informal social structures, such as dispute resolution mechanisms mediated by customary & indigenous leaders. Tribal figures not only operate in parallel to the state but also hold key positions within state institutions, providing the role of an intermediary between the state & the citizens. Yemen has been tackling this issue of formal informal binary for decades, having established mechanisms for state society interface. The Department for Tribal Affairs in the Ministry of Interior (MoI) serves to draw sheikhs into the formal framework & to encourage a level of compliance. The Ministry of Justice (MoJ) accredited public notaries to work under the local court. The arbitration law regulates the relationship between formal & informal laws. So, justice in Yemen should be seen as a spectrum or a continuum, not clear cut binary. Actual application is context dependent, be it the state, religious, tribal, village or family authorities. Formal institutional processes, such as elections & appointments, define legal legitimacy in principle. In practice, however, local legitimacy prevails, which depends on the identification & solidarity between formal actors & local communities. The hybridity in justice governance has produced starkly different manifestations across locations. In Sana’a, the DFA holds a full control over security & justice issues. Simultaneously, they are defying the constitutional & statutory laws of Yemen & oppressing women’s rights, including the imposition of Mahram. Civic space for political freedom & human rights is almost non existent. In Aden, the IRG holds the formal legitimacy & willingness to cooperate with the donor community. The de jure government, however, remains fragmented & often paralyzed, having yet to reestablish security & justice capacity. Such capacity gap is ironically offering a greater space for civic participation & restorative justice. Simultaneously, the reemergence of community justice in Aden risks the reification of patriarchal social norms at the expense of gender & youth justice.

Project outputs:

• Output 1. Community safety enhanced through inclusive processes.
• Output 2. Increased awareness of rights & use of fair & effective formal & informal justice systems.
• Output 3. Gender justice capacity strengthened through gender inclusive institutions.
• Output 4. Protection of detainees strengthened & reintegration into community supported.

Project strategy:

In the absence of unified state authority, the project follows “people centred approach” & the “local turn” in governance programming, aimed at bridging the relationship between the community (as rights holders) & authorities (as the duty bearers). Intermediaries between the state & people are given greater attention, such as civil society, customary leaders & communities. Efforts are made to maintain the vertical linkage between subnational interventions & state level peace making. Partnership with humanitarian country team also aims to reinforce the horizontal linkage between thematic humanitarian protection (justice for children, justice for Women) & comprehensive developmental transformation (justice for all). The project builds upon the key policy concepts as below:

• Human Rights-Based Approach (HRBA), including UN Human Rights Due Diligence Policy (HRDDP) on United Nations support to non United Nations security forces.
• Gender Equality & Women’s Empowerment (GEWE), with additional inclusivity considerations, such as age, disability & displacement.
• People Centred Justice (PCJ), encompassing restorative justice.
• Humanitarian Development-Peace (HDP) nexus, following OECD – DAC recommendation.

Project locations:

• The project targets Aden & Sana’a in Yemen. Aden is the interim capital of the country & the seat of the IRG. Sana’a is the historical capital & the largest city currently under the control of the DFA. Resources & activities are equally distributed between Aden & Sana’a. Some activities, such as those on gender justice, however, were implemented only in Aden, given the political challenges in Sana’a.

Project implementation:

By the 1st quarter of 2023, US $ 3,289,013 was received, including USD 290,000 from our clients global rule of law program. The project conducted extensive consultations to specify target participants & select implementing partners:

• Responsiveness for Relief and Development Foundation (RRD).
• Afaq Shababia Foundation (ASF).
• Yemen Women Union (YWU).
• National Prisoner Foundation (NPF).
• Mysarah for Development.
• Public Works Project (PWP).

Key activities implemented so far include:

Community safety:

• Risk-awareness outreach on Improvised Explosive Devices (IEDs).
• Community police workshops to improve access to security services.
• IED first responder training.

Legal empowerment:

• Legal counselling for pre trial detainees.
• Community mediation committees to address pre trial cases.
• Referral to women & child protection services.

Gender justice:

• Expansion of a Gender Based Violence (GBV) shelter.
• Women protection training for justice professionals.
• Protection cash for detention cases involving women & children.

Detainee protection:

• Rehabilitation of prisons to improve living conditions.
• Vocational training for male & female detainees.
• Prison farming for female detainees.

Infrastructure rehabilitation:

• Police stations, prosecutor’s office, local courts & mediation centers selected as “model.”

At the beginning of 2023, the UN Country Team (UNCT) led by the Resident Coordinator’s Office (RCO) initiated the introduction & implementation of the UN Human Rights Due Diligence Policy on UN support to non UN security forces (HRDDP) in Yemen. Our client in Yemen participates in the HRDDP processes.

Role objectives

• This mid term evaluation aims to provide our client, the donor, government counterparts, civil society partners & other stakeholders with an impartial assessment of the results generated to date. The evaluation will assess the project’s relevance, effectiveness, efficiency, impact & sustainability / catalytic; identify & document evidence based findings & provide stakeholders with recommendations to inform the design & implementation of future interventions.

Specific objectives are to:

• Assess the relevance & strategic positioning of the project to promote inclusive access to justice & people-centred rule of law in Yemen.
• Track the progress made towards project results, including any unintended results & capture lessons learned for future interventions in Yemen.
• Appraise whether the project management arrangements, approaches & strategies, including monitoring strategies & risk management approaches, are well-conceived & efficient.
• Analyse the extent to which the project applies the rights based approach, gender equality & women’s empowerment, social & environmental standards & participation of other socially vulnerable groups.
• Collect evidence based findings & suggest strategic directions for future programming.
• Provide practical recommendations on project sustainability to inform any course corrections (if required / where relevant).

Scope of work:

The review will cover the project period from 1 September 2021 to 28 February 2023 covering all project locations (Aden & Sanaa) & all target groups. It will cover conceptualisation, design, implementation, monitoring, reporting & evaluation of results in consultation with all project stakeholders. It will evaluate all outputs, covering:

• Results framework & the project’s progress against it in terms of its relevance, coherence, effectiveness, efficiency, impact & sustainability.
• Risk log & the project’s 2 track approach to activities in Sanaa & Aden, including the implication of the HRDDP compliance & the possibility of changing target locations.
• Monitoring & evaluation arrangements & the project’s implementation, including the use of 3rd party monitoring.
• Measures to mainstream gender equality & women’s empowerment.
• Partnership at different levels, including with communities, civil society, authorities, UN agencies & donors.
• Project progress reports, including the inception report.

Review Questions:

• In reference to Organization for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) evaluation criteria, the evaluation seeks to answer the following questions, focusing on relevance, coherence, effectiveness, efficiency, impact, & sustainability.:

Relevance:

• Was the project appropriate & strategic to the main goals of inclusive access to justice & people-centred rule of law? Did relevance continue throughout implementation?
• To what extent was the project in line with the national development priorities, the country program’s outputs & outcomes & the SDGs?
• Was the project relevant to the needs & priorities of the target groups / beneficiaries? Were they consulted during design & implementation of the project?
• Did the project’s theory of change clearly articulate assumptions about why the project approach is expected to produce the desired change? Was the theory of change grounded in evidence?

Coherence:

• To what extent did the project complement interventions by different entities, especially other UN actors?
• How were stakeholders involved in the project’s design & implementation?

Effectiveness:

• To what extent did the project achieve its intended objectives & contribute to the project’s strategic vision?
• To what extent did the project substantively mainstream gender equality & women’s empowerment?
• What factors have contributed to achieving or not achieving intended project outputs & outcomes?
• To what extent has the project succeeded in fulfilling female & male beneficiaries’ practical & strategic needs for inclusive access to justice & legal empowerment?

Efficiency:

• To what extent was the project management structure as outlined in the project document efficient in generating the expected results?
• To what extent have the project implementation strategy & execution been efficient & cost effective?
• To what extent has there been an economical use of financial & human resources? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes?
• To what extent have the M&E systems utilized by the project enabled effective & efficient project management?

Impact:

• What are the early indications of inclusive access to justice & people centred rule of law?

Sustainability:

• Are there any social or political risks that may jeopardize sustainability of project outputs & the project’s contributions to country program outputs & outcomes?
• To what extent will financial & economic resources be available to sustain the benefits achieved by the project?
• To what extent are lessons learned being documented by the project team on a continual basis & shared with appropriate parties who could learn from the project?

Cross cutting themes:

Environmental, climate change & social safeguard:

• How can the project reconsider its approach to contribute to enhancing diversity & inclusion?

Gender equality:

• Is the gender marker assigned to this project representative of reality?
• To what extent has the project promoted positive changes in gender equality, participation & the empowerment of women? Were there any unintended effects?

Disability:

• Were persons with disabilities consulted & meaningfully involved in program planning & implementation?
• How RP ensured that persons with disabilities are included in project activities? To what extent activities designed to engage such persons?

Methodology:

The evaluation team (international & national consultants) will develop a methodology, including desk review, stakeholder engagement (virtual or in person by the national consultant), data collection & analysis. The evaluation will be carried out in accordance with our clients evaluation guidelines & policies, United Nations group evaluation norms & ethical standards; OECD / DAC evaluation principles & guidelines & DAC evaluation quality standards. It is expected that the evaluation will employ a combination of both qualitative & quantitative evaluation methods to capture the project results & generate evidence to substantiate all findings. Evidence obtained & used to assess the results should be triangulated. The evaluation team should propose their own methodology & detailed action plan as part of the application process, which may include:

• This would include project document (contribution agreement); theory of change & results framework; program & project quality assurance reports, annual workplans, consolidated mid year & annual reports, results oriented monitoring report, financial & funding reports, highlights of project board meetings & technical / financial monitoring reports.
• This would include an inclusive sample of project beneficiaries, key government counterparts, representatives of key civil society organizations, UN country team members & implementing partners.
• Development of evaluation questions tailored to the different needs & participation of various stakeholders.
• All interviews should be undertaken in full confidentiality & anonymity. Prior to engaging in interviews or focus group discussions, the evaluation team must obtain written informed consent from all stakeholders, but especially those from vulnerable categories. The final evaluation report should not assign specific comments to individuals but indicate patterns according to respondent categories.
• The evaluation team is expected to follow a participatory & inclusive consultative approach that ensures close engagement with the Evaluation Managers, implementing partners & male and female beneficiaries.
• This could include the sample size & characteristics; the sample selection criteria, the process for selecting the sample (e.g., random, purposive), if applicable, how comparison & treatment groups were assigned & the extent to which the sample is representative of the entire target population, gender representation, including discussion of the limitations of the sample for generalizing results.
• Other methods such as outcome mapping, observational visits, group discussions, etc.
• Data review & analysis of monitoring & other data sources & methods. The methodology will be further updated after the selection process. The methodology should be robust & innovative enough to ensure high quality, triangulation of data sources & verifiability of information. The final methodological approach including interview schedule, field visits & data to be used in the evaluation should be clearly outlined in the inception report & be fully discussed & agreed between our client, the donor & the Evaluators. The evaluation methodology needs to employ a gender sensitive approach & inclusion principle & this needs to be elaborated in the evaluation report, including how data collection & analysis methods integrated gender considerations, use of disaggregated data & outreach to diverse stakeholder groups. The findings of the evaluation should lead to the elaboration of specific, practical, achievable recommendations that should be directed to the intended users.

Ethics of evaluation:

• Evaluations in the UN are conducted in accordance with the principles outlined in the UNEG “ethical guidelines for evaluation.” The consultants are required of full compliance, including establishing protocols to safeguard confidentiality of information obtained during the evaluation. The Evaluator upon signing the contract will also sign this guideline which may be made available as an attachment to the evaluation report.

Contract deliverables:

• In line with our clients financial regulations, when determined by the country office & / or the consultants that a deliverable or service cannot be satisfactorily completed, that deliverable or service will not be paid. The consultants / evaluation team will be expected to deliver the following:
• The inception report should be carried out following & based on preliminary discussions with our client after the desk review & should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) & prior to the country visit in the case of international Evaluators.
• Immediately following an evaluation, our client may ask for a preliminary debriefing & findings.
• Our client & the donor will review the draft evaluation report & provide comments to the evaluator within 10 days, addressing the content required (as agreed in the inception report) & quality criteria as outlined in the evaluation guidelines.
• Comments & changes by the Evaluator in response to the draft report should be retained by the Evaluator to show how they have addressed comments.
• The final report should address comments, questions & clarification. The final report should also contain a stand alone executive summary of no more than five pages.
• Evaluation brief & other knowledge products the Evaluator is expected to prepare a 4 page knowledge product summarizing the findings & lessons learned to enhance the use of the evaluation results. The minimum content that needs to be included in the inception & evaluation reports is provided in the annex section. It is expected that the Evaluators will follow our clients evaluation guidelines & UNEG quality checklist & ensure all the quality criteria are met in the evaluation report.

Project reporting

Our clients country office in Yemen will be responsible for the contract & performance management of consultants & will designate an Evaluation Manager & an Evaluation Focal Point.

Key competencies

• The international consultant must have extensive experience in strategic programming of development assistance in active conflict setting countries within the broader areas of access to justice, rule of law & democratic governance. It is mandatory that the national consultant has substantial knowledge & experience of gender & monitoring & evaluation of similar initiatives in volatile environments.
• Minimum Master’s Degree in relevant disciplines (International Development, Legal Studies, Social Sciences, Gender Studies, or related fields).
• At least 5 years of experience working on issues related to access to justice, rule of law, democratic governance projects in conflict environments.
• At least 5 years of experience in designing & leading program evaluation in a conflict & fragile context.
• Proven knowledge & understanding of M&E methodologies, including qualitative & quantitative data analysis skills & participatory data collection approaches.
• Proven knowledge on results based management.
• Proven knowledge of the politico legal & social economic dynamics in Yemen
• Demonstrated experience in gender sensitive evaluation & analysis.
• Demonstrated understanding of issues related to gender & other cross cutting areas such gender equality, disability issues, rights based approach & capacity development.
• Native fluency in spoken & written Arabic as used in Yemen is mandatory.
• Fluency in spoken & written English with good report writing skills is recommended.
• Knowledge of programming principles & procedures, the UN evaluation framework, norms & standards, Human Rights Based Approach (HRBA).
• Demonstrates commitment to the UN values & ethical standards.
• Displays cultural, gender, religion, race, nationality & age sensitivity & adaptability.
• Treats all people fairly & with impartiality.
• Good communication, presentation & report writing skills, including proven ability to write concise, readable & analytical reports & high quality academic publications in English.
• Ability to work under pressure & meet deadlines.
• Flexible & responsive to changes & demands.
• Experience managing research & evaluation teams.
• Client oriented & open to feedback.

Further information

Institutional agreement:

• Our client will recruit two individual consultants an international & a national to work as a team. The international consultant will be responsible for overall evaluation delivery, including the development of evaluation methodology, the provision of policy & technical guidance & the quality assurance of inception & final reports.
• The national consultant will be responsible for fieldwork travels (as necessary) & data collection, such as access to stakeholders (esp. authorities), individual interviews, focus group meetings & site visits, in line with the leadership provided by the international consultant.
• The teamwork between 2 consultants should be collaborative & well orchestrated to combine consultation based local knowledge with comparison based global guidance, leading to the generation of local global synergies. Both are expected to contribute equally to all stages of evaluation, including evaluation methodology, analysis of key findings & formulation of recommendations.

Evaluation Manager, Focal Point, Reference Group & Commissioner:

• Our clients country office in Yemen will be responsible for the contract & performance management of consultants & will designate an Evaluation Manager & an Evaluation Focal Point.
• The consultant will report directly to the Evaluation Manager & Focal Point & work closely with the PIAJY project team. The Evaluation Manager & Focal Point will coordinate with the PIAJY project team to provide relevant documents for desk review & the list of available stakeholders & their contacts for fieldwork.
• With assistance from our client, a national consultant will take responsibility for conducting the meetings, subject to advanced approval of the methodology submitted in the inception report. The project team will refrain from participating in the meetings between the Evaluator & the evaluation participants.
• The Evaluation Manager will convene an evaluation reference group membered by Technical Experts from our clients side, donors & implementing partners. The reference group will review the inception report & the draft evaluation report to provide detailed comments related to the quality of methodology, evidence collected, analysis & reporting. The reference group will also advise on the conformity of processes toour client & UNEG standards. Detailed comments will be provided to the Lead Evaluator in an audit trail within the agreed timeframe. Comments & changes by the evaluators in response to the draft evaluation report should be retained by the Evaluators to show how they have addressed comments.
• The Evaluation Manager will develop a management response to the evaluation within 2 weeks of report finalization.
• The final report will be approved by the Evaluation Commissioner with the support of relevant stakeholders.

Logistics

• The international consultant will be home based & working remotely.
• The national consultant will be based in Yemen & will travel within Yemen according to the evaluation methodology agreed upon by the international consultant & approved by the Evaluation Manager.
• Both international & national consultants will work full time & be using their own ICT devices.
• Payment will be performance based & subject to our clients approval of deliverables, as detailed in the section on payment milestones.