COLLABORATIVE END EVALUATION CONSULTANCY

TERMS OF REFERENCE

COLLABORATIVE END EVALUATION CONSULTANCY

Make 12.4% Work Project (April 2018 – June 2022)

1. General information

1.1 About Light for the World International

Light for the World is a global disability & development organisation, enabling eye health ser­vices and empowering people with disabilities in some of the poorest regions of the world.

We want to break down unjust barriers to unlock the biggest potential! We aim at systems change, because we want our impact to last. We work with partners, because together we are stronger

1.2. About Light for the World in the Uganda

In Uganda, Light for the World started operating in the early 2000s. Our work in Uganda focuses on two thematic areas: Economic Empowerment, and Child Eye Health. Our innovative disability inclusion work focuses on young people with disabilities, their right to inclusion and ensuring improved socio-economic standards of living.

The Make 12.4% Work Initiative brings together key stakeholders to create enabling environments for inclusive economic growth, allowing people with disabilities to fully access the job market. This programme is led by trained young people with disabilities called Disability Inclusion Facilitators. Disability Inclusion Facilitators (DIFs) through a systematic capacity development programme, the Disability Inclusion Academy. The DIFs are trained to support inclusive skills development, inclusive governance, inclusive employment, and entrepreneurship.

Other projects and programs that support our work on economic empowerment are Mainstreaming Ambition: Vulnerable Youth in Business – the project aims at facilitating access to employment; in both the formal and informal sector for young women and men with disabilities. We Are Able! – designed to empower, amplify voices, and create resilience among people with disabilities (PWDs) and other excluded groups, particularly those faced with food insecurity in areas of protracted crises and Leave No One Behind: the project aims to contribute to humanitarian and long-term measures (in line with a humanitarian-development nexus approach) to combat COVID-19 and its impact on affected vulnerable people and target communities in South Sudan and Uganda.

We further run the Child Eye Health Programme (123 I Can See) – aims at increasing access to eye health, starting at a community level through strengthening and equipping health systems by raising awareness, detection, and referral of people with visual impairments.

2. Evaluation context

2.1 Project to be evaluated

Make 12.4% Work was co-created with a group of young persons with disabilities, and it has been the space where we have been learning together on how to remove barriers to waged, self-employment and livelihood programming for young women and men with disabilities. As part of the Make 12.4% Work program we came up with the Disability Inclusion Academy and the Disability Inclusion Facilitator approach, that come with training curricula, tools (game based materials, tools to measure change..), Competency Based Frameworks, a Community of Practice and a wide range of knowledge and behavioural change communication products. The approach, with its tools, materials and products have triggered much attention from Light for the World, from institutional donors, disability specific and mainstream organisations. New programs have been developed incorporating the approach, and Disability Inclusion Academies and the work with Disability Inclusion Facilitators have been pulled off in South Sudan, Ethiopia, Mozambique, Tanzania, Kenya, Burkina Faso, Bangladesh, and Nepal. The Initiative currently has a membership of over 140 companies and development organiztions.

Project title: Make 12.4% Work

Implementation date: April 1, 2018 – June 30, 2022

Location/Intervention zones: Country-wide with focus on Moroto, Gulu, Lira, Nwoya, Arua, and Kampala

Operational partners: National Union of Persons with Disabilities Uganda (NUDIPU), Reach A Hand Uganda (RAHU)

Target groups: Young women and men with disabilities

Project budget

Phase I – 325,610 GBP

Phase II – 578,912 GBP

Donor: National Lottery Community Fund (UK)

Project objectives

Providing a platform for steering, sharing learning and influencing government and other key stakeholders in their work towards inclusion in SDG 8: Decent Work and Economic Growth

Empowering young people with disabilities, building local expertise for disability inclusion and strengthening disability structures

Embedding disability inclusion at organisational/ program level of private sector and mainstream livelihood organisation

Challenging stereotypes and changing attitudes among families, communities and society at large

Expected results and indicators

1. Make 12.4% Work governance structure is effectively influencing the national policy environment, supporting the economic inclusion of people with disabilities in Uganda. Indicator: percentage of action points of meetings of Steering Committee and two task forces that are followed up.
2. Mainstream livelihood organizations, public and private sector organizations have become disability inclusive in their employment/recruitment and programming practices. Indicator: Number of livelihood organizations, companies, government ministries and programs that have become more disablity inclusive.
3. Youth with different types of impairments play a leading role in promoting and sustaining disability inclusive practices among public, private and development sector. Indicator: Number of Disability Inclusion Facilitators located in different regions of Uganda that have the capacity to build relationships, train and coach and monitor public, private and development sector organizations to include youth with disabilities in the workplace and livelihood programs.
4. Youth with disabilities have enhanced access to soft skills development, professional training and livelihood opportunities. Indicator: Number of persons with disabilities that access livelihood opportunities, job opportunities or professional training through Make 12.4% Work.

Main activities implemented

· Quarterly Steering Committee and Task Force meetings.

· Development and maintenance of Make 12.4% Work digital platforms (website and social media).

· Training and engagement of a team of Disability Inclusion Facilitators.

· Training and technical support to companies on inclusive employment.

· Training and technical support to mainstream development organizations on inclusive livelihood programming.

· Employability skills training and work experience placements for young people with dissabilities.

· Connecting young people with disabilities to employment and livelihood opportunuties.

2.2 Justification for the evaluation

Make 12.4% Work is a flagship program for Light for the World. Now that Make 12.4% Work comes to an end, and a range of new programs incorporating the DIF approach are in the start-up phase, we feel it is a perfect moment to document the impact of the approach, and its added value; even more important: we want to learn what should we stop doing or could be done more or different to further improve the approach and ensure sustainability, enable lasting system change and bring about an optimal positive impact on the lives of persons with disabilities and their families; and possibly: what should we stop doing?

The targeted users of the outcomes of this evaluation are management, specialists and practitioners at Light for the World at large, Light for the World implementing partners, program partners and stakeholders, other disability specific and mainstream organisations active in programs that target persons with disabilities, Organizations of Persons with disabilities and their umbrella organisations at different levels and potential funders of disability inclusive economic empowerment programs or programs that focus on strengthening the capacity of disability structures.

In summary, we look at a summative evaluation to systematically document Make 12.4% Work program products, results and impact. The evaluation is meant to provide evidence about the merit of the program, and triggering learning to support scale up of different elements of the approach.

3. Evaluation objectives (3 pages maximum)

3.1 Overall objectives and expectations for the evaluation

The overall objective of the end term evaluation is to systematically document the elements of the Make 12.4% Work approach and assess the contribution of the approach to positive changes in the lives of persons with disabilities and their families. We further look for evidence on what elements of the approach have worked particularly well, and what elements of the approach do/ did not work well, and what enabling environment/ context is best suited to aid replication of the approach ; with clear attention for optimizing impact, sustainability, and system change.

Light for the World, as the organization who commissions this evaluation, expects this evaluation to be collaborative in nature, where Light for the World and partners, especially organisations of persons with disabilities at all levels, are strongly engaged in the development of the data collection tools and methodologies. We further expect the team of Disability Inclusion Advisors, Disability Inclusion Facilitators, as well as our Learning & Innovation team to play a crucial role in the implementation of the evaluation.

3.2 Specific objectives

The evaluation aims to document and evaluate changes at the level of individual program participants, Make 12.4% Work partners, members and strategic partners as well as looking at changes in disability inclusive programming and eco system level changes – mindset and overall system capacity on disability inclusion and changes in policies and their implementation.

Evaluation objects

As for individual program participants, the evaluation aims to look at changes at the level of young women and men with disabilities reached through the different interventions of the program: program participants with disabilities who benefited from livelihood program interventions implemented by M12.4% W members, the Disability Inclusion Academy, Disability Awareness Trainings and other individuals who interacted with DIFs and young women and men who were in work experience placements and/ or were matched with job opportunities.

The objects for evaluating changes at organisational level will be members of Make 12.4% Work and Uganda and Business and Disability Network who went through a Disability Inclusion Soring Card session and came up with Disability Inclusion Action Plans, as well as Make 12.4% Work partners and National and District level OPDs.

The objects for evaluating changes at program level will be members of Make 12.4% Work, where program/ project level staff has been trained on Disability Awareness and Disability Inclusive Programming.

The objects for evaluating changes at eco system level will be members of Make 12.4% Work Steering Committee, Task Forces on Inclusive Livelihood and Inclusive Employment, and District Service Committees.

Specific objectives

• Evaluate the strength and limitations of the approaches used under the different elements of the Make 12.4% Work approach with a focus on:
• The Disability Inclusion Academy – Disability Inclusion Facilitators as agents of change
• Building capacity on Inclusive Livelihood Programming
• Supporting Employers to promote inclusive employment
• Preparing job seekers with disabilities for the workplace
• Influencing policy change and implementation
• Expanding knowledge and learning by doing
• Changing mindset and attitude toward persons with disabilities
• Use quantitative and qualitative methodologies to document changes in the lives of young women and men with disabilities, regarding economic status of the person and his or her household, social status, and self-esteem/ confidence and assess the contribution of the program
• Document evidence of contributions of Make 12.4% Work interventions to its members and strategic partners in becoming more disability inclusive as organisations, with a focus on creating more disability inclusive workplaces and program practices
• Highlight evidence of contributions of the M12.4%W program towards changes at ecosystem level with attention to mindset, policy change and implementation; and
• Draw lessons and formulate recommendations that inform efforts to scale up elements of the Make 12.4% Work approach to other programs in Uganda and other countries.

Evaluation criteria

This evaluation will adhere to the following evaluation criteria :

· RELEVANCE is the intervention doing the right things?

· COHERENCE how well does the intervention fit?

· EFFECTIVENESS is the intervention achieving its objectives?

· EFFICIENCY how well are resources being used?

· IMPACT what difference does the intervention make?

· SUSTAINABILITY will the benefits last?

3.2 Evaluation criteria and questions

Relevance and coherence

• To what extent do the different Make 12.4% Work components respond to the needs of young women and men with disabilities, government actors, companies and mainstream livelihood organizations in the area of disability inclusion/ inclusive economic empowerment ? With attention for intersectionality
• Does the way the program is designed fit the context of Uganda ? And what would need to be considered when replicating the approach outside of Uganda?
• Are the activities/ approach and output of the project consistent with the overall goal, attainment of objectives and intended impact and outcomes ?
• Are the program implementing partners relevant towards advancing the project objectives ?
• Recommendations to improve the relevance of the project to address barriers to access economic empowerment opportunities for young women and men with disabilities

Effectiveness

The Disability Inclusion Academy – Disability Inclusion Facilitators as agents of change

• To what extent have the DIFs been instrumental to bringing about change at the level of M12.4%W members and their organisations/ workplaces?**
• How effective has the Disability Inclusion Academy been to equip young women and men with knowledge, skills and behaviour to train and coach mainstream actors on disability inclusion **
• What strategies have been in place to enhance ownership and sustainability of the DIF model? What has worked well, and what could be done more?**
• How have OPDs been engaged in the preparation and implementation of the DIF model? What has worked well, and what could be done more/ differently? **

Building capacity on Inclusive Livelihood Programming

• To what extent do targeted mainstream livelihood organisations feel that they have been equipped sufficiently to become more disability inclusive in their practices?
• How effective are the methodologies and the tools used to support mainstream organisations to become more inclusive in their programming?
• To what extent have the Disability Awareness Trainings and trainings on Disability Inclusive Programming contributed to the disability confidence of staff of mainstream livelihood organisations?

Supporting Employers to promote inclusive employment

• How effective has the methodology and tools been to support employers to create a more inclusive workplace?
• How do different system actors consider the effectiveness of the Uganda Business and Disability Network in creating more disability inclusive workplaces?
• What is the contribution of the Disability Inclusive Employer Award to enhance more inclusive employment?

Preparing job seekers with disabilities for the workplace

• How do job seekers perceive the soft skill training, and the guidance into work experience placements as provided under the M12.4%W project?
• To what extent have the interventions to prepare job seekers for the job market led to actual jobs and/ or better access to the labour market?

Influencing policy change and implementation

• How effective have the two task forces (Inclusive Livelihood Development and Inclusive Employment) been in coming up with evidence and experience based recommendations for the M12.4%W Steering Committee?
• What are examples of policy influencing or informing that have been brought about through this structure of task forces and steering committee?

Expanding knowledge and learning by doing

• How has M12.4%W contributed to the knowledge base on disability inclusion in employment and livelihood programming?
• What has been the contribution of the Disability Inclusion Challenge to support solution building around Inclusive Economic Empowerment?
• What has been the value of other co-creation and innovation processes under M12.4%W?

Changing mindset and attitude toward persons with disabilities

• To what extent has M12.4%W contributed to changes in attitudes on the abilities of persons with disabilities? What method or approach has worked well?**

Efficiency

• Are the resources used in proportion with the number of persons with disabilities and changes brought about at the level of organisations reached out to?**

Impact

• What are the key achievements of Make 12.4% Work at: individual (young people with disabilities supported to access work), DIFs, mainstream actors, and at policy level
• Is there emergent evidence of change beyond the initiative to wider system actors both nationally and internationally?
• Has the Make 12.4% Work contributed to changes that positively influence the long-term social, economic, and institutional environment for individuals, communities, and institutions related to the project?

Sustainability and potential for upscaling

• What elements of the approach have proved to be effective and are ready for scale up?
• What elements need further contextualization to be scaled up in other Light countries and beyond?
• What are factors that enhance sustainability and what factors might negatively influence the sustainability of the program?
• On which aspect(s) is there a need to focus on in the future?

4. Evaluation methodology

4.1 Data Collection methods

We are looking at an evaluation that goes beyond traditional data collection methodologies. That is why we expect the evaluation team to propose methodologies that may include and may go beyond:

• Desk survey
• Focus Group Discussions
• Key Informant Interviews
• Storytelling
• Outcome mapping/ Outcome harvesting
• Photo voice

4.2 Actors involved in the evaluation

Light for the World is commissioning this evaluation and also expects to play a major role in the evaluation process itself, eg in data collection tool design and the development of the knowledge products.

NUDIPU and RAHU will also be engaged in the evaluation process being the main implementing partners in this evaluation

We further expect Disability Inclusion Facilitators to be engaged in the design of the methodologies as well as the data collection process.

To make this work we will establish a committee to guide the evaluation, and jointly organize a training for the DIFs directly engaged in the data collection process. Once the contracting process will have been completed we will have regular check in meetings with the team and come up with a detailed planning with clear roles and responsibilities. **

5. Expected deliverables and proposed schedule

5.1 Deliverables

Deliverables of the process evaluation include, but are not necessarily limited to, the following:

· Inception Report: This includes a detailed work plan, proposed methodology, planned list of study participants to be consulted and sampling framework, data collection and analysis tools, qualitative and quantitative protocols for data collection, and draft outline for the final report, for review by the program staff. The inception report should also include a revised proposal as per the particulars of the work plan, methodology and logistics.

· Presentation of preliminary findings: The evaluation firm will present preliminary findings to the Evaluation Group that will include Light for the World and partners representatives. This will help to identify and collect gaps related to data; further frame the task to focus on what is required in detail; and provide other timely feedbacks. The presentation will also keep the Program staff up to date about the status of the evaluation.

· Draft evaluation report: This will include all the evaluation data, analysis, findings, conclusions, recommendations, and lessons learned as defined by the outline presented in the inception report.

· Validation and learning workshop: The evaluators will present their findings and recommendations to the Scholars Program staff and other relevant partners to provide an opportunity to discuss and provide feedback.**

· Final evaluation report: The consultant/evaluation team should produce an evaluation report document describing all the process and findings of the evaluation with workable recommendations. Recommendations should be specific and include relevant details for how they might be implemented. The report should include a succinctly written Executive Summary. The final report will be submitted within 10 to 15 days after receiving comments and reflections on the first draft report (both in hard and soft copies). Three hard copies for each of draft and final report are required to be submitted and it should be accompanied by soft copy.

· Knowledge Mobilization products: In consultation and collaboration with Light for the World, the consultants will prepare at least one product per program component to share key finding and lessons learned from the Make 12.4% Work program with internal and external stakeholder.

5.2 Evaluation date and schedule

Total duration of mission

Propose a table indicating the different stages (for example: head office briefing, review of the literature, launch report, field mission, field feedback, drafting of final report) and the time periods when these stages are planned to take place

Inception Phase

Activities

a) Reviewing Make 12.4% Work background documents: proposal, communication, annual reviews, M&E framework, program reports, etc.

b) Develop both qualitative and quantitative data collection methodologies in close collaboration with Light for the World team and Disability Inclusion Facilitator team

c) Prepare draft inception report, including the proposed approach, methodology, work plan, evaluation matrix, and outline for final report

d) Arrange an inception meeting with Light for the World and M12.4%W partners to discuss the draft inception report and gather feedback

e) Prepare final inception report**

Timeframe : 2 weeks

Period: April/ May ‘22

Data collection Phase Activities

a) Meetings and discussions with stakeholders

b) Survey, interviews, focus group sessions and other data collection methodologies

Timeframe: 3 weeks

Period: May ‘22

Analysis and Reporting Phase

Activities

a) Conducting results analysis

b) Follow-up interviews where necessary

c) Preparation of draft report and review by Light for the World and M12.4%W partners

Timeframe: 2 weeks

Period: June ‘22

Validation and learning workshop

Activities

a) Presentation of findings, recommendations and lessons learned to Light for the World and partners

b) Incorporation of feedback collected from the workshop and other stakeholders

Timeframe: 1 week

Period: June ‘22

Final report

Activities

This report will address each of the specific objectives of the evaluation with a focus on documenting the approach and specific recommendations for scale up

Timeframe: 1 week

Period: July ‘22

Knowledge Mobilization product(s) Activities

These products are a crucial element of the evaluation and are meant for internal as well as external audiences so that the approach, its added value and recommendations for scale up can be widely disseminated

Timeframe: 2 weeks

Period: July ‘22

6. Resources

6.1 Expertise required from the consultant(s)

The consultancy firm should consider including the following experts/ experience in its team:

• Experts on conducting participatory impact evaluation, ideally with affinity with community based development, and diversity & inclusion
• Innovative evaluation methods like video based impact evaluation, photo voice, appreciative enquiry, outcome harvesting
• Understanding how learning and change occurs, and experience with documenting learning and change processes

6.2 Budget allocated to the evaluation

The budget ceiling for this evaluation is 17,000 USD. Please note that Disability Inclusion Facilitators can play a major role in data collection so less time needs to be budgeted for enumerators.