Terms of Reference for conducting an evaluation of CONCORDIA’s emergency response measures

Terms of Reference for conducting an evaluation of CONCORDIA’s emergency response measures in Austria, Bulgaria, the Republic of Moldova and Romania

1. Short description of the organisation

CONCORDIA Social Projects (CONCORDIA) is an association consisting of a non-profit foundation based in Vienna and seven non-profit sister organisations in Austria, Bulgaria, Germany, Kosovo, the Republic of Moldova, Romania and Switzerland. As an internationally active, independent aid organisation, CONCORDIA’s core mission is to support children, youth and families in need by providing direct and immediate assistance on the ground. It is CONCORDIA’s main task to ensure the protection of children and to support children and young people in developing their potential in order to consequently lead a self-determined and independent life.
Since 2016, Austria is also a project country – therefore, CONCORDIA Austria was established – with a Family and Learning Centre that focuses on the specific challenges of integrating children, young people and families with a migrant background. Despite the low level of child poverty compared to other European countries, the social background in Austria continues to have a decisive influence on the further course of life.
CONCORDIA Bulgaria was established in 2008 and builds on vast experience working with children, youth and families from marginalised communities, supporting them in their way towards leading a self-determined life.
CONCORDIA Romania has over 30 years of experience in implementing programmes and projects supporting at-risk children, youth and families on their path to an independent and self-determined life. CONCORDIA Romania is a certified social service provider by the Ministry of Labour, Family and Social Protection and all the social services are licensed according to existing national law and CONCORDIA quality standards.
CONCORDIA Moldova was set up in 2004 with the overall vision to make a substantial contribution to the social integration of disadvantaged groups (children, youth and families especially) – and in so doing, drive the whole country forward. To that end, CONCORDIA Moldova runs direct services and capacity-building programmes in 56 localities across the country, providing social services to socially vulnerable children, families and elderly people.

1. CONCORDIA’s response to the crisis situation

With the outbreak of the war in Ukraine in February 2022, a wave of refugees crossed the borders – among others – to Bulgaria, Romania and the Republic of Moldova, but also to Austria. With an experience of 31 years working in social services, CONCORDIA intervened responsibly and quickly by providing support to the people coming in the project countries.
The Family and Learning Centre of CONCORDIA Austria started providing support to refugees from Ukraine in the form of counselling to families with regard to school and labour market integration, offering German language classes to Ukrainian women, offering employment opportunities and assisting with administrative issues.
CONCORDIA Bulgaria also reacted promptly by providing accommodation at its Social Centre in Sofia (Sveti Konstantin), food, clothing, psychological support, first aid, accompaniment to institutions, employment support and assistance in travelling further to other countries.
CONCORDIA Moldova welcomed refugees at the border in Moldova (Palanca), provided them with food and places to sleep and organised long-term accommodation. To this end, vacant CONCORDIA buildings were re-opened in Tudora, Bolohani, Nemțeni, Ruseștii Noi, Stăuceni and Rîșcani and refugees were accompanied to the capital Chișinău. CONCORDIA Moldova developed specific projects to help refugees, providing food packages and sanitary kits throughout the country, but in the meantime supporting also Moldovan host families with these packages as well as with urgently needed firewood to keep their houses warm over the winter. In particularly poor areas, shower and laundry services were provided.
In Romania, the Emergency Situation Inspectorate in Bucharest started coordinating aid measures among a network of NGOs and agencies and acting as managing body of involved authorities. CONCORDIA Romania was put on the list of partnering NGOs to assist with accommodation, transport and direct services for refugees from Ukraine. The organisation provided food packages and basic products for refugee families and opened two transit centres in Bucharest (Casa Iuda) and in Ploiești (EduCampus) providing accommodation for refugees in transit. In addition, refugees were supported with travel support and other special needs (e.g. social tickets).

1. Purpose and objective of this evaluation

The main purpose of this external programme evaluation is to assess CONCORDIA’s emergency response in the above-mentioned countries with the aim of having an information base for our future plans for post-emergency and humanitarian activities.
The last year has been a tremendous learning opportunity for the CONCORDIA team at all levels of the organisation and the evaluation will help us take stock of the results achieved, as well as to reflect on the lessons learnt and the potential process and services improvement moving forward.

For CONCORDIA, providing emergency aid was a new intervention as it is not primarily a humanitarian organisation. Therefore, it is very important to get insights and receive learnings about the effectiveness and efficiency of its measures.

To this end, two interventions areas shall be assessed:

(1) CONCORDIA’s direct emergency activities in Austria, Bulgaria, Romania, Moldova
(2) CONCORDIA’s management processes: internal decision-making processes within the CONCORDIA family (all countries) and inside each CONCORDIA-country

For the first intervention area, the evaluation findings and recommendations shall contribute to giving insights about CONCORDIA’s interventions to support refugees from Ukraine who were either in transit through the respective countries or who stayed in one of these four countries. Two phases shall be distinguished in the evaluation of CONCORDIA’s emergency response programme:

1st phase: phase of direct/immediate support from March-September 2022;
2nd phase: phase of integration into school, labour market, society etc. between October 2022-March 2023; particular attention shall also be given to the measures undertaken by CONCORDIA in the fields of child protection and capacity-building.

The main guiding questions shall assess the following categories:

Impact:
– To what extent has the intervention generated effects (positive, negative, intended, unintended) on the refugees (and host families)?

Effectiveness:
– To what extent did the intervention achieve its objectives?
– How did the collaboration with other stakeholders go?
– How have the relations and dynamics between local communities and refugee families developed? How did CONCORDIA mitigate tensions?
– Is there something CONCORDIA missed for children, women, the elderly?
– What could have been done better for the different groups – women, children, the elderly?
– What worked well? What were the strengths of CONCORDIA’s intervention?

Relevance:
– To what extent did the intervention respond to the needs of the programme participants?
– What was overall challenging in addressing the needs of refugees on the ground?
– How appropriate were the services provided (differentiated between phase 1 and 2)?
– Are there age groups that CONCORDIA served better with its services?

Based on the findings, CONCORDIA would like to have the necessary information base to be able to decide, whether to do emergency activities again and, if yes, what to do better in the future.
For the second intervention area, the focus shall lie on the way in which decisions regarding the emergency programme were taken, as well as on coordination and management processes within and between countries. The purpose is to learn on an organisational and management level for future similar situations that require immediate and complex responses.
The main guiding questions shall assess the following categories:
Impact:
– To what extent were processes established successfully, how well did organisational structures and decision-making processes respond to the challenge?

Effectiveness:
– How effective did the process of coordination and decision-making continue throughout phase 1 and phase 2?
– How effective were the dynamics between different countries and organisations and how did they evolve?
– What was overall challenging in terms of establishing processes, organisational structures and making decisions?
– To what extent were the established processes, organisational structures and staff capacity appropriate for the humanitarian intervention?

Relevance:
– How were decisions made in the beginning of the conflict? How did the process of setting up and deciding on phase 1 go?

4. Qualifications of the evaluation team

The evaluator (or the evaluation team) shall have the following key qualifications:
– A well-founded background in social science (university degree);
– A minimum of 3 years’ experience and expertise in the evaluation of similar programmes and appropriate knowledge of data collection methods and data analysis;
– Participation in at least three evaluations, ideally in the field of emergency response/ humanitarian aid/ social service provision/ civil society interventions;
– A solid understanding of Humanitarian Principles and the Core Humanitarian Standard on Quality and Accountability;
– Fluency in English (reading, speaking, writing); knowledge of Romanian is considered as big advantage;
– Sound MS Office and IT skills;
– Sound experience in participative research methods is an advantage;
– At least one letter of recommendation from relevant institutions/NGOs.

5. Approach and methods

The evaluation consists of several phases:

Contract and Kick-off meeting:

Contract is signed and a discussion of the assignment takes place between the evaluator/evaluation team and CONCORDIA professionals based in Vienna (headed by the International Programme Director). First documents, including available data, are provided to the evaluation team.

Desk Study:

The evaluator(s) studies/study all necessary documents; existing data need to be analysed and interpreted. Upon request and availability, further information can be provided by CONCORDIA.
Inception-Phase: In the inception phase, a meeting will take place where the design of the evaluation and the methodology on data collection and analysis will be discussed and agreed upon. For efficiency, the evaluators are asked to send their suggested methodology and evaluation plan beforehand to the CONCORDIA team. Data triangulation and quality control are very important and need to be discussed in the inception phase.

Field/Research-phase:

Data needs to be gathered, analysed and interpreted. It is expected that the evaluation will include quantitative and qualitative data disaggregated by sex and age. While most data will be collected through online interviews and other tools, some travel capacity in the countries would be recommendable to be able to also have in-person conversations.
Intermediary phase: after 1-2 months, another meeting will take place with the Vienna team where the evaluator(s) will present the key intermediary findings (feedback workshop) after the field trips/interviews.

Final Draft Report:

Submission and presentation of final draft report, inclusion of comments from partners and CONCORDIA-Team.

Final Report:

Submission of the final report, 6 months after the kick-off meeting, the latest.

For the different phases it is expected that data and information will be obtained through different methods, which could include:
– analysis of relevant literature and documents
– structured or semi-structured interviews face-to face or online
– group discussions face-to face or online
Following time schedule is envisaged, but the final working calendar will be jointly agreed:

Task Proposed time Relevant activities

Kick-off meeting
By 30.06.2023
discussion of the assignment; first documents, including available data, are provided to the evaluator(s)Finalisation of desk study
By 31.07.2023Analysis and interpretation of existing data and documents
Finalisation of the inception phase
By 15.08.2023
Presentation of the design of the evaluation and the data collection and analysis
Finalisation of field/research phase
By 15.10.2023
Conduction of first interviews; gathering, analysis and interpretation of field data
Intermediary meeting
Mid-October 2023
Presentation of key findings (feedback workshop) to the Vienna team after the field trips/interviews
Submission of final draft report
By 30.11.2023
Submission and presentation of final draft report; inclusion of comments from partners and contractor
Submission of the final report
By 20.12.2023
Submission and presentation to the contractor

1. Expected results (deliverables) from the evaluation

CONCORDIA expects a final evaluation report to be delivered in English and ideally, should not exceed 30 pages (annexes not included). A description of the instruments is to be provided in English.
The final report should include:
➢ Analysis of external context (environment) where the emergency response activities took place.
➢ Main results of the research regarding the two focus points of the evaluation (emergency implementation and management process)
➢ Annexes including all findings and results of the data analysis, as well as stakeholders consulted.
➢ Applied research tools (e.g., questionnaires, interview schedules, focus groups’ agenda etc.).
➢ All relevant literature sources.
Ideally, the final report should use the following structure:
➢ Title page
➢ Acknowledgements
➢ A table of content
➢ An executive summary
➢ An introductory chapter setting the context and the nature of the problem
➢ An explanatory chapter of the research methodology and design, the research tools and key research questions
➢ A chapter presenting in detail the findings and the results of the data analysis
➢ A chapter with detailed conclusions and recommendations
➢ References
➢ All necessary annexes

7. Coordination responsibility

Ms. Nadja Kohlbach-Horesovsky will be the contact person for this evaluation.
Contact details: nadja.kohlbach-horesovsky@concordia.or.at, +43 1 212 81 49–26

8. Budget

The total budget of the evaluation must not exceed EUR 10,000 (including VAT, if the case). The financial proposal should include all occurred expenditures, e.g., travelling costs for the evaluator(s), office supplies, protocol, etc. Please note that the evaluator(s) will be responsible for paying income taxes and any other due taxes as per applicable national legislation.
The consultant will submit invoices to CONCORDIA Austria as following:

– The first invoice after the approval of the inception meeting – 20% of the total budget
– The final invoice after the approval of the final report – 80% of the total budget

9. Application procedure

The application will be sent via email to: nadja.kohlbach-horesovsky@concordia.or.at and must include:

1. CV/CVs focused on the skills and experience requested;
2. A cover letter underlining the motivation for applying for this assessment and how the candidate(s’) skills and experience match the requirements of the evaluation;
3. A short financial plan that clearly states how the budget (EUR 10,000) will be spent. All costs (including travel costs) must be included in the budget frame.
   If invited for an interview, candidates are asked to prepare a brief research plan (max. 2 pages) to be presented and discussed during the interview.
   The name of the bidder(s), phone numbers, email-addresses must be specified.
4. Selection procedure
   Only shortlisted candidates will be contacted for an interview (online or offline). Applications which do not include the financial offer will not be considered. Interactive research designs will be preferred.
   The contract will be awarded based on the best combination of expertise and the financial and evaluation research plan.

Advertised: 8.5.2023

Deadline for submission: 5.6.2023 (interviews take place on a rolling basis)