4.1 In-country research
The purpose of the in-country research is to provide a more in-depth understanding of the performance analysis and key trends and themes in humanitarian operations in specific crisis responses. The country studies will be used to understand how the humanitarian system operates and how it performs in different crisis contexts. They will also be used to understand the specific constraints and features of the different context types, and the role these play in how assistance is planned, coordinated, and provided. This component also provides the opportunity to interrogate key policy questions of this study period by exploring important contextual examples of those themes and issues, such as contexts affected by climate change or inter-state conflict.
For the sixth edition, ALNAP will strengthen its partnership approach with local researchers for this component and ideally work with local research institutions or learning networks to implement this component. Through this approach, local researchers will feed into the overall study matrix of the global report, help
determine key thematic issues to focus on in each context, and co-produce country reports to be published alongside the global report. These will present context specific findings that can be used to support local change for those who participated in the research. Working with institutions and learning networks rather than sole researchers should also help support the sustainability of dissemination plans and the uptake of findings.
Up to six country cases will be chosen to achieve a representative balance of the main characteristics that shape the crises to which humanitarians respond, while also capturing key crisis events in the study period that enable the examination of key thematic issues. These will include consideration of the following (not mutually exclusive) factors:
- A spread across geographic regions
- At least one protracted/long-term crises
- At least one ‘forgotten’ or underfunded crisis
- At least one context featuring strong locally led action
- At least one ‘politically estranged’ context
- At least one context experiencing interstate war
- At least one context experiencing the effects of climate change
The in-country studies provide a critical opportunity to consult with people affected by crisis and with national stakeholders and thereby ensure that the views of national authorities and national and local civil society organisations (CSOs) are represented in assessing the performance of the system. The in-country research will include interviews, FGDs and documentary research. The in-country research will be designed to achieve broad representation from different humanitarian actors and the entities and individuals engaged in crisis response within a given country but also to explore thematic questions pertinent to that context.
Within the six selected contexts, a sampling frame will be used to identify people affected by crisis for a series of FGDs, seeking a balance across age group, gender, and ethnic identity. The type of organisation engaged with, and type of
assistance provided will also be included in the sampling frame, to elicit a wide set of experiences with humanitarian action. In some contexts, there may be security and access issues that require research engagement with people affected by crisis via online FGDs, one-to-one interviews or diary-based data collection.
KIIs and FGDs will be conducted according to an interview protocol designed on the basis of the SOHS study matrix and tailored to key issues in each context. They will be audio recorded with the prior permission of the interviewees, transcribed, and then coded using MaxQDA according to a coding matrix provided by ALNAP which will be used for consistency across all the SOHS Study components.
A small number of FGDs with crisis affected people will be conducted in late-Autumn 2024 to understand their key priorities that the research should consider. The research framework and study matrix will be updated to reflect those perspectives. The bulk of the FGDs and KIIs will take place from January 2024 to late 2025, following discussions with country research leads on refining research questions and approaches for those contexts.
Table 4: Country-level key informant interviews (KIIs): interviewees by type (per country)
4.2 Aid recipient surveys
The aid recipient surveys carried out in previous editions of the SOHS provided valuable insights into how recipients experience humanitarian assistance. They shed light on the effectiveness of the mechanisms used by humanitarian actors to improve the quality, relevance, and accountability of their work, and facilitate comparisons of the perspectives of humanitarian professionals and aid recipients on the quality of aid.
The sixth edition will use a mixture of in-person and remote modalities – including computer-assisted web interviewing (CAWI), computer-assisted personal interviewing (CAPI) and computer-assisted telephone interviewing (CATI) –to collect data from aid recipients in selected humanitarian response settings. The surveys will use largely the same research questions from previous editions to provide consistent comparisons over time. As such, Aid recipients will be asked for their opinions of the timeliness, quantity, and quality of aid, whether it addressed their priority needs, and how it could be improved. A small number of questions may be altered or added to explore issues particularly pertinent to the key issues and policy questions for this period. For example, a question could be included to ask about the extent to which people affected by crisis have trust in the humanitarian system.
Previous SOHS reports have relied solely on remote methods for the survey. A portion of the surveys in each context will be conducted in-person for this edition to include the views of people who do not have access to mobile phones. The sample frame will also be constructed by in-person engagement to increase the level of confidence that people included have actually received assistance from humanitarian action, which is harder to verify accurately through mobile contact. A mapping of active humanitarian organisations in the locality will also be conducted to increase confidence that the people reached by the survey have received humanitarian assistance (rather than assistance from development actors or no assistance).
The surveys will aim to reach approximately 5,000 aid recipients across five contexts.
The sample will have a gender and age balance and seek to be as representative as possible of different social sectors, which will be supported by the range of modalities used.
4.3 Global aid practitioner survey
The purpose of the global aid practitioner survey is to gather qualitative and quantitative information from the widest possible range of humanitarian aid practitioners, while allowing trends to be tracked from previous SOHS editions. The survey will seek to elicit a current appraisal of the system as it relates to respondents’ direct experience, as well as an assessment of whether and how these areas have changed since data was last collected in 2021.
Questions in the online survey will therefore remain largely the same as in the SOHS 2022 report to maintain a baseline comparison across years, although some questions may be omitted and others added based on their usefulness to the analysis and relevance to current policy questions.
The survey will be available online in at least Arabic, English, French and Spanish, with other key languages – for example, Ukrainian – considered. Support and Advisory Group (SAG) members will be identified to promote and disseminate the survey among their respective organisations and networks.
The links to each survey will be disseminated by the study team and SAG members, and published on the ALNAP website. In addition, the team will seek permission to place the online survey and links on the following websites and networks: ReliefWeb, DARA, InterAction, ICVA and OCHA. We will also request that key networks of local actors kindly disseminate the survey to their members to increase the diversity of perspectives provided. For example, the Asia Disaster Risk Reduction Network, the NEAR network, and the Pacific Island Network of Non-Government Organisations. The survey will also be disseminated through targeted use of social media and through in-country networks supported by local research partners engaged in the in-country research.
The survey will be launched by February 2025 and kept open until September 2025. The analysis will be completed by the end of November 2025.
4.4 Key informant interviews: HQ/global level
The HQ/global or regional-level KIIs are designed to gather perception-based data from key stakeholders in the humanitarian system at these levels to complement the in-country KIIs (described above). The study team will conduct interviews with approximately 60 key informants online, or in person where travel is not required.
The bulk of the interviews, which will be conducted concurrently with other study activities, will be completed by October 2025 to feed into the main analysis period alongside the data produced by other components. The bulk of the interviews will be held across the period August 2024–October 2025, to allow the team to conduct both broad-based and exploratory interviews at the start of the data-collection period and more targeted interviews to gather data for specific research question and chapters. However, additional interviews will take place over the course of the writing up to March 2026 to help fill evidence gaps or explore hypotheses from the initial data analysis taking place in Autumn 2025.
The initial selection of interviewees will be designed both to achieve broad representation from across the humanitarian system and with key development and peace actors (as indicated in Table 3) but also to explore key thematic issues of particular relevance to this study period as outlined in Section 4.9. While national authorities and national NGOs will mainly be interviewed in the in-country research and can input via the practitioner surveys, the co-leads will also conduct some interviews of local actors at an early stage to better engage with those perspectives throughout the research cycle.
Table 5: HQ/global-level key informant interviews (KIIs): interviewees by organisation type
4.5 Organisational mapping and analysis
The second component contributing the statistical analysis will map the organisational configuration of the system. The sixth edition of the SOHS will build on past analysis to provide the longitudinal trend on key organisational statistics, including original data collection and analysis to provide the following:
- Number of organisations engaged in humanitarian action worldwide
- Largest humanitarian organisations, based on staff and annual humanitarian expenditure
- Total international staff
- Total national staff
- Building on the analysis conducted for the fifth edition, we will also seek to explore pay differentials between different type of staff and organisations.
Organisational data collection and analysis will, wherever possible, be disaggregated according to the balance of gender, ethnicity, and local/international staff, and at different levels of leadership or seniority across organisations. Gaps in the availability of such data will also be noted.
4.6 Evaluation synthesis
The evaluation synthesis is designed to condense and synthesise findings from the large number of evaluations conducted within the international humanitarian
system each year, revealing a broader picture of overall system-level performance. It will summarise findings of evaluations undertaken between January 2022 and December 2025. The evaluation synthesis will be a particularly rich source of data to inform the assessment of longitudinal performance against the different DAC criteria, as well as evaluating performance on key thematic areas.
The study team will compile documents primarily from the ALNAP database of evaluations within the HELP Library, as well as other public and non-public (i.e. internal organisational, or ‘grey’ literature) sources, and record the findings for each using a specific matrix for the evaluation synthesis. This matrix will retain a similar structure to that used in the fifth edition, including a rating system to weight findings based on evaluation quality. Although the synthesis analysis will remain mainly qualitative, the matrix will help to ensure the greatest possible degree of comparability across the findings and avoid potential bias.
The evaluation synthesis method will include two steps:
Step 1:
Categorising and coding the findings and recommendations from each evaluation report in an evaluation synthesis matrix. The matrix and coding framework will build on the protocol used in the evaluation synthesis for the SOHS 2022, as well as being informed by other evaluation syntheses and the United Nations Evaluation Group (UNEG), OECD and other guidance. While the framework will evolve in the early stages of the analysis it will likely include the following fields:
- ID#
- evaluation title
- year
- evaluator
- published/unpublished
- quality score
- commissioning agency
- evaluation type
- scope and timeframe
- subject area
- findings against criteria and indicators presented in the SOHS study matrix and the OECD criteria (see Table 1 and 2)
- core conclusions
- weaknesses
- good practices
- priority recommendations
Step 2:
Synthesising findings against each indicator in the SOHS study matrix listed as relevant to the evaluation synthesis in a summary report. The synthesis findings will be presented in a structure based on the analytical framework presented in Table 1 , drawn from all relevant areas of the evaluation synthesis matrix (findings, conclusions, recommendations). The synthesis will take into account the strength of evidence for each finding on the basis of the number, breadth and quality of evaluations supporting it. It will also present an overview of trends in evaluation approaches and quality in the period.
4.7 Financial flows to humanitarian emergenices
The data collected for this component will support the descriptive analysis of the size and shape of the humanitarian system and how that has shifted over time, while also contributing key quantitative data points for the performance analysis across different OECD criteria and related policy questions.
The full set of financial analysis questions will be finalised as part of the initial consultations. The initial areas for data analysis are:
- Total IHA trend 2012-2025
- Proportion provided by donor governments, including breakdown by government and any significant changes/trends, 2022-2025
- Proportion of IHA provided by private sector and private donations, 2022- 2025, and indication of trends
- Funding volumes to types of emergencies – complex emergencies, disasters caused by environmental hazards, refugee displacement, 2022 – 2025, and indication of trends
- ‘Humanitarian’ funding from multilateral development banks, 2022-2024
- Deep dive on World Bank crisis/emergency instruments
- Volume of funds through the United Nations Central Emergency Response Fund (CERF), country-level pooled funds, Common Humanitarian Fund (CHF) and START Fund, 2022–2025
- Overview of delivery channels: UN agencies, NGOs, the Red Cross and Red Crescent Movement, private organisations (as implementers, depending on what data allows), host governments, unspecified, 2022-2024
- Concentration of funding among agencies; top 20 recipients of aid in study period and comparison to previous period
- Changes in distribution and concentration over the past 10 years; how many countries account for 50% of IHA and how many countries occupy the ‘tail’ of emergencies that received less
- Disbursements of forgotten crisis pooled funds (CERF underfunded crises; CBPF in countries that are on the ECHO forgotten crisis list; could also include internal donor allocation mechanisms for forgotten/overlooked crises)
- Total stated requirements and levels of funding UN-coordinated appeals, by country, 2022–2025
- Requirements and funding per intended recipient in UN-coordinated appeals, 2022-25
- Requirements and funding per technical sector in UN-coordinated appeals, 2022-2025
- Total IHA for cash and voucher programming, 2022-2025
- Proportion of IHA for cash and voucher programming compared to all other IHA, 2022-2025
- Populations in need and populations targeted and reached in UN coordinated appeals (also disaggregated by gender and age), 2022-2025
- Trends in levels of requirements met over course of 10-year crises
- Trends in levels of requirements met over first 5 years of a crisis (by crisis type)
- Analysis of patterns in sectoral spending over course of crises
- Which countries have been receiving large volumes of IHA for the longest period and have consecutively featured in the top 10 recipients (i) every year; (ii) 8–10 times; or (iii) 5–8 times
- Volumes of IHA for disaster risk reduction (DRR), disaster preparedness, broken down by donor, 2022-2025
- Unearmarked funding: 2022–2025 trends in volumes, donors and recipient agencies
- Specified funds for early/anticipatory action
- Cost data on particular aspects of humanitarian distribution and comparison in-country, e.g. cost of transporting a particular relief item, cost of fuel, to highlight differences in operating costs across countries
- Total flows to national and local NGOs, direct and (where possible) indirect, 2022-2025
- Proportion of aid to national and local NGOs compared with other groups, such as the UN agencies, Northern-based international NGOs, 2022-2025
- Mapping of local/national organisations receiving funds by sector/ geography, 2025 size of humanitarian financial flows compared to other significant financial flows in largest recipient countries of IHA in 2025 and in case-study countries: national governments’ non-grant revenue, peacekeeping, remittances, non-humanitarian ODA, including expenditure on DRR and climate-change adaptation (CAA)
- Volumes of IHA for early recovery in conflict settings, 2022-2025
- Other data, including pertaining to the thematic focus areas (described below) on detailed analysis of flows of development, peace and climate funding
4.8 Literature review
Key policy and practice themes will be determined by the co-leads for the literature review of research reports and academic work published within the study period.
A set of topics will be determined during the inception phase based on the study matrix to feed into the main analysis of data in Autumn 2025 and additional topics will be chosen after that point to help fill evidence gaps highlighted by other research components or to explore newer issues emerging towards the end of the study period.
The methodology will follow a similar structured search and screening approach to inclusion, analysis, and presentation of findings used in the last report.
4.9 Thematic studies
This edition of the SOHS will again include some choice primary thematic studies; a research component adopted for the fifth edition to provide objective performance data designed to fill key research gaps of contemporary relevance. Several topics were identified in the inception phase that require additional efforts to explore in the sixth edition. The majority of these topics will be examined through additional data collection in the existing research components outlined above. For example, they can be captured through commissioning of additional data in the financial analysis or conducting targeted global KIIs in these topics. Only one – Sufficiency and the humanitarian prioritisation challenge – is likely to be commissioned as an independent research study due to the depth of additional data gathering required to answer important research questions on that theme. The list of topics ALNAP will explore as new themes for this edition are outlined below with brief descriptions of the topic, research methods, and likely research questions.
Sufficiency and the humanitarian prioritisation challenge
After years of steady increase, followed by a relative plateauing of humanitarian funding, the humanitarian sector is facing an unprecedented drop in funding levels during the study period for the sixth edition. It will be critical to understand and reflect the impacts of this drop and how humanitarian agencies grappled with a fall in financing – in particular, to understand how this has affected how
different humanitarian actors make needs-based allocation of scarce humanitarian resources. Evaluations are unlikely to capture these issues in detail in time for the drafting of the SOHS report in late 2025 and therefore reflecting these trends adequately will require a dedicated primary research component. This thematic issue will be explored in a commissioned piece of research. The precise research methodology will be discussed with the consultant lead.
Several key questions can be raised that examine both decisions made and their implications:
- Where and who have been prioritized for targeting by humanitarian actors in this period of economic constraints? Why were decisions made?
- What has been the impact for people affected by crisis who are not targeted as a result of budget cuts to humanitarian programmes? Are actors working across the HDP nexus in these settings to address broader vulnerabilities?
- How have prioritisation decisions had an impact on performance of humanitarian assistance? For example, in terms of efficiency, effectiveness, relevance or sustainability?
Making sense of the development, peace, and diplomacy gap
Attempts to strengthen linkages with development and peace efforts have increased significantly in the humanitarian sector over the past 8 years, as so-called ‘HDP nexus’ approaches are touted as the answer to an over-stretched humanitarian system. But the promise of nexus approaches has been hampered by several challenges – these range from a lack of consensus on what good HDP linkages
look like in practice, to challenges incentivising stronger development engagement in fragile settings, particularly those featuring ‘politically constrained/estranged’ states.
Data exists to support understanding of the drivers for the HDP nexus and how organisational efforts are performing. However, additional KIIs will be conducted to capture some understudied aspects of the nexus made more important by the
current situation of interstate conflict and geopolitical turmoil. The aim of these KIIs would be to better understand the political decision-making dynamics that serve as the root cause for many of the known challenges and dysfunctions in foreign aid and diplomacy that are hampering better development and peace engagement in fragile settings. Potential research questions to be explored include:
- How is the ‘toolbox’ of foreign policy options (e.g. diplomacy, foreign aid, sanctions) framed and decided by political operatives and appointees in aid-providing states?
- What, if any, are the opportunities for influencing these decisions to address the known gaps and challenges in HDP nexus efforts?
- How do aid-receiving states navigate, advocate, and influence the nature of third-party interventions in their crises?
Looking outside the multilateral humanitarian system
International humanitarian assistance and protection efforts are rarely the only form of support available to people affected by crises. People in crises draw on a wide mix of assets and resources which include both domestic sources – community self-help, government salaries, private sector philanthropy, religious giving – as well as private international sources – such as philanthropic foundations, remittances, or zakat. Getting an adequate picture of this mix of resources is challenging. Few organisations collect data on these resource flows, and figures that do exist for remittance tracking or zakat, are disaggregated only down to the country level.
Some private sector actors already form a key part of the multilateral humanitarian system through supply chains, while others – large, small, local, or international – operate outside it.
Existing research components will be expanded to collect additional data on support outside of the multilateral humanitarian system. This will include targeted KIIs, tailored questions in the in-country research and additional data collection commissioned within the financial statistics component to capture some of these funding flows. Draft questions that may be answered include:
- Where and how much do the top 10 non-DAC donors give for humanitarian aid?
- Who are the primary recipients of this aid and what programmes and activities do they fund with it?
- How do non-DAC donors define their humanitarian work in contrast to development or wider foreign policy aims and how do they target/prioritise?
- What is their view of the multilateral humanitarian system and their motivations for engaging or not engaging?
- What is the volume and estimated value of support provided by different types of international and local private sector actors [13] in a crisis response, and how does this compare with volume and cost of the humanitarian response?
- How are the decisions and behaviours of private sector actors influenced by the protracted nature of crises in the most extremely fragile settings?
- Where are the perceived risks and opportunities for private sector actors to engage more systematically in crisis response?
- How well does the multilateral humanitarian system coordinate with these wider networks and systems?
Climate change
While the pressure mounts for humanitarians to determine how they will respond to the growing climate crisis and where they can add most value, there is a dichotomy over their appropriate role. Some actors are arguing for a larger piece of the pie, scaling up anticipatory action and broadening and solidifying the humanitarian remit to include disaster risk reduction and resilience programming. Others are calling for a stronger nexus approach, working with actors who are better positioned to address the underlying structural problems and inequalities rather than expanding their own function.
To better understand how humanitarian actors are adapting in relation to the climate crisis, existing research components will gather additional data on these issues. Additional data points will be captured through targeted KIIs, in-country research in affected contexts, and via the financial statistics component to explore financial flows of climate financing to humanitarian agencies. Potential questions that will be explored include:
- What operational role should humanitarians be expected to play in the climate crisis? What are the options and entry points?
- Where can the humanitarian system add most value, and what are the tensions and trade-offs for an already stretched sector?
- How much funding are humanitarian donors putting towards climate action?
- What tools and approaches are humanitarian donors investing in and prioritizing to support a humanitarian response to the climate crisis?
- What is the range of anticipatory actions for climate change being conducted by humanitarian actors and what is their impact, including locally led climate adaption models?
[13] For example, multinational corporations without a disaster/service delivery focus; local private sector (formal and informal); and private sector companies (both domestic and international) with disaster/service delivery focus that can replace or be sub-contracted by humanitarian agencies.