Effectiveness

Definition

What has humanitarian action achieved? How well and for whom?

Effectiveness measures the results achieved by humanitarian action across different groups, and the quality of humanitarian assistance

Explanation of definition

Focus on what humanitarian action has resulted in, in practice, and relate those results to the lives, livelihoods and protection of people affected by crisis. Consider both intended and unintended results where the contribution of humanitarian action can be established. As highlighted by the OECD, it is important to explore unintended effects to identify both negative and positive results (OECD, 2021). For example, use open-ended enquiry when evaluating protection programming. Protecting people affected by crisis means reducing a range of risk factors (physical, legal, economic) that are spread across a complex system. The complexity of protection issues means that it is difficult to predict the full range of possible consequences at the start of a response. Therefore, it is not enough to only consider intended results (ALNAP, 2018).

Refrain from simply listing which activities and outputs have been delivered.^[1] Instead, focus your analysis on results at the outcome level. For example, don’t just count the number of training courses on protection referral systems provided to frontline service providers. What were the results of those trainings in terms of referral ratings and quality of protection provided? Relate the results to the overall objectives of humanitarian action, i.e. to protect and save lives, to alleviate suffering and to maintain human dignity.

Critically, the effectiveness criterion is also about evaluating the quality of results. To determine an acceptable level of quality – what is ‘good enough’ – use global frameworks such as the Sphere standards.^[2] Also seek the perspectives of affected people, across different groups, on the quality of programming.

Identify key factors of success and failure when evaluating effectiveness. These factors can be internal (e.g. related to the design or implementation of humanitarian action) and external (e.g. factors related to the context). Consider if and how partnerships with other actors contributed to the effectiveness of the humanitarian response.

Explore the assumptions underpinning the logic of the humanitarian response. Do these assumptions accurately reflect the context of the crisis and potential results of humanitarian action? Unpack and interrogate the logic or theory of change of the response.

Where possible, evaluate intended or unintended results related to the environment and climate crisis, including both positive and negative results (i.e. environmental damage). Humanitarian actors can, for example, worsen deforestation if sustainable building practices are not adopted when providing shelter for people affected by crisis. See section 11.3 Environment and climate crisis.

When to select effectiveness

Evaluating effectiveness provides an opportunity to understand the outcomes of humanitarian action beyond a mere description of activities and outputs. This is critical to improving programme performance and it complements ongoing monitoring.

How effectiveness relates to other criteria

An evaluation that combines effectiveness with relevance, and with coverage and inclusion, will provide an overview of what has been achieved and how well, and also if the humanitarian response is doing the right things for the right people. Humanitarian action might be highly effective in doing what an organisation set out to achieve, but it might have become irrelevant to the needs and priorities of affected people if the context has changed but programming has not adapted.

By combining effectiveness with efficiency, evaluations can capture valuable information on the timeliness of humanitarian action. Results need to be achieved at the right time, when humanitarian action is most needed. Note that some humanitarian actors may choose to evaluate timeliness under effectiveness, especially if efficiency is not included as a distinct criterion.

Effectiveness also links to the impact criterion, which explores results at a higher level. Evaluating impact means analysing intended and unintended results, but these are broader and usually longer-term in nature.

There are two key entry points here. First, trace how power dynamics have influenced which results have been prioritised – for instance, are the results most visible in reports aligned with those valued by the communities affected by crisis? Second, explore how positionality has shaped the assumptions built into programme design. Whose perspectives informed the theory of change? Were assumptions tested in practice – such as that community leaders would represent everyone fairly in targeting or planning? Shifting the lens is an opportunity to surface assumptions that may have limited the effectiveness of humanitarian action, particularly for specific groups.

See Chapter 11 for further methodological implications, particularly key considerations for putting people affected by crisis at the centre.

See Chapter 11 for further methodological implications, particularly key considerations for putting people affected by crisis at the centre.