Can funding constraints push us toward better, more focused evaluation?

ALNAP’s recent evaluation guide was largely written before the current funding constraints hit the humanitarian system. But one of its key principles, the thoughtful and utility-focused use of evaluation criteria, is more relevant than ever. Read this commentary from Sarah Gharbi, one of the guides 3 co-authors, exploring the risks and opportunities the current funding environment presents.

At ALNAP, we continuously encourage evaluation practitioners - both commissioners and evaluators - to put utility at the forefront of the evaluation process. When applying evaluation criteria, this means we do not recommend using the list in our guide as a rigid checklist. Instead, we advocate for a thoughtful and flexible approach, where the needs of evaluation users guide the selection of criteria and the development of evaluation questions.

The rigid application of criteria - often resulting in an overload of evaluation questions - is a tale as old as time in evaluation practice. This not only affects the quality and use of evaluations, but ultimately limits what we are able to learn from them.

In the current funding environment, these long‑standing challenges risk becoming even more pronounced. So what can we, as evaluation practitioners, do to continue enhancing the quality and use of evaluations to support learning and accountability in the humanitarian sector?

It is too early to have a full picture of what the funding constraints will have on humanitarian evaluation practice. But when I have spoken to ALNAP members and other evaluation stakeholders, some issues are already emerging:

Fewer resources for individual evaluations might mean less workdays for evaluators, less funds for travel to affected contexts, and less willingness to pay for experienced evaluators. And yet,  the scope of evaluations is maintained – the same number of questions are expected to be answered by a smaller and more inexperienced team.

We’re also hearing that many organisations have had to drastically downsize their M&E or MEAL teams, leaving more responsibilities on programme or operational staff.

And perhaps, most seriously of all, we are hearing about increasing pressure on evaluators to only include positive results in evaluations, omitting what has not worked, as organisations fear repercussions and termination of support from donors.

These issues carry real risks for evaluation practice. If the scope of evaluations is not adapted to the resources available, we risk producing  lower-quality evaluations  that are feel even more of a ‘tick-box’ exercise. If commissioners insist on using all criteria in an evaluation but cannot staff the evaluation team correctly, the quality will suffer. Reduced MEAL capacity can weaken evaluation processes across the board - from weaker terms of reference to weaker follow-up - leading to decreased learning. And if evaluators are pressured to skew results towards the positive and we don’t assess what didn’t work, how can we truly learn or be accountable?

But in the spirit of a half glass full, there could also be opportunities with the new funding environment.

Perhaps we will see more joint evaluations as agencies face fewer resources for evaluations. This could really enhance inter-agency learning and, if I may be optimistic, increase the chances of learning and accountability at system level. Funding restrictions might also mean greater opportunities for local evaluators. If commissioners within the international humanitarian system are forced to abandon the traditional and biased view that international evaluators are more competent because they can’t afford to fly them half way across the world, could it mean that they will be more open to locally led evaluation teams?

My hope is that in these times of hyper prioritisation, we in the humanitarian evaluation community, can make better, more strategic decisions . Let us not evaluate everything, let us not use all the evaluation criteria in a single evaluation. Instead, let us do more focused and useful evaluations. If resources for evaluations are constrained, let us really think about what we need to know. Let us work together, joining forces for collective learning. And above all, let us always put affected people at the center.