Addressing protection in humanitarian evaluations: what ALNAP’s new guidance needs to achieve

In June this year ALNAP brought together hundreds of practitioners across the sector to validate proposed updates to the OECD-DAC criteria. Together, they have been reviewing the original ALNAP guide published in 2006, to support greater precision in humanitarian evaluations and address significant changes in the humanitarian landscape since.

These new standards are being developed at a time when the humanitarian system is under immense pressure. The highest number of conflicts since the end of the second world war alongside the increased frequency and intensity of climate shocks are driving soaring humanitarian needs amidst shrinking aid budgets. Decreasing respect for the rules of war are meanwhile having devastating consequences for civilians in need of protective humanitarian action - from Gaza and Myanmar to Ukraine and Sudan, among too many other crises.

Navigating difficult aid dilemma’s

Too often humanitarians are the last actors present trying to preserve humanity between unrestrained or targeted violence and the fall out for civilians. How humanitarians and donors design and deliver humanitarian responses in such environments has far reaching implications for at-risk populations.

The Gaza crisis continues to demonstrate the dangers of unsafe and undignified aid delivery for an increasingly desperate population facing a high risk of famine and restricted humanitarian access. Lack of safe equitable access to assistance for crisis-affected populations alongside increased attacks on aid workers has weaponised the safe unhindered delivery of humanitarian assistance.

Successive crises from the 1990s to the present have informed commitments and standards intended to enhance protection for at-risk civilians. The independent review of the UN’s failure to protect civilians in Sri Lanka was closely followed by the Secretary-General’s Human Rights Up Front initiative and the Inter-Agency Standing Committee (IASC) commitment to the centrality of protection in humanitarian action (2013). Several related reviews since then, including the Independent Whole of System Review of Protection, IASC annual centrality of protection reports, Global Protection Cluster annual reports and ODI review of the IASC protection policy (2022), highlight concerns over increasing threats and challenges alongside decreasing compliance with international law. In 2018, ALNAP also published its Guide to evaluating protection in humanitarian action which aimed to give evaluation teams and commissioning staff the tools to navigate the challenges that protection work raises.

Proposed updates to evaluation standards for humanitarian action need to capture these challenges more effectively and align with the sectors own minimum standards.

Making protection central to humanitarian evaluations

Civilian populations in crises need safety, their dignity respected, and access to clean water, food and health care. This is the bare minimum with respect to the obligations of parties to a conflict under international humanitarian law. And while not all humanitarian crises and subsequent evaluations will involve armed conflict, there is a complex relationship between vulnerability to climate shocks, alongside fragility and heightened disaster risk. Rapid onset disasters further present specific protection risks for affected populations due to the breakdown in law and order and the potential for displacement triggered by such events.

ALNAP’s 2018 guide on evaluation of protection in humanitarian action was created as a companion to the Evaluation of Humanitarian Action guide, offering insights specific to protection. It provides guidance on navigating the complex cause-effect relationships that underlie protection risks, as well as the challenges of accessing and managing sensitive data. As the humanitarian sector increasingly adopts Artificial Intelligence for data analysis, careful attention will need to be paid to the ethical, governance, access concerns and risks associated with data bias.

With this background, humanitarian evaluation guidelines must better align and ensure a level of accountability to the IASC commitment to the centrality of protection. The centrality of protection cannot be satisfied by statements committing to ‘mainstream protection’ or relegating it as a cross-cutting issue only. In practice, the centrality of protection brings in and advances commitments to gender, inclusion, accountability to affected populations, and human rights.

• Making protection central to humanitarian evaluations requires careful consideration of three levels of action. The first is whether there is a relevant and appropriate overarching strategic approach to the crisis context and protection risks, e.g. ‘civilians simply need food and water’ will not cut it anymore.
• The second is to examine whether sufficient dedicated protection, prevention and responsive activities* relevant and appropriate to the crisis context were supported**.
• Finally, at the sectoral response level, it is necessary to assess whether necessary protection principles*** including but not limited to gender, inclusion, and accountability to affected populations were adequately integrated throughout the response activities.

Through adopting a three level of action approach, evaluators can operationally centre protection at the system or response level, individual agency or donor level, as well as sector specific evaluations.

Opportunity to learn lessons and deliver better

The update of ALNAP’s 2006 humanitarian evaluation guidelines presents a critical opportunity to reflect on the many hard lessons identified, though not necessarily learned, in the aid sector. It can further support greater coherence through alignment with current humanitarian norms and standards that have evolved considerably since 2006.

More concretely, aligning humanitarian evaluation standards with the centrality of protection and providing the necessary operational guidance have significant potential to improve aid effectiveness and the experiences of crisis-affected populations.