When disaster strikes, non-governmental organizations are among the first on the scene. But recent interventions by NGO's, notably in Rwanda after the 1994 genocide and Indonesia following the 2004 tsunami, have called attention to the need for long-term monitoring of their impact on those they claim to be helping.
When disaster strikes, non-governmental organizations (NGOs) are among the first on the scene. The United Nations estimates that there are now more than 37,000 international NGOs, with major donors relying on them more and more.
Inevitably, there are problems. Both the Rwandan genocide of 1994 and the 2004 Indian Ocean tsunami saw chaotic competition among NGOs. Yet there have also been landmark successes. More than 1,400 NGOs operating in 90 countries helped to get 123 countries to ratify the treaty banning landmines. But the sheer scale of the disaster relief “industry” – plus the longer-term development efforts of NGOs – is raising serious concerns about how to measure their performance.
Flexibility allows NGOs to be innovative in ways that organizations like the UN often cannot. But there are few international rules on what an NGO actually is, and the lack of control can lead to unpredictable consequences. In Chad recently, the French NGO L’Arche de Zoé tried to smuggle children out of the country without obtaining permission from either parents or the government.
Among the questions being asked by NGOs, the UN, and national donors is how to prevent the recurrence of past mistakes. The wake-up call for most NGOs came after the Rwandan genocide, when hundreds of small organizations tried to set up ad hoc operations in refugee camps in the Democratic Republic of Congo and Tanzania. Some camps turned into staging posts for armed factions. In the ensuing chaos, more than 50,000 refugees died from cholera.
There was also mayhem following the Indian Ocean tsunami. At one point, more than 400 NGOs were on the ground in Aceh, Indonesia, competing for resources, personnel, and funding. Many of the lessons learned in Rwanda were forgotten or ignored as smaller NGOs with little or no experience in dealing with disasters caused much of the confusion.
The situation in Indonesia led the UN to adopt a new “cluster” system to improve coordination. And, after a review of the Rwanda debacle, 400 NGOs and UN organizations working in 80 countries got together in the Sphere Project to develop a common humanitarian mandate and handbook of standards outlining the minimum performance required of any NGO working in a disaster zone.
Access every new PS commentary, our entire On Point suite of subscriber-exclusive content – including Longer Reads, Insider Interviews, Big Picture/Big Question, and Say More – and the full PS archive.
Subscribe Now
As the number of post-intervention reviews increases, a rudimentary framework for evaluating the impact of NGOs has appeared. Rather than simply looking at project inputs and outputs, the emphasis has turned towards measuring the overall impact of an operation.
The idea is to find out if the lives of the people on the receiving end were changed for the better in any sustained way. More and more donors are also insisting that NGOs provide measurable proof that they make a difference.
That sounds fine in theory, but in practice there are drawbacks. By demanding quantifiable results, donors may force program managers to choose easily achieved targets over less measurable actions that accord with sound humanitarian principles. Or reporting about aid programs may be skewed to keep donor funds flowing. The greatest danger is that humanitarian relief will be tailored to meet donors’ demands, rather than actual needs.
Until recently, the record on evaluating responses to humanitarian emergencies has been patchy at best. CARE, as both a relief and development agency, can take a long-term approach to disasters, matching emergency relief with a rehabilitation and recovery phase. But this is not an option for NGOs that focus only on emergency responses.
Once their allotted time is up – or their funds run out – they tend to pack up and leave. Even for NGOs that stick around, determining their relief efforts’ impact in the middle of a crisis is difficult, if not impossible. Emergencies are chaotic: staff and resources are stretched, the local population is very unlikely to be able to provide meaningful feedback, and pre-crisis baseline data are largely unavailable, so comparisons are complicated. Moreover, all too often, events move too quickly to be measured accurately. And, until recently, donors who were willing to pay for relief were less likely to finance follow-up evaluations.
As a result, emergency relief evaluations often rely on little more than guesswork and assumptions. A 2004 report by the Humanitarian Policy Group cited a survey carried out in Ethiopia after UN agencies said that humanitarian efforts had averted widespread famine in 2000. The claim sounded credible until the subsequent survey showed that the area’s crude mortality rate had actually risen to six times the normal base rate. Most of the deaths were from communicable diseases, which malnourished people may well have contracted after crowding into feeding centers.
The HPG therefore recommended long-term monitoring of future humanitarian responses, and said that success or failure should be judged in a broad context rather than by a narrow focus on a specific project. Many people who survive an earthquake or a flood, for instance, may soon face another crisis if the disaster also destroys their only means of earning a living.
New and more sophisticated analytical tools are needed to understand these long-term effects, along with sufficient training to ensure that new methods are applied properly in the field. A recent innovation has been the Coping Strategy Index, devised by the World Food Program and CARE, which analyzes how people cope with short-term food crises while also taking into account their future vulnerability to hunger.
NGOs do the lion’s share of the world’s humanitarian work, and some mistakes are inevitable. But as we deepen our experience of humanitarian relief and development, we must learn the lessons of the past and understand how much more there is to know.
To have unlimited access to our content including in-depth commentaries, book reviews, exclusive interviews, PS OnPoint and PS The Big Picture, please subscribe
In the United States and Europe, immigration tends to divide people into opposing camps: those who claim that newcomers undermine economic opportunity and security for locals, and those who argue that welcoming migrants and refugees is a moral and economic imperative. How should one make sense of a debate that is often based on motivated reasoning, with emotion and underlying biases affecting the selection and interpretation of evidence?
To maintain its position as a global rule-maker and avoid becoming a rule-taker, the United States must use the coming year to promote clarity and confidence in the digital-asset market. The US faces three potential paths to maintaining its competitive edge in crypto: regulation, legislation, and designation.
urges policymakers to take decisive action and set new rules for the industry in 2024.
The World Trade Organization’s most recent ministerial conference concluded with a few positive outcomes demonstrating that meaningful change is possible, though there were some disappointments. A successful agenda of reforms will require more members – particularly emerging markets and developing economies – to take the lead.
writes that meaningful change will come only when members other than the US help steer the organization.
When disaster strikes, non-governmental organizations (NGOs) are among the first on the scene. The United Nations estimates that there are now more than 37,000 international NGOs, with major donors relying on them more and more.
Inevitably, there are problems. Both the Rwandan genocide of 1994 and the 2004 Indian Ocean tsunami saw chaotic competition among NGOs. Yet there have also been landmark successes. More than 1,400 NGOs operating in 90 countries helped to get 123 countries to ratify the treaty banning landmines. But the sheer scale of the disaster relief “industry” – plus the longer-term development efforts of NGOs – is raising serious concerns about how to measure their performance.
Flexibility allows NGOs to be innovative in ways that organizations like the UN often cannot. But there are few international rules on what an NGO actually is, and the lack of control can lead to unpredictable consequences. In Chad recently, the French NGO L’Arche de Zoé tried to smuggle children out of the country without obtaining permission from either parents or the government.
Among the questions being asked by NGOs, the UN, and national donors is how to prevent the recurrence of past mistakes. The wake-up call for most NGOs came after the Rwandan genocide, when hundreds of small organizations tried to set up ad hoc operations in refugee camps in the Democratic Republic of Congo and Tanzania. Some camps turned into staging posts for armed factions. In the ensuing chaos, more than 50,000 refugees died from cholera.
There was also mayhem following the Indian Ocean tsunami. At one point, more than 400 NGOs were on the ground in Aceh, Indonesia, competing for resources, personnel, and funding. Many of the lessons learned in Rwanda were forgotten or ignored as smaller NGOs with little or no experience in dealing with disasters caused much of the confusion.
The situation in Indonesia led the UN to adopt a new “cluster” system to improve coordination. And, after a review of the Rwanda debacle, 400 NGOs and UN organizations working in 80 countries got together in the Sphere Project to develop a common humanitarian mandate and handbook of standards outlining the minimum performance required of any NGO working in a disaster zone.
Subscribe to PS Digital
Access every new PS commentary, our entire On Point suite of subscriber-exclusive content – including Longer Reads, Insider Interviews, Big Picture/Big Question, and Say More – and the full PS archive.
Subscribe Now
As the number of post-intervention reviews increases, a rudimentary framework for evaluating the impact of NGOs has appeared. Rather than simply looking at project inputs and outputs, the emphasis has turned towards measuring the overall impact of an operation.
The idea is to find out if the lives of the people on the receiving end were changed for the better in any sustained way. More and more donors are also insisting that NGOs provide measurable proof that they make a difference.
That sounds fine in theory, but in practice there are drawbacks. By demanding quantifiable results, donors may force program managers to choose easily achieved targets over less measurable actions that accord with sound humanitarian principles. Or reporting about aid programs may be skewed to keep donor funds flowing. The greatest danger is that humanitarian relief will be tailored to meet donors’ demands, rather than actual needs.
Until recently, the record on evaluating responses to humanitarian emergencies has been patchy at best. CARE, as both a relief and development agency, can take a long-term approach to disasters, matching emergency relief with a rehabilitation and recovery phase. But this is not an option for NGOs that focus only on emergency responses.
Once their allotted time is up – or their funds run out – they tend to pack up and leave. Even for NGOs that stick around, determining their relief efforts’ impact in the middle of a crisis is difficult, if not impossible. Emergencies are chaotic: staff and resources are stretched, the local population is very unlikely to be able to provide meaningful feedback, and pre-crisis baseline data are largely unavailable, so comparisons are complicated. Moreover, all too often, events move too quickly to be measured accurately. And, until recently, donors who were willing to pay for relief were less likely to finance follow-up evaluations.
As a result, emergency relief evaluations often rely on little more than guesswork and assumptions. A 2004 report by the Humanitarian Policy Group cited a survey carried out in Ethiopia after UN agencies said that humanitarian efforts had averted widespread famine in 2000. The claim sounded credible until the subsequent survey showed that the area’s crude mortality rate had actually risen to six times the normal base rate. Most of the deaths were from communicable diseases, which malnourished people may well have contracted after crowding into feeding centers.
The HPG therefore recommended long-term monitoring of future humanitarian responses, and said that success or failure should be judged in a broad context rather than by a narrow focus on a specific project. Many people who survive an earthquake or a flood, for instance, may soon face another crisis if the disaster also destroys their only means of earning a living.
New and more sophisticated analytical tools are needed to understand these long-term effects, along with sufficient training to ensure that new methods are applied properly in the field. A recent innovation has been the Coping Strategy Index, devised by the World Food Program and CARE, which analyzes how people cope with short-term food crises while also taking into account their future vulnerability to hunger.
NGOs do the lion’s share of the world’s humanitarian work, and some mistakes are inevitable. But as we deepen our experience of humanitarian relief and development, we must learn the lessons of the past and understand how much more there is to know.