This blog post originally appeared on the Stanford Social Innovation Review website on November 13, 2013.
By Christina Triantaphyllis and Matt Forti
Americans spent $39 billion in private philanthropy on the developing world in 2010, and the United States remains the highest net donor of aid. Many of these dollars flow through billion-dollar global nongovernmental organizations (NGOs). But how effective is this spending? Too often, donors rely on readily available metrics, such as the percentage of dollars spent on overhead vs. programs, instead of considering true measures of impact and cost effectiveness. The widely held belief that low overhead indicates greater effectiveness runs deep in the social sector and has been well documented in articles such as Stanford Social Innovation Review’s “The Nonprofit Starvation Cycle” and Bridgespan’s “Stop Starving Scale,” and in founder and president of Charity Defense Council Dan Pallotta’s TED talk (which generated both praise and censure). But fixing the problem is far easier said than done.
Increased awareness has not necessarily translated into changed behavior for global NGOs. “The general climate in the United States is that the sole criterion for evaluating a charitable gift is overhead,” said Richard Stearns, CEO of World Vision US. “The right question to ask is, ‘What impact is the organization having per donated dollar?’ When we ask the wrong question [about overhead], we punish the organization that’s investing enough [in administration] to have real impact.” (A recent New York Times article on “How to Choose a Charity Wisely” demonstrates persistent challenges in shifting the conversation to metrics that matter.)
What stands in the way of philanthropists, nonprofits, and other stakeholders asking and answering the right question? In 2012, The Bridgespan Group surveyed two-dozen leaders of US-based global NGOs with budgets exceeding $100 million, all operating in 10 countries or more. The results revealed numerous barriers to obtaining true estimates of both cost and impact:
Fragmented measurement and evaluation (M&E) systems do not permit cross-country comparisons of programs and sites. Only 32 percent of global NGOs surveyed reported having uniform metrics or logic models to guide programs. Many cite a desire to do so without a clear funding source. One NGO leader was frustrated with funds that “do not cover longitudinal, cross-country M&E systems that would allow us to compare results in Liberia with those in Nicaragua or Costa Rica.”
Organizations under-invest in M&E activities that would enable impact-per-dollar program measurement. NGOs reported spending one-third as much on M&E as the 5 to 10 percent recommended by experts. In addition, nearly half of NGOs were unable to report total measurement costs, either because staff time spent on measurement was not documented or because costs were rolled into other cost categories.
Sufficient investment in M&E activities goes beyond randomized control trials to prove isolated outcomes. It also takes systematic use of cost and impact data to inform organizational strategy for scaling effective and efficient programs. “It is so much easier to measure our financial rather than our social bottom line,” explained an NGO leader. “Until we can figure that out, we will be stuck with financial measures of effectiveness.”
A lack of unified financial systems hinders real-time, true-cost data. NGOs rely on the generosity of funders who, for the most part, restrict their investments to specific programs, resulting in a patchwork of fragmented, short-term engagements across countries and continents. Thus, NGOs do not understand all costs associated with delivering impact, and instead focus on program or country vs. headquarters expenses.
Only 36 percent of global NGOs surveyed reported having a unified financial system across all countries, programs, and offices that would allow efficient reporting of true costs.
“We struggle with using several different financial software packages for different projects, and significant (and as yet unmet) financial resources will be needed to create a unified system,” said another NGO leader.
Cost-per-impact is certainly not for all organizations, especially those focused on advocacy, disaster relief, or holistic community development, where it is extremely difficult to summarize impact by a single outcome aligned against relevant costs. For others, there are a number of ways to shift the conversation to focus on solutions:
- Funders should be true partners and reward performance based on a cost-per-impact metric. As one NGO leader put it, “No one has made the case that you need strong measurement and financial systems to get to impact and cost per result. Everyone wants to get through with Band-Aids and chewing gum.” Global NGOs need both patient funders and value-added collaborators in the journey toward cost-per-impact measures.
- Board members and other stakeholders should serve as champions for these efforts, pushing for longer-term investments. Global NGO leaders pointed to the pervasive cost-cutting mentality that many business-minded board members bring. “The knee-jerk reaction is to point to a well in Bangladesh that will not be built or to the number of children not vaccinated instead of considering what will help us determine how to serve 100 times that many children in the future," said one leader. Another said, “It took opening up the hood and showing the dirt … showing the board that we have five ledger systems and too many people working on this.”
- Rankings and ratings systems need to emphasize measurement of effectiveness. While ratings systems such as Charity Navigator and GuideStar continue to focus on overhead rates, accountability, and transparency, newer sites such as GiveWell and 3ie are putting cost-per-impact on the map in global development—a positive trend that needs support from NGOs, funders, and the public to change the system.
We have seen a rise in media attention for organizations like the Against Malaria Foundation that can capture cost-per-impact and cost-effectiveness metrics. But we have a long way to go—major challenges include coming up with uniform ways of measuring cost and outcomes to arrive at comparable measures. NGOs with multiple programs on several continents face a tremendous amount of complexity in developing these uniform measures and implementing systems that efficiently collect and synthesize data.
But with the support of funders and board members, NGOs could start by focusing on a few core programs in a subset of countries where measuring cost-per-impact may be more manageable, and then building on the knowledge gained from early experiences. We also need a coordinated, supportive movement toward ratings that are based on results for beneficiaries. All this will take honest dialogue between NGOs and their funders to move the conversation to cost-per-impact.
Christina Triantaphyllis (@cltrianta) is a consultant in Bridgespan’s Boston office.
Matthew Forti is director, One Acre Fund USA, where he coordinates all US functions and oversees performance measurement for One Acre Fund, a nonprofit that assists over 135,000 smallholder farming families in East Africa to double their farm profits and eliminate persistent hunger. Matt is also advisor to the Performance Measurement Capability Area at the Bridgespan Group (@BridgespanGroup), an advisory firm to mission-driven leaders and organizations that supports the design of performance measurement systems for continuous learning and improvement.
