When a philanthropic organisation funds a new initiative, it’s often hard for anyone involved to discern how the project is working out, much less improve it. Jeri Eckhart- Queenan and Matthew Forti believe that the rigour of corporate performance measurement should also be the standard in philanthropy.
As those who lead corporate philanthropies and serve on boards of non-governmental organisations (NGOs) well know, measurement is a hot topic these days. Funders increasingly assess grantee results, demanding to know what is being achieved with their investment dollars. And charities and NGOs increasingly find themselves measured, rated and categorised based on their purported effectiveness and efficiency.
Yet one particularly valuable form of measurement is still sorely underutilised. Performance measurement — measuring the results of initiatives in an ongoing way to learn and improve — is highly valued and widely practised in the private sector and even in parts of the public sector, but few third-sector organisations (non-profits or NGOs) understand its benefits, much less possess the know-how to implement an effective performance-measurement system. And funders, driven in part by a bias in the field to use measurement as a tool to evaluate impact only at the end of an initiative, have not generally encouraged grantees to engage in ongoing performance measurement, nor supported their efforts to do so.
Just as corporations do not wait for audited financial statements to manage profitability, corporate philanthropies and their grantees should not wait for impact evaluations to manage performance. A well-run performance measurement system can tangibly improve results for the people an organisation is trying to help or the cause it supports. It can help leaders make better decisions about allocating scarce resources. It can support rapid innovation. It can also lower the cost of learning.
10,000 Women, Five Lessons
We have learned a great deal from the power of performance measurement used with 10,000 Women, a Goldman Sachs philanthropic initiative implemented in collaboration with leading business schools, including London Business School. 10,000 Women is a $100 million, multi-year investment to provide underserved women with a practical business education and the support services they need to grow their businesses and create jobs for their communities. The programme is delivered in 22 countries including India, China, Brazil, Nigeria, Turkey, Afghanistan and Rwanda. Early results show that more than 70 per cent of graduates increase revenues, and 50 per cent hire additional employees within six months of graduation. In working on this project and studying others, five lessons emerged.
Begin with the end in mind
Before thinking about metrics or systems, it is essential for organisational leaders to be very clear about the specific results for which they will hold themselves accountable. While it sounds straightforward, this level of rigour is rare in corporate philanthropy.
The main question to address: can the senior team — whether the corporate philanthropist or grantee — articulate, in one sentence, whom their organisation serves and what change they seek to create? Until an organisation’s senior leaders and funders are in sync on these issues and have committed to allocating the necessary resources, there is no point in measuring anything. When trying to assess how they are doing, the leaders of many large, sophisticated organisations often find themselves overwhelmed by an endless flow of data generated by colleagues who lack clarity on the definition of success.
With 10,000 Women, the end result was clear from the outset. As Dina Habib Powell, Global Head of Corporate Engagement, explained, “Economic research showed that investing in women would drive economic growth and subsequently lead to healthier, safer, better educated families and communities. But to have this ‘multiplier effect,’ we first had to be sure that we were creating growth. So our key measure of success was going to be that businesses were increasing revenue and creating jobs.”
Specifically, the end result or intended impact was ultimately defined as: “Over five years, 10,000 underserved women around the world will expand their businesses and create jobs.” Being this specific (about target population, time frame and desired outcome) was a critical first step in getting measurement right. That specificity also helped those in charge of philanthropy to commit the money, time and capacity needed to make tangible progress towards its goal.
Anchor measurement in your theory of change
With a clearly defined intent for impact, an organisation’s leaders should ask, “How, exactly, do we achieve results?” Answering this question means describing the organisation’s theory of change or specifying the complete set of programmes, activities, organisational capabilities and relationships that are required to achieve the outcomes the organisation will hold itself accountable for.
Once this theory is explicit and agreed upon, it is much easier to decide what information to collect, because the organisation can more easily define the inputs (resources needed), outputs (activities) and outcomes (the results from those activities). It’s also easier to avoid the temptation to collect lots of information just because it’s interesting; the best performance measurement systems target data that will inform specific decisions.
10,000 Women translated its theory of change into a logic model that allows for metrics, tools and timing to be anchored explicitly in what the initiative is trying to achieve. As Marcele Carneiro Gama Viana, its project manager at Brazilian university, Fundação Dom Cabral, put it: “We’ve been able to learn quickly what is working and where our Scholars [participants] need more help. For example, we were very surprised to learn that 90 per cent or more of the Scholars in our first cohort had not given formal feedback to their employees over the past 12 months. We knew we needed to revise our human resources module to ensure a focus on this critical issue. The measurement has helped us make quick adjustments like this, so the next cohort can benefit right away.”
Create a culture of measurement
How can a third-sector organisation create a culture of measurement? The way 10,000 Women has been managed, two critical factors should be stressed.
- Make sure you have the commitment of top leaders. Goldman Sachs’ leaders publicly committed at the launch of 10,000 Women that they would measure the success of the initiative to ensure they were making a difference in the lives of women business owners. Doing so set an important expectation that all staff and partners would need to work together to embrace measurement.
- Plan opportunities to review data, learn and improve. The initiative also developed a formal, disciplined approach to learning and data-driven decision-making. For example, consider its formalised ‘learning forums’, which include leadership academies (annual in-person conferences), an online portal (to post documents and hold virtual discussions), issue-specific task forces and quarterly learning calls. These forums provided an opportunity to reflect on and interpret information from the measurement system, resulting in recommended improvements to the programme. Moreover, internal quarterly portfolio reviews provide opportunities for the initiative’s leadership to decide which recommendations to adopt, how to implement and monitor them and what resources will be needed.
Ensure that all contributors benefit
Regardless of the organisation, most of us have experienced a ‘system solution’ that was imposed upon us without our input. This occurs when someone in top management unilaterally orders a new change, a new process, a new programme to be adopted throughout the organisation. Such systems fail, despite well-intentioned efforts by smart people.
In the social sector, this problem of top-down design surfaces when data systems are designed by and for only some stakeholders. The result is beneficiaries who fail to respond to surveys, staff who don’t fully input the data they are supposed to, grantees who provide incomplete information and funders who collect, but don’t read, reports. Measurement only works when those who contribute to and use the system — as data providers, data collectors, analysts, overseers — benefit from the system.
Camfed International, a 10,000 Women partner that has financed access to education and provided business training and leadership development for more than one million girls and young women in rural Africa, designs its measurement systems to ensure stakeholders at all levels — the families, community, district and country — benefit from the data. In doing so, Camfed has empowered its clients and also influenced policy that affects the hundreds of thousands of women and girls it does not directly assist.
As Camfed Zambia’s Executive Director Barbara Chilangwa (the country’s former Permanent Secretary for Education) explained, “When we enter a new community, we first work with the planning officer to understand what local policy changes they want to make and how our data can support their efforts. We then align our measures to ensure Camfed data is directly comparable to government data and even ask the officials to help us design our instruments and train our enumerators. Then, as data comes in, they see the difference Camfed makes and they are armed with the data to institute policy changes.” For example, Camfed often helps local officials implement preexisting (but often ignored) policies against corporal punishment by showing how child protection leads to improved attendance and performance in Camfed schools.
Get better at measurement over time
With experience, an organisation can use measurement to improve the measurement system itself. This might manifest in less expensive data collection, zeroing in on a smaller set of vital measures or more rigorous analyses — all of which can contribute directly to improved results for beneficiaries.
10,000 Women has seen dramatic improvement in its measurement system. Data was originally housed, transmitted and analyzed through somewhat cumbersome spreadsheets; but roughly two years into the initiative, an IT system was developed that has dramatically lowered the cost of inputting data and accelerated analysis and learning. The philanthropic leaders also invested in hiring and training local measurement staff at each partner university; the additional capacity resulted in higher response rates, improved data quality and the use of data for programme improvement at the local level. Currently, as the initiative has grown and participant survey responses have reached critical mass, leaders are conducting sophisticated analyses to create a simplified set of metrics for future participants.
In this regard, Camfed provides another example. According to the organisation’s Director of Impact, Laurie Zivetz, Camfed has lowered its measurement-related costs in recent years by equipping data collectors with handheld computers and, in some cases, mobile phones. In communities where these technologies are being used, information about payments that Camfed has made on behalf of its clients, which used to be collected quarterly, is now being collected and conveyed back to the organisation’s managers weekly or daily.
Funders’ critical role
Funders can and should play a critical and collaborative role in utilising performance measurement. They can encourage grantees to get clear about the impact they intend to create and to develop a rigorous theory of change for getting there. Next, they can provide grantees with the resources they need to develop an internal measurement capacity, signaling a recognition that measurement should occur throughout a programme’s life cycle, not only at the end.
Funders can also create open environments for grantees to share their results, good or bad, in the spirit of mutual learning. They can facilitate ‘shared measurement systems’ in which grantees delivering similar activities and/or working with similar beneficiaries agree to share metrics, data systems or both.
Finally, corporate philanthropists, well versed in the practice of performance measurement, can lend their expertise and time, as well as their funds, to help grantees build and manage effective systems. As Powell puts it, “We decided at the outset to make a strong investment in measurement given how committed we were to impact. Seeing how quickly we’ve been able to learn and adjust really validates this decision. With stronger programmes, the women leaders we work with are creating more jobs and greater change in their communities.”
