In my last post (“Measurement that Benefits the Measured”), I argued that social sector organizations could improve their impact by engaging the individuals, families, or communities they seek to benefit. I shared three forms of engagement – voice, co-creation, and ownership – that organizations might undertake to enhance the role these constituents play in their own progress.
In this post, I want to look at constituent engagement in measurement from another angle – which is how organizations decide what outcomes they seek, and therefore what and how to measure. Put another way – how would measurement look different if the people an organization serves had a say in the design of a performance measurement system?
I’ve recently had the opportunity to explore this topic with three low-income women, each of whom has participated in the programs of a different nonprofit organization, all of which provide intensive services to constituents for one to two years. Here is how these women would measure their own success:
Candice: “Ask me what my goals were in joining and whether or not I have met them.”
Most nonprofits develop theories of change that identify the few outcomes they want to hold themselves accountable to.This makes perfect sense in a world where funding flows are typically attached to, and program evaluations judge success by, achieving a common outcome for all constituents. But as Candice’s statement reminds us, that isn’t how constituents see the world. They join programs for a variety of reasons, and they judge success based on whether or not their own goals are reached.
Yajaira: “Ask me if I’m making better decisions for me and my family now that I’ve been out of the program a few years.”
Driven by cost considerations, most nonprofits measure outcomes only while a constituent is in their programs. But Yajaira’s statement reminds us that most constituents aren’t looking for short-term fixes – they want true, root cause change in how they live their lives. Often this can only be understood by measuring success after (sometimes, long after) constituents have completed a program.
Adebanke: “Ask me if I would have paid full-price for the program now that I’ve experienced it.”
Most nonprofit programs are heavily or fully subsidized, so organizations focus on catering to the needs and desires of their funders, not their constituents. But Adebanke’s statement reminds us there is insight to be gained by asking constituents a few simple questions that gauge the value they capture from a program – whether it is willingness to pay, probability of referring friends or family members, or otherwise. The nonprofit sector’s relatively slow embrace of basic consumer feedback systems suggests there is a ways to go before the perspectives of constituents are regularly known and integrated.
Collectively, these three responses suggest constituents would indeed measure in very different ways than typical nonprofit organizations choose to. So how can these approaches be reconciled?
For nonprofits serious about continuous improvement, measuring the degree to which constituents achieve their own goals (Candice), the sustainability of impact after a program ends (Yajaira), and the value constituents perceive from programs (Adebanke) are all critical. These aren’t the only measures you will need; you will also want to specify and measure across all of your constituents a common set of input, output, and outcome measures that best test the validity of your theory of change. We believe both types of measurement matter for continuous improvement, and continuous improvement is the best way to refine your programs and increase the odds you will ultimately “prove” a program’s worth in the eyes of funders, policymakers, and peers.
What’s your take? How has your organization integrated constituents' views of success into how it measures?
