I recently read a fascinating book: Poor Economics. Like many global development aficionados, I marveled at the authors’ wizardry in carving out a pragmatic middle ground between the pro- and anti-aid camps with their simple yet rarely practiced prescription to let the data decide. I’ve also read nearly a dozen reviews of the book that aptly describe the authors’ recommendations for poverty alleviation, but was disappointed to find not one that touched on the book’s implications for measurement itself. Here is my take:

• Rigorous measurement can be a powerful input into decision making at all levels: The vast majority of illustrations in Poor Economics were of simple design choices that led to dramatic improvements in outcomes. For example, offering Zambian women a family-planning voucher without their husbands present, rather than in their presence, reduced the likelihood of an unwanted birth a year later by 57 percent (see pages 116-7 of the book). Providing a tiny incentive (less than $2 per family in dried beans) increased immunization rates in an Indian village more than sevenfold (pages 62-3).
  
  These and other examples make me wonder: What if every nonprofit employed a director of quality whose job was to elicit quality improvement ideas from those who know programs the best—front-line staff and beneficiaries—and then set up mini-experiments to test the ideas and implement the winners?
  
  Sam Walton, founder of Walmart, notes in his autobiography: “Our best ideas usually do come from the folks in the stores. Period." I doubt many nonprofits would say the same. And so, while the authors and the institution they represent (Jameel Poverty Action Lab) are busy scaling evidence-based programs and policies, I also hope they will use their growing heft to unleash a quality revolution by scaling the supply of, and generating greater demand for, ‘measurement gurus’ at global development nonprofits.

• The best organizations never stop measuring: The book debunks the popular evaluation myth that you launch a model, refine it, test it once through a randomized control trial, ideally receive a passing grade, and then take your report card to funders for the resources you need to scale. For example, the authors profile the educational NGO Pratham, whose reading and arithmetic program was deemed highly effective by a rigorous impact evaluation in 2000. Not content to stop there, the organization has contracted for nearly every new ‘edition’ of its program to be evaluated, testing innovations to increase reach, deepen impact, and reduce cost (pages 84-6). One such innovation was to shift responsibility for program implementation to community volunteers; another was to test the success rate of government teachers in its remedial education program. I doubt every innovation Pratham tested was effective and ultimately integrated, but I am sure its commitment to innovation and evaluation is a key factor in its ongoing success.

• Results can be very context dependent: ‘What works’ in one location, with one beneficiary group, at one point in time simply may not work when one or more of these variables are changed. Context, it seems, is king when it comes to effective social programs. In Kenya, a small scholarship offered for the next year of schooling to girls who scored in the top 15 percent on an exam resulted in the girls doing much better (pages 99-100), but in the US, rewarding children for achieving long-term goals (such as getting high grades) was not successful (page 100). Similarly, a conditional cash transfer program, OPORTUNIDADES, increased secondary school enrollment in Mexico (page 79), but when replicated in New York City, achieved only modest gains and was terminated early. These findings should caution us against declaring broad victory when a program works in one setting, no matter how great the effects.

• Defining and testing a full ‘theory of change’ really does matter: The authors describe the historical reluctance of most microfinance institutions (MFIs) to gather rigorous evidence to prove their claims that microfinance transforms people’s lives (pages 170-2). Recent evaluations have shown microcredit is a useful financial product, but despite powerful anecdotes of success, not a poverty alleviator for the average entrepreneur. My hypothesis: MFIs could have learned this themselves years ago if they had articulated a full theory of change (describing exactly how their resources and program activities would result in measurable outcomes) and sought to test the link between the key intermediate outcome (high repayment rates) and ultimate outcome (improved quality of life).

I hope this book encourages nonprofit organizations around the world to reflect on how they are using measurement to test their theories of change, innovate their programs, and improve their services.

I invite other readers of the book to share their takeaways and reflections!