This blog post originally appeared on the Stanford Social Innovation Review website.
Nonprofits on the journey to using measurement as a continuous improvement tool will eventually need to decide whether to create a full-time position to lead their performance measurement function. According to research we’ve just completed with nonprofits of varying sizes and domains, most organizations that do so will prioritize the wrong skills in their search for a measurement director.
Nonprofits often assume the best person for the job will be a “data wonk”—someone who can execute complex analyses that will reveal powerful insights about their programs. While few would argue the value of such a skill set, what nonprofits most need in a first measurement director are consultative, interpersonal, and change management skills. The person in that position must do far more than crunch numbers: their most important job is often to lead a broad-scale organizational change in mindset and behavior so that everyone comes to see the value of using measurement for learning and improvement.
Horizons for Homeless Children, a $10-million early childhood development organization in Boston, hired its first measurement director in 2011, when the organization became more serious about measuring child and family outcomes. Though the need for a director was clear, the role was not easy to fill. “We struggled with the right job description,” said CEO Asa Fanelli. “We thought: ‘Should it be someone who has been doing [measurement] for years?’” However, as the hiring process progressed, it became clearer that strong interpersonal and strategic skills were critical characteristics. Eventual hire Nathan Hutto noted: “Especially when you’re starting up, it is more important to have someone who is good at designing measurement around the vision for the organization and building relationships with the staff, children, and families to ensure measurement is helpful to them. In my role, I bridge different departments and need to be able to speak to people at all different levels.”
Several organizations we spoke with hired the wrong measurement director the first time around. A senior leader in one of them said, “He was a classic researcher and statistician, with a great technical background but an inability to be flexible.” The measurement director wanted to use the most rigorous methods possible to evaluate impact, but the organization needed to focus first on improving its model, which called for a different kind of measurement. Within a year, he was replaced with an internal hire who truly understood the needs of the organization. Another individual provided the technical skills that the director lacked, and the arrangement has proven successful.
Indeed, our research found that nonprofits can successfully use contractors, additional hires, or even pro bono assistance from graduate students to access deep statistical capabilities, so long as measurement directors have the base-level analytical and quantitative skills needed. In our experience, the most important of these analytical skills are an ability to: frame and test hypotheses (for example, do youth who attend the program more frequently do better?), quickly execute basic data analyses (for example, compare test scores by youth attendance rates), and zero in on potential improvements (for example, what are the root causes behind low program attendance, and how do we deal with them)?
To address this full range of necessary skill sets, you need to search for your measurement director in a different way. First, broaden the role to recognize the intersection of measurement with organizational learning, strategic planning, and knowledge management. Second, consider a more diverse set of candidates beyond evaluators and academics, such as those in strategy or program roles inside and outside your organizations, who may possess strong cultural competence and change management experience. Third, during interviews, test a candidate’s experience in working directly with staff to influence the strategic direction of a program or initiative using measurement.
Over time, organizational measurement priorities may evolve. For example, you may want to begin measuring to improve results and to prove that program models work in order to attract the resources for replication. This task puts a much greater premium on statistics experience, writing and speaking skills, and credibility with the evaluation community—qualities which may be missing in your current staff, board, and outside advisors. Your measurement director may be able to evolve as well. But sometimes a new person in that role or another leadership position will be required to take the organization to the next level.
Our research into how nonprofits build internal capacity to measure and manage performance spotlighted some other important lessons—about the vital role of leadership commitment, the importance of incentives in nurturing a culture of continuous improvement, and why organizations may want to engage outside expertise early and often to support internal capacity. These lessons are explored in Bridgespan’s latest white paper "Building Capacity to Measure and Manage Performance," which profiles the performance measurement practices of organizations such as Roca, Youth Villages, and BRAC and includes job descriptions, interview questions, and job posting strategies for hiring a measurement director.
What lessons would you share about hiring your first measurement director?
