This article originally appeared in the October 29, 2009, issue of The Chronicle of Philanthropy.

While all philanthropists strive to maximize the impact of their money, far too few feel they are succeeding. It's not for lack of good intentions.

Rather, excellence in philanthropy is difficult to measure.

One can point to individual success stories in an annual report, but no market pressures force foundations to track achievements and failures across their entire portfolio of grantees. To make a deep and broad difference, foundations need to force themselves to seek consistently high standards.

Why is this so hard? Because philanthropists hold the purse strings, nearly everyone has a vested interest in telling them they are doing a great job. Few have designated "truth-tellers" to inform them when their approaches are flawed: when generic research reports won't generate insights that can actually enable better investment decisions, or when their demands for grantee performance data require expensive information-technology systems — systems that few charities can afford unless a foundation pays for them. Thus, the true voices of the grant recipients (and, worse, the voices of the beneficiaries) are often missing.

In today's turbulent times, when effective philanthropy matters more than ever, philanthropists know they need to change, but many do not know how. A few pioneering grant makers are explicitly incorporating continuous self-improvement tools. They realize that to maximize their impact, they need to commit to getting clear about what success looks like for their investments, get real about what it takes to make change happen, and get better over time.

As we write in November's Harvard Business Review, to get clear, thoughtful philanthropic investors typically apply two tests to their decision making. First, they don't over-rely on evidence too early in the process.

They typically ask, "What's in and what's out?" when setting boundaries and trying to define success. Most investors want evidence to inform this decision, but some questions don't lend themselves to analysis — whether it's "better" to support clean-air projects or early education, for example.

While data can support numerous promising paths, the truth is that philanthropy is inherently personal and values-driven, even in professional institutions. In the absence of objectively "right" answers, donors must discuss their underlying beliefs, values, and assumptions.

On the flip side, their second test is to ensure that they don't over-rely on values and beliefs too late in the process.

Personal values must be balanced with evidence and data, or else the best analysis can get lost in unhealthy, complex, and inefficient decision-making processes. Once certain boundaries have been established, data can help decision makers define success, translating "what matters most" into concrete, realistic goals.

Getting real means asking, and answering, hard questions: Are the available resources sufficient to match our ambition? Who else has created innovative solutions to this problem — foundations, nonprofit groups, government, companies — and what's known about what doesn't work? How strong are the organizations already working on the cause and how much can they deliver? How capable is our foundation (our staff, our processes) to deliver? How will we know if our strategy is working?

Honest answers can help donors avoid the common mistakes of falling in love with their own ideas, or failing to challenge their own theories about how change works or to hold themselves accountable to the standards of clarity and logic that they require of grantees.

Philanthropic approaches vary: Some philanthropists, smaller ones in particular, choose to make a series of grants to help great projects achieve their goals but rarely design strategies about how to promote broader social change. Others, typically larger investors, hope that their portfolio of investments will collectively "add up" to more than the sum of the individual grants. Whatever the approach, decision makers need to be realistic about the resources and time required to make the desired changes occur. And they need candid feedback from others working on the same cause both in crafting and reality-checking their strategies.

Finally getting better requires discipline. It's nearly impossible to improve over time without critical feedback; one philanthropist likened it to "learning to tie your shoes in the dark." But dollars for measuring performance are scant compared with program money going out the door — a major point of under-investment in a foundation's own capacity to learn and improve.

If philanthropists step back with a clear-eyed, data-driven look at the portfolio and its strategy — its risks, its potential — unprecedented opportunities may bubble up to make a difference. Perhaps it means a new collaboration among donors or a new way to work with government. Perhaps it means doubling an investment in key grantees at a critical time.

The experience of the Bill & Melinda Gates Foundation's Pacific Northwest initiative grant-making team illustrates one thoughtful approach to walking this talk: In this case, the foundation's effort to combat family homelessness in the Puget Sound region of Washington State.

From 2000 to 2007, the foundation's Sound Families Initiative, a $40-million public-private partnership, provided support for almost 1,500 affordable housing units with on-site support services. Its investment prompted another $200-million from other sources. Nevertheless, the number of homeless families in the region didn't drop significantly. The grant-making team began a strategic analysis to understand why.

The initial findings were promising: 75 percent of families served had successfully completed the transitional housing program, and 89 percent of these had moved into permanent housing. Of those, more than 90 percent were still in permanent housing one year later. But what of the 25 percent who didn't successfully complete the program?

Further investigation showed that they were more likely to face multiple barriers, including mental illness, substance abuse, or domestic violence, and therefore needed more intensive support. The foundation realized that one-size-fits-all services weren't an optimal use of scarce resources: Some families were getting less support than they needed, while others were being offered relatively expensive services to help overcome barriers they didn't necessarily face.

The team had to make a choice: to define success as building housing units (which it had done well) or as ending homelessness (which it had done less well). It chose to recommit to its mission of ending homelessness and changed its strategy and grant making to promote more focused services to families according to their needs.

The foundation did the hard work of holding itself accountable for results for making sure the work really added up to more than the sum of individual grants.

Foundation officials defined clear goals, rigorously collected and analyzed data from grantees and other organizations working on housing issues about what was and wasn't working, reflected on the barriers to making change happen, and dramatically changed their approach in response to what they had learned.

Above all, they had to acknowledge, to themselves and others, not only what they had learned but also what as yet remained unknown.

In our experience, developing a clear strategy requires the humility to listen to the field, the clarity and discipline to question one's own assumptions, and the willingness to say "no" to enticing options.

By embracing a similar approach for getting real to get better, donors can improve their chances of maximizing their impact at a time when they are most urgently needed.

Copyright © 2009 The Chronicle of Philanthropy