Impact: The Journey from Measurement to Meaning

Impact-01

Was there ever a golden age when nearly everyone thought charities were doing a good job? Yes, some demographers say. The Civic generation (born 1945 or earlier) were inclined to trust charities, at least the ones with big names or visible work, without questioning.

But Civics are no longer the majority in our population. Boomers, Gen Y, Gen X and Millennials challenge all kinds of authority, including charities. And when those demographics reach for their chequebooks, disburse government dollars, determine foundation grants or report through the media, they no longer assume that charities are effective. They want proof.

As with any emerging idea, some of the new demands and assumptions about impact or effectiveness are misguided. The charity with the lowest operating expense, the cheapest cost per dollar raised, the tiniest staff budget or even the most people served is not automatically the most effective in its class. And if it is, its impact is almost never built on the characteristics I’ve just mentioned.

Is a heavy TV a better TV ?

Imagine Canada’s chief economist Brian Emmett points out (with a little help from marketing guru Seth Godin) that when we approach impact measurement, we risk “select[ing] priorities that are more easily measurable and perhaps more easily fundable.” That’s as helpful as weighing a TV set to evaluate its clarity.

In the meantime, efforts to measure impact seem marked by misguided effort, irrelevant statistics and ambiguous, unhelpful conclusions. Here’s an excerpt from Evaluation’s Next Generation, Srik Gopalakrishnan’s review of the U.S. foundation sector for the Stanford Social Innovation Review:

A recent survey on nonprofits and their relationship with data found that only 50% of nonprofits are tracking any kind of outcome data. Foundations are no different. A benchmarking study of evaluations in foundations found that only 38% of foundations reported, “Management regularly models the use of evaluation in its own decision making.” The authors of the study go on to suggest that this appears to be less about the willingness of foundations to use evaluation, and more about the poor fit between the decisions they need to make, the evaluation questions they ask, and the data that nonprofits produce. In other words, evaluation isn’t giving people what they need.

But there are hopeful signs. More charities recognize that they should be eager to demonstrate impact in a way that reaches beyond bodies in programs. More donors and funders want to give wisely and invest time in the research, thought and site visits that lift their own sights beyond numbers served.

Impact measurement done well and promoted unceasingly could actually counteract the meaningless, sometimes harmful measures that some agencies and members of the public use. The right questions are not easy to define, let alone answer. They lead us into mysteries such as long-term hunger alleviation, improved life expectancy, environmental impact, sustainability, changes in family and community life, and lifestyle-related diseases – trends that can take years, sometimes generations, to track.

In Canada, some organizations and funders have recognized that the most meaningful impact, the most sustainable systemic changes, are a collective responsibility. That leads to even more questions! But at least we are on the way.

Carol and I hope that wherever you are in your efforts to discern the impact of your charity or network, the ideas in this issue of Mackenzie Strategic Philanthropy will help you move ahead.