22: Corey Newhouse | A Shift from Monitoring and Evaluation, to Learning and Quality Improvement

In today’s episode of Creating Community for Good, Corey Newhouse sheds light on the evolution of evaluating nonprofit work via data

Subscribe on Apple Podcasts, rate, and review.

The evolution of evaluating nonprofit work via data

We’re seeing a shift from monitoring and evaluation, to learning and quality improvement. This reflects greater appreciation for the need for more dynamic and sophisticated understanding of root issues we aim to solve. Click To Tweet

How do we know if an organization is worth contributing to? Impact monitoring and evaluation is paramount today, but it wasn’t always so — and it won’t be much longer. Determining philanthropic priorities has changed generationally. Going back to the times of the Silent Generation (think: our oldest, living or recently-passed generation), there were far fewer nonprofits than now, and they were supported primarily because of their brand, celebrity endorsement, relevance to society and scope of service. There was trust, legacy and loyalty. It evolved over the last decade or so to focus on impact monitoring and evaluation to prove relevance, effectiveness, efficiency and impact of activities in the light of specific objectives. Return on investment and overhead were most closely examined — almost obsessively. While the intention behind the rigor and review is good, it caused many to lose sight of the bigger picture. Nonprofits were then forced to reallocate funds from the program to reports. That’s why evaluation is evolving again now. In this episode, Corey Newhouse — consultant, analytical brainpower, and founder of Public Profit — unveils a new measurement: are we addressing the root issue with our work (not just putting a band-aid on it) and are we tracking learnings over time?

Moving the needle towards progress

Public Profit is a consultancy that works with mission-driven organizations, helping them collect the right set of data for great decision-making and meaningful reporting. Corey gets to learn for a living and supports organizations ranging from big national philanthropies and sports associations, to tiny organizations teaching cultural arts and dance — and everything in between. The real joy in her work is helping colleagues in the mission-driven field to do things in a reliable and consistent way that is backed up with data-informed insights.

Numbers are really great at showing how consistently and broadly something is happening, but they are terrible at explaining why… so how do we collect the right set of data and properly evaluate it, to move the needle on the philosophy in addition to the actual metrics? Measuring impact for nonprofits is challenging if we use numbers alone and they only tell one side of the story. Numbers help us collect information about the quality and the consistency of the experience itself, but they aren’t able to engage in meaningful learning conversations. This is why we need to gather qualitative information through interviews, focus groups, and case studies, to help us give life and meaning to the metrics — showing the range of experiences and how lives are improving.

Alignment Via Discovery and Shared Learnings

One of the biggest challenges is figuring out what programs to fund? Is it the program the organization has been working on, or is it the new idea the major donor has presented and will fund? Nonprofits must be open to new projects, but also disciplined, committed, and informed about programmatic efficacy to stay true to their Mission, Vision and Theory of Change. (Reference Episode 1 with Aila Malek for more on Theory of Change.) Statistical insights paired with tracked learning focus nonprofit efforts AND keep donors interested and invested. (Note: check out Episode 20 with Ryan Oliver where we discuss the value of a mutually agreed upon “Decision Tree” for how to do this best!)

Surveys, interviews, observation, and focus groups allow us to make sense of information in order to improve operation, increase impact and prove learning. Rich data collection is achieved by actually listening to people and their concerns, and even though you may not be able to fulfill every request, it is much better than no action at all. Accountability for engaging in the process of inquiry and reflection and improvement is more important than hitting a certain mark. This is the message we need to communicate back to donors so that we can maintain alignment.

How do we get that representative sample?

Filling a survey or an interview may feel like a burden. Be clear when during surveys about how much time it’s going to take, what are the objectives of the survey, and what is the person surveyed getting in return. Gift cards, swag, access to webinars, executive summaries… anything that you can give in return for people’s time and insights are helpful.

Collect the data that you want to use and be thoughtful about collecting actionable information. Let’s not just keep collecting information that keeps reinforcing that things aren’t working well. Let’s try instead to collect information about how we can make things work better. Try to think about where you are today, and where you want to be tomorrow.

Lastly, if you are a nonprofit, seek to respond to the following:

Why are things different than we think they should be?
What is the thing that we are trying to shift or to change, that can get us closer to what we want to be different?
We are asking for this investment so that we can…
We are doing this so that we will…
What’s the difference between where we are today and where we are going tomorrow?
How do we know we are making progress? What are the fundamental changes and what are the lessons learned?

Listen to the full episode to collect all the insights, make sense of the data to tell a story through numbers, and start moving the needle towards progress.

Key Takeaways:

(05:53) – Why deep understanding about the complexity and challenges of root issues nonprofits seek to address is critical to understanding what qualitative and quantitative information should be collected to measure impact and performance.

(09:30) – Acknowledging a shift: Greater appreciation for how complex and challenging this work really is and the shift from monitoring and evaluation, to learning and quality improvement. That reflects greater appreciation for the need for more dynamic and sophisticated understanding of root issues we aim to solve.

(18:24) – Why quantitative data is better when it is backed by a qualitative understanding.

(21:48) – How to tell a story through numbers and have a more dynamic engagement with donors.

(30:16) – How to conduct better surveys that actually get filled out and techniques to make them more enjoyable and reciprocal for your subject.

(45:49) – The single most important piece of information that a nonprofit could share with their donors.