Top 3 Data Pitfalls and What to do About Them

Intake form? Check!

Pre-survey? Check!

Post-survey? Check!

There you are – chugging along with your evaluation practice. Nice job! A pat on the back is well-deserved. So now that you are feeling pretty solid about your data, we want to share a few things to think about that can take you to the next level. Because there is always room for improvement, right?

Here are some common pitfalls we see nonprofits make. But don’t worry! They are pitfalls – not cliffs – you can climb out of these.

Pitfall #1: Not communicating back to constituents. Yes, your donors, program directors, and grant writer want to know what you found in your data analysis, but what about your clients? What about the people who shared all their opinions and personal information with you? Clients are often left wondering, what happened to all that information I gave? Let clients know what you are doing with the information they share, so they feel valued. Post a small board in your lobby or a flyer in the bathroom that says, “We heard you! Thanks to your input, we will now be making (fill in the blank) change to our programs.” You can also post results and findings on social media or include a blurb in your next client communication. But remember to make the communication client-focused – make it relevant for them.

Pitfall #2: Only focusing on rigor.  Don’t get me wrong here – rigor is great. We need rigorous evaluation to make big advances in the field. But for many nonprofits, getting caught in the trap of rigorous evaluation is not a good place to be. Maybe you have it stuck in your head that data must be collected with a five-point, Likert scale, 10-page survey in order to be valid. In-depth, rigorous data analysis takes a significant amount of time and expertise. There are times when rigor should take priority, but for continuous feedback considering focusing on utility.

Assessment activities are a great alternative to surveys if you are looking for some quick feedback from your clients. Simple things like role play, writing or drawing prompts, and photography are a great way to gauge whether or not your participants are experiencing change. For example, we love chalk talk: Place flipchart paper around your program area. Write a prompt on the paper (e.g. draw or write about what Program Name means to you OR how can we make Program Name better?) Leave the flipchart paper up for a week or two and see what kinds of responses you get. This is a great way to allow participants to contribute on their own time in their own way. Plus, it’s low-cost, simple, AND actionable! You have immediate data that you can use. But don’t forget to look for themes and trends – just as you would with any other forms of data.

Pitfall #3: Only collecting data for the sake of accountability. When we’re strapped for resources, time, and energy, it’s easy to get stuck in the accountability trap. We throw reports together for funders, enter our data in the database black hole, and slap numbers on our one-pagers just to get something out the door. At ResultsLab, we argue that data should be for YOU first – collecting data for learning and improvement. Otherwise, you can risk your program staff getting burned out and resentful toward data collection.

We love using simple dashboards and visuals to get staff talking about the data. Regular standing meetings with your staff to look at the latest trends in data will allow space for conversation, reflection, and action. Posting these visuals in a high-traffic area will also help to show staff that the information they are working so hard to collect has high value for your organization.