What makes an evaluation report compelling? (or not)

By Samantha Grant

Okay, time for true confessions here. How many research or evaluation reports do you have sitting on your desk? You know there was blood, sweat, and tears put into the creation of those documents, but somehow you don't feel compelled to read them. Why not?

I'm willing to guess that the answer is either: A. you don't have time or B. the reports are way too boring. (By the way, reason A is just a disguise for reason B.)

The truth is that many evaluation reports are dull, but there are also great ways to spice them up by focusing the message and using pictures and stories to illustrate points. I have set a goal of unlearning some report-writing habits to make mine more interesting, and thus more likely to spur action.

A new learning experience that I'm taking part in is the American Evaluation Association's eStudy series: An Executive Summary is Not Enough: Effective Reporting Techniques for Evaluators. In it, Kylie Hutchinson sets out to teach practitioners to create meaningful reports.
One of the reasons for ineffective reports may be bad writing habits. In an editorial, Jane Davidson argues that our social science training has taught evaluators some behaviors that need to be unlearned. She notes four ways:

1. Including models or theories without connection
2. Leaping to measurement too quickly
3. Reporting results separately by data source or type
4. Organizing reports like a master's thesis

In my experience, I've noticed that we train evaluators and researchers to design reports without thinking about our audiences. But if the end user isn't involved in the creation, evaluation data will never be used for change. Davidson states, "Evaluations cannot produce useful answers unless they actually ask useful questions in the first place!" I wanted to stand up and cheer for that line. Asking useful questions requires that an evaluator is in tune with stakeholder needs.
In thinking about this, I realized that as educators, our formal training may have created some behaviors that need to be unlearned. Take for example the youth worker who has training in formal education. What works inside a classroom may not translate to an after-school program or a summer camp. Academic training is important, but we need experience to be good at our jobs.
How about you? Have you read an evaluation report that really drew you in? What was it about that report that made it compelling? Can you see the need for unlearning bad habits?

-- Samantha Grant, evaluation director

You are welcome to comment on this blog post. We encourage civil discourse, including spirited disagreement. We will delete comments that contain profanity, pornography or hate speech--any remarks that attack or demean people because of their sex, race, ethnic group, etc.--as well as spam.