Formative Evaluation

Tamra d'Estrée

Conflict Resolution Program, University of Denver

Interviewed by Julian Portilla, 2003

This rough transcript provides a text alternative to audio. We apologize for occasional errors and unintelligible sections (which are marked with ???).

A: What we're trying to do is develop ways to track those changes better because unfortunately when people look at the impact of this work they tend to look either at individual change, which we tend to find that yes that does happen, or they look at kind of subsequent systemic change where all these other complicated factors come into play. It is very unlikely you're going to see very much change at that level. So how do you see when an individual might take their new learning or new awareness of options or new insights into the other and do something slightly differently in the way they work?

Q: Have you found any particular evaluation technique more favorable to come up with that kind of result?

A: Well, that's what we're trying to develop. We're basically trying to develop a framework that alerts practitioners or whoever would be using the evaluation framework to be looking at that next level of tracking. Or maybe even building it into the workshops itself. It's not uncommon that what you track in an evaluation in any kind of social intervention ends up being what then people train for. Like in education if you start tracking people. You know these schools have these educational assessments now, and whatever is in the tests is now what people are training for so that it will show up in the test. And that sounds negative, but in actuality it may not be bad because if part of what you start to track in this work is not just changes in individual attitudes but changes in whether or not people then take new insights into their workplace or their institution or their networks and do something differently and have their institution or their network do something different vis-…-vis the other side. If you start to track that, then you can think about how people might recognize that it might be important to train that way or to do the workshop in such a way that you are not just having people's attitudes change.

Rather, you have them discover new insights about the other, but having them have thought through in the workshop how are they going to take that new insight out into the world and do something differently with it in their institution or their network. So you build it into the first step, into the intervention, how that next level of change might actually happen, and you get people to plan for it.

Q: It sounds like the evaluation in that case would lead the substance of the workshop, so that you would need to know what you're going to evaluate before you start planning for this. So then you can target that particular goal.