FROM THE BLOG

Evaluation or research?

Screen Shot 2017-07-20 at 10.10.14 am

I sometimes encounter confusion or misunderstandings from colleagues or associates about the differences between research and evaluation. Because this happened again just recently I feel I would like to outline five ways in which evaluation differs from social or market research.

1. Empirical vs applied knowledge

Research involves the production of generalisable knowledge, sometimes theoretical, other times practical. That is, its prime purpose is to gather knowledge from a key group of respondents so that broader inferences can be drawn about a population of interest. Evaluation, on the other hand, is more interested in specific, applied knowledge so that evaluative conclusions can reliably be drawn about what is being evaluated.

2. Value-free vs determining value

Whereas research aims to be value-free, that is researchers collect data and then draw conclusions that are linked to the empirical data, evaluation is about determining the merit or worth of something by assigning values. Like social researchers, evaluators collect data, but they then examine how the data lines up with previously determined standards (also known as criteria or benchmarks) and then make value judgements from this evidence. This can sometimes be a complex or involved set of tasks.

3. Evaluators explore the issue of causation

Importantly, evaluation should not ignore the exploration of the complex and often difficult issue of causation. Evaluators are interested in the full of changes to a community of interest following the rollout of an initiative, program, activity or whatever is being evaluated. But even if they observe changes that are consistent with expectations of their own or their client, they need to know whether these changes have come about as a result of the program or initiative, or for other reasons. Exploring causal pathways is unique to evaluation.

4. Controlled vs natural setting

Research is usually conducted in a controlled setting. For example, qualitative research may involve conducting focus groups in group rooms with a one-way mirror for the client to observe, or in-depth interviews in a respondent’s workplace setting. Evaluation, on the other hand, is usually about observing behaviour within natural community settings, where a program or initiative has been delivered. Consequently, a number of moving parts and external influences need to be taken into account. Additionally, sometimes the concept of ‘transferability’ is explored, that is, the extent to which the program may be successful or otherwise in other settings.

5. Theory neutral vs theory-dependent

Both social research and evaluation strive to increase our understanding in a range of areas, but the kinds of understanding they contribute to are different. While research is intended to increase a body of knowledge on a particular topic, in evaluation, knowledge is only a means to an end. The value of evaluation is the gaining of insight into how the intended program or activity is intended to work. This is known as program theory. It involves a systematic examination and explanation of the internal mechanisms that make a program work, its outcomes, as well as the context in which it works best. Although not all evaluations are theory-based, they all involve a critical analysis of the workings of a system, so that the elements that makeup that system are then tested. Established evaluation scholars Pawson and Tilley (1997) claim that without a theory of why an intervention may be effective, along with the conditions that promote its potential to work, research into its usage is blind.

Although there are considerable overlaps between research and evaluation, evaluation can be considered to be a separate discipline to social research. And that is not just my theory!  Read my blog about Impact Evaluation here, how to conduct evaluation on a shoestring here, and more about cause and effect here.

Reference:
Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: SAGE.

Back to All Posts