Sometimes research and evaluation can be confused or conflated, as I explain in my new online evaluation course for local government, which you can see here.
When commissioning a project it is important to be clear about whether you require research or evaluation. Because, despite them being related and sharing some common approaches, they are different disciplines that result in considerably different outcomes. I have identified five ways in which research and evaluation differ.
1. Empirical vs applied knowledge
Research involves the production of generalisable knowledge, sometimes theoretical, other times practical. That is, its prime purpose is to gather knowledge from a key group of respondents so that broader inferences can be drawn about a population of interest. Evaluation, on the other hand, is more concerned with specific, applied knowledge so that evaluative conclusions can reliably be drawn about what is being evaluated.
2. Value-free vs determining value
Whereas research aims to be value-free, that is researchers collect data and then draw conclusions that are linked to the empirical data, evaluation is about determining the merit or worth of something by assigning values. Like social researchers, evaluators collect data, but they then examine how the data lines up with previously determined standards (also known as criteria or benchmarks) and then make value judgements from this evidence. This can sometimes be a complex or involved set of tasks.
3. Evaluators explore the issue of causation
Importantly, evaluation should not ignore the exploration of the complex and often difficult issue of causation. Evaluators are interested in the full of changes to a community of interest following the rollout of an initiative, program, activity or whatever is being evaluated. But even if they observe changes that are consistent with expectations of their own or their client, they need to know whether these changes have come about as a result of the program or initiative, or for other reasons. Exploring causal pathways is unique to evaluation.
4. Controlled vs natural setting
Research is usually conducted in a controlled setting. For example, qualitative research may involve conducting focus groups in group rooms with a one-way mirror for the client to observe, or in-depth interviews in a respondent’s workplace setting. Evaluation, on the other hand, is usually about observing behaviour within natural community settings, where a program or initiative has been delivered. Consequently, a number of moving parts and external influences need to be taken into account. Additionally, sometimes the concept of ‘transferability’ is explored, that is, the extent to which the program may be successful or otherwise in other settings.
5. Theory neutral vs theory-dependent
Both social research and evaluation strive to increase our understanding in a range of areas, but the kinds of understanding they contribute to are different. While research is intended to increase a body of knowledge on a particular topic, in evaluation, knowledge is only a means to an end. The value of evaluation is the gaining of insight into how the intended program or activity is intended to work. This is known as program theory. It involves a systematic examination and explanation of the internal mechanisms that make a program work, its outcomes, as well as the context in which it works best. Although not all evaluations are theory-based, they all involve a critical analysis of the workings of a system, so that the elements that makeup that system are then tested. Established evaluation scholars Pawson and Tilley (1997) claim that without a theory of why an intervention may be effective, along with the conditions that promote its potential to work, research into its usage is blind.
Although there are considerable overlaps between research and evaluation, evaluation can be considered to be a separate discipline to social research. And that is not just my theory! Read my blog about Impact Evaluation here, how to conduct evaluation on a shoestring here, and more about cause and effect here.
If you work for local government and would like to be taken through 8 simple steps to conducting an evaluation, go to the course here to see more information or to enrol.
Reference:
Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: SAGE.