I recently purchased a loaf of ‘low carbohydrate’ bread (see image). Now, looking at this loaf, I think you will agree there is no denying that this loaf would be lower in carbohydrates than other loaves. But that is possibly because it has a great big hole in it.
I’m only kidding, of course, but this ‘hole-grain’ loaf of bread presents a useful metaphor for arguing for the importance of well-defined performance indicators as a measure of success when evaluating a program, initiative, service, or product.
Performance indicators are measures of an outcome. They key is to make sure the indicator is a reasonable, useful and meaningful measure of the intended outcome. In this case the intended outcome is lower carbohydrate bread; the indicator would be the claim that this bread has 25% lower carbohydrates than regular bread. This is important information to have at hand, because to simply say the intention was to create a loaf of bread with fewer carbohydrates than other loaves is too broad, too general to develop specific performance targets. So, in the development of the recipe for the bread, it was important to identify a performance target of 25% less than regular bread.
There are some common features that represent good performance indicators. Here are three, as identified by USAID Centre for Development Information and Evaluation:
Direct
A performance indicator would measure as closely as possible the result it is intended to measure. If ‘low carbohydrate’––defined by the health and nutrition industry––is 25% less than regular bread per loaf, then that is the measure to use.
Objective
An objective indicator has no ambiguity about what is being measured. This is to ensure there is no disagreement over interpretation of results. It should measure one phenomenon at a time. In the case of our hollow loaf of bread, there may be reason to suspect there is some ambiguity about whether the claim is actually 25% fewer carbohydrates than other bread per weight, or whether they’ve managed to reduce the level of carbohydrates by providing a smaller loaf.
Reliable
The data used to measure the performance should be of sufficiently reliable quality for confident decision-making. As a consumer, had I known I would only be receiving half a loaf of bread, I may have opted for a regular loaf of bread with higher levels of carbohydrates.
Performance indicators are important measures that describe how well a program is achieving its objectives. When chosen well, and communicated clearly, they are effective means to measure progress towards objectives. My hole-grain bread claimed to be low carbohydrate. And no doubt it was. But its hollow nature made me question how claims can be justified in ways in which we may not be aware. As evaluators we need to be able to identify these to ensure we are making sound judgements.
And finally, some domestic advice: should you happen to purchase a similarly hollow loaf of bread, I don’t recommend putting it under the grill with cheese on it, as it’s just messy.