Program evaluations are conducted for a range of reasons. Most broadly, those who commission evaluations usually wish to know if their initiative is meeting its intended objectives. It may also be necessary to know how efficient and effective the program has been, and whether it has delivered value for money.
Depending on the objectives of the evaluation as well as the budget and timeline, an approach will be taken to evaluation that may involve one or more research or evaluation tools. Typically, evaluations involve at least talking with end users of a program or initiative as well as those who are delivering the program. These conversations are often conducted as in-depth interviews, either in person or over the phone, focus groups, stakeholder interviews, paired interviews or any other appropriate configuration of discussion in response to the project objectives. In many cases, a program evaluation also involves analysis of existing data and records, desk research, and other types of surveys, and stakeholder consultation.
The known benefits of program evaluation are that the client will have specific questions answered, such as how to improve the program delivery, for example, or how to enhance their credibility as a service delivery organisation, or how to justify or enhance their program budget.
However, in addition to the known benefits of program evaluation, there are often a number of unintended benefits that often arise. These come about as a response to the process of the evaluation; that is, because the evaluation is being conducted at all, rather than relating to any particular outcomes or findings.
Program participants feel listened to.
Despite delivering an effective service, it is often the case that a cohort of end-users of program participants have not previously been asked their opinion of the service, or how it has impacted on them personally. One of the unintended benefits of program evaluation is that end users have the chance to openly and honestly provide feedback anonymously to a third party. They feel listened to and validated. This reflects well on the program providers and can result in greater buy-in of the program into the future.
Program providers are forced to review their systems and processes.
A program evaluator needs to be given information such as data and client records that are in a coherent and organised format such that the evaluator can analyse and make sense of them. This often means working closely with the service provider or client to source the relevant information, and organise it in a way it may not have been done before. Systems may need to be set up for the ongoing gathering and updating of this information, if the evaluation is a longer term project. This process itself can benefit the client or service provider in that they will have a system to continue to use. This may include a new working data base, an intuitive spreadsheet, a new filing system, or a more effective way for internal staff to share information.
New networks and connections can be formed.
An effective program evaluation involves reviewing the outcomes of a program from a number of perspectives, from the end user to the program provider, stakeholder and sometimes general public. By documenting the range of people involved, and having some of them work together in the process of the research and evaluation, new connections can be made that may benefit the future of the program.
Increased awareness of the program.
In our experience, the process of evaluating a program or initiative has helped to increase awareness of that program in the broader community. Throughout the course of the evaluation process, which may last from months to years, the talk of the program increases as a result of the evaluation process itself, and as a result, more people come to know about the initiative.
The process can be cathartic.
Program evaluation in the disaster recovery sector involve talking with people who have experience personal or community tragedy, hardship or unexpected change. Holding sensitive, open and non-judgemental conversations with individuals within these communities can be a positive personal experience for them. If the interviews are conducted at the right time when the respondents are ready, they are often perceived as helpful to them and a contrast to previous bureaucratic conversations they have had to have with government officials about other practical matters relating to rebuilding or emergency responses.
The findings that come from a program evaluation should directly help the client or service provider better service its own clients–– or end users–– of a program. The evaluation should answer specific questions for the service provider or client. But the range of unintended positive outcomes should not be overlooked. These benefits have other broader consequences that allow lives to be improved and systems to be enhanced, which ultimately go on to have positive effects on others in broader communities.