Program evaluations are conducted for a range of reasons, including exploring the extent to which a program or initiative has met its intended objectives. It may also be necessary to know how efficient and effective a program has been, and whether it has delivered value for money.
To explore these areas of enquiry, one or more evaluation tools and research methods will be used. The in-depth interview is a research tool that is intense, involved and comprehensive in terms of its role in understanding someone’s experience and perspective. An in-depth interview may involve asking respondents how they experienced the program or service, their personal challenges, how likely they would be to use the service or program again, and importantly, what they would have done had the initiative not been available.
When program participants are approached to provide input for an evaluation project, they can be reticent to be involved. Will they know enough to be able to help me? Could their support be withdrawn if they are not entirely positive about the program? Will their ideas be meaningful enough? Will they be articulate enough? These are not surprising concerns, particularly since the term ‘evaluation’ – until explained – can seem like a personal review of sorts. Respondents may feel they are the ones under review, rather than the initiative they have participated in.
Once the objectives of the evaluation have been explained and respondents are made to feel at ease, the interview takes place. There is a common unintended benefit that arises time and time again:
Program participants feel listened to and are able to clarify their own thinking.
The evaluation process itself, and particularly the in-depth interview, gives respondents a chance to clarify their thoughts and views about their involvement in an initiative. It is quite likely they have never been asked their opinions before. It is not uncommon – at the end of the interview – for respondents to express gratitude for the opportunity to have been involved. Here are some examples in which this has happened to me:
Disaster recovery programs
Program evaluation in the disaster recovery sector involves talking with people who have experienced personal or community tragedy, hardship or unexpected change. Conducting sensitive, open and non-judgemental conversations with people in these communities can be a positive personal experience for them. If held at the right time, many of these people find it comforting to share their story with someone who is independent and not there to help them with what may seem like endless paperwork.
Behaviour change initiatives
I am currently evaluating an environmental program for North Sydney Council that is designed to encourage commercial property owners to upgrade their buildings to be more environmentally sustainable. In my series of in-depth interviews with property owners a number of them have found the process surprisingly helpful, as it has enabled them to clarify in their own minds their attitudes towards the environment, the psychological barriers to change, and the reasons for their involvement or not.
Community arts and cultural development projects
Project managers and creative producers of regional arts and cultural programs are working with multiple stakeholders, including project participants, community members and council officers. They juggle conflicting interests, clashing personalities and varied agendas. A number of these people have told me that participating in in-depth interviews as part of an evaluation process has been cathartic and helpful. It has enabled them to clarify the challenges in their own minds, articulate their views and be listened to. They leave the interview with more clarity and a sense of the best way to move forward with the project.
Conclusion
When organisations commission an evaluation they understandably need to justify how they will benefit from the process. They may gain a better understanding of systems and processes, the outcomes may help them decide whether to continue or discontinue a program or initiative, or to better understand its strengths and weaknesses.
But the unintended benefit of enabling project participants and stakeholder to feel listened to and encouraged to clarify and articulate their own thinking brings additional benefits, regardless of the outcomes of the evaluation. When stakeholders and participants have felt valued and consulted, they can only think more highly of the organisation that funded the evaluation. This can only be a positive thing for the future of an organisation’s relationship with its client base and stakeholders.
Image: Turkish delight. marcellinaincucina.blogspot.com