Program Evaluation Toolkit for Harm Reduction Organizations
“In this society, if you use drugs, you become an expert at noticing and knowing because your life depends on it, but we rarely get treated like what we are: subject matter experts. People rarely think to ask a local expert about the deep questions and have a 5-minute conversation rather than a 5-week data collection effort.”
—Luka Bair
Earlier in this module, we highlighted some important benefits of program evaluation for individual harm reductionists to consider. Now let’s talk about some key considerations when making the case for program evaluation within your organization and with members of your community. This often involves addressing mistrust or fear of negative consequences.
When it comes to making the case for program evaluation for your team or community, it is helpful to consider the following.
Meaningful involvement of people who use drugs (PWUD) is embedded within the core principles of harm reduction and is something that every program should consider when laying the foundation for an equitable and culturally responsive evaluation. Meaningful involvement of PWUD in program evaluation processes is about granting decision-making power to people with lived experience of drug use to inform the design, implementation, and analysis or reporting of an evaluation effort.
Here are a couple of questions to ask yourself as you consider what meaningful involvement can look like for your program:
WHAT IS LANGUAGE JUSTICE?
Language justice is the practice of respecting everyone’s fundamental right to communicate and receive information in the language that they prefer and feel most articulate and confident using. It is also includes actions that work to dismantle language barriers and traditional systems of language-based oppression and discrimination.
Additional information on how organizations can meaningfully involve people who use drugs can be found here: AIDS United - Meaningful involvement of people who use drugs [factsheet].
Emotional or ideological hesitance to program evaluation refers to the hesitation among staff or clients to engage in or contribute to a program evaluation due to uncertainty about the utility of the process. Both emotional hesitation (not feeling right about it) and ideological resistance (not feeling it is the right thing to do) come from somewhere and, in many cases, are connected to a lack of trust in the integrity of the process, not feeling comfortable asking clients to share more information than is required for services, or having been burned by a poorly executed harm reduction program evaluation in the past. Whatever the reason, it is important to intentionally work to understand what might be fueling this reluctance and implement strategies that build trust, address concerns, and encourage authentic engagement.
In some cases, there may be a tendency to avoid evaluation in order to avoid facing uncomfortable or difficult truths. Acknowledging and naming that discomfort is the best way to overcome it. Evaluation can break the vicious cycle where lack of clarity allows problems to worsen and unpredictable crises to occur.
Voices from the Field: Al Forbes
“I think without evaluation tools, without inflection tools, you're out there working but no one knows exactly what you're doing. And I find that some people avoid evaluation because they're afraid of what it might show.”
The hostile political environment is another key consideration. Across the country, harm reduction program staff and clients are constantly at risk of being targeted, arrested, or shut down because of stigmatizing policy and public outcry against active drug use. Similarly, the political environment traditionally ranges from very hostile to extremely hostile for people who use drugs.
For many programs, collecting data on individuals who receive services, as well as the services themselves, can feel both risky and incriminating. Mistrust of the government and of government agencies is still very much a sensible attitude for PWUD, so while a program might feel they're suddenly operating in a more permissive environment, many PWUD won't be able to see it that way.
Harm reduction—as a movement—was created in response to the threats posed by a political and legislative program: the War on Drugs. For as long as the dominant political and legal regime supports the War on Drugs, harm reduction will be exposed to political hostility. In Barbie’s case, despite support from all other stakeholders, a single political figure’s opinions seriously hampered an effort to use evidence-based evaluation data to improve services.
Voices from the Field: Barbie Zabielski
“We asked questions and we told people that ‘you matter to us’ and now we're not doing anything with it. We almost feel like [not being able to deliver solutions due to political interference] has hurt our credibility, our ability to ask questions…It’s very frustrating to be a public health agency, and not to be following best practices because of opinion.”
Addressing the impact of hostilities toward harm reduction programs doesn’t happen overnight, but with the increased visibility of the benefits of harm reduction, there is a reason to be hopeful.
Table (1.3). Below are some strategies for making the case and addressing hesitancy.
| CONSIDERATION | STRATEGIES | EFFECT |
|---|---|---|
| Meaningful involvement of people who use drugs |
|
Influence impedes resistance and leads to identification and ownership regarding evaluation, common goals, increased commitment, trust, and improved cooperation |
| Emotional or ideological hesitancy |
|
Increased communication and knowledge about the evaluation process and intended outcome |
| Hostile political environment |
|
The potential “costs” of data evaluation will be more easily accepted |
Adapted from Taut, S. and Brauns. D. (2003). Resistance to evaluation: A psychological perspective. Sage Publication; 9(3): 247-264.
Each evaluator has their own motivations for contributing to thorough evaluation processes, but there are common traits and attitudes among those who find value in an organization reflecting on itself and the work it does. Read through the following Voices from the Field section to hear how one experienced evaluator found a passion for the process and learned to make the case for evaluation to others.
