The goals of drug rehabilitation are fragile. Many clients struggle to reach full sobriety by the end of treatment, and many that do relapse. Knowing about post-treatment success and its environments would provide rehabilitation centers with a wealth of feedback that could serve to refine treatment, as well as help market forces to promote the best services over ineffective ones.
In a perfect world, accurate feedback, shared honestly, would advance the field rapidly and help eliminate unnecessary costs to the public. The problem is, follow-up data are hard to obtain, too often unreliable, and describe an ever-moving target.
There are some obvious questions that beg to be asked of former clients when considering long-term rehabilitation success:
- Have you relapsed?
- If so, how persistently, and have you sought help?
- Is your current situation better than it was before treatment?
Let’s consider what it would take to gather answers to these questions. First, we would have to know how to contact former clients. Considering the population, this is very tricky. But more so for some facilities because they may serve a large population while others treat just a handful of clients. In the case of the former, do you need to find every former client, or do you consider parameters for selection? If your facility is small, you may not have enough former clients to tell you anything, and data from just one client might create misleading results.
For the moment, let’s consider a case where we have contacted a recent group of former clients that is large enough to provide valid feedback. Will enough of them respond? Current response rates for many surveys are far below what you would need to learn something generalizable. And keep in mind that error is now affected by the population size because it’s so small. Without going into the math, when populations are very large (i.e., tens of thousands or more), the sample size alone governs generalizability, but when the population size is small (e.g., the total number of former clients from your facility), the sample now needs to be a large portion of the total population to be valid.
Even if you got a decent response, you would have to gather additional concomitant data to determine if the responding clients were similar or dissimilar to non-responding clients and in what ways. These data are likely kept in the clients’ EMR (e.g., sex, age, rehabilitation history, abused substance, etc.), but you could easily imagine that non-respondents are very different from respondents. Thus, you may find that everyone that responded has stayed sober during the entirety of the post treatment period, but this can’t stand as a success rate. They are likely different from the non-responders in ways other than responding, and without the data from the non-responders, you are likely to learn very little from even more complex analyses.
Let’s continue the hypothetical. Your perfectly representative and large enough group of former clients have all responded to your survey. Are they telling the truth? It’s one thing to respond to personal, or even incriminating questions when you are currently in treatment, but reinforcers change during post treatment. Incentives to lie are strong, and even data that are derived from believable outside sources (e.g., arrests, incarceration, or emergency room visits) might be difficult to gather and would not generalize well.
Finally, let’s say they all told the truth. Even these data might not connect to current clinical environments, practices, or practitioners. The constructs under your microscope are not fixed. This makes them both more interesting and more difficult to describe.
Fortunately, there is a solution. Data collected from current clients about the current rehabilitation environment and their experience with treatment and therapists will tell you a great deal about long-term success. It might not sound like it does, but our research shows how powerful the right questions persistently offered to the right persons are. The power of feedback coming from participants in real time is a remarkably valid predictor of both current and long-term success.