Alcoholics Anonymous (AA) occupies such a prominent place in popular culture and addiction treatment that it seems a little odd to question its effectiveness. But odd, counterintuitive questions about AA have been accumulating for years, and we finally have a book that collects them all in one place.

Lance Dodes, MD, a psychiatrist who has spent decades studying and treating addiction, and freelance writer Zachary Dodes, have authored The Sober Truth: Debunking the Bad Science Behind 12-Step Programs and the Rehab Industry (Boston: Beacon Press;2014).

Their book makes a searching and fearless scientific inventory and finds very little evidence that AA is effective. The underlying data and arguments are compelling.

The authors correctly note that randomized controlled trials (RCTs) are the gold standard in medicine and that AA comes up woefully short in this department. The Cochrane Collaboration, which synthesizes the best evidence available, found only eight RCTs of adequate quality that looked at AA or 12-step Facilitation (TSF), the professional treatment approach that strongly encourages AA participation. The resulting meta-analysis concluded, “No experimental studies unequivocally demonstrated the effectiveness of AA or TSF approaches for reducing alcohol dependence or problems” (Ferri M et al, Cochrane Database Syst Rev 2006;3:CD005032).

The usual rebuttal is that RCTs are perhaps an unfair standard to apply to AA. For example, it would be unethical to prevent people from participating in community-based AA groups to create the necessary control group. Moreover, AA doesn’t keep records, which complicates researchers’ ability to verify even basic factors such as meeting attendance, let alone drinking behavior and alcohol-related consequences. With that said, TSF, a psychotherapy, is amendable to RCTs—we just don’t have very many of them and the few that do exist have methodological problems.

The Dodes then explore the usual evidence about AA’s effectiveness. Theypoint out that these studies are all observational in nature, meaning “this research is conducted passively… without interventions or controls.” Selection bias and spontaneous recovery are devastating methodological problems in these types of investigations and may be responsible for most of AA’s apparent success.

In the case of AA, selection bias refers to the characteristics of people who choose to affiliate with the program. Everyone agrees that people with significant involvement in AA have better outcomes than those who do not. This is mere correlation, however, and doesn’t establish that AA is responsible for the outcomes, say the Dodes.

They observe, “the dilemma facing AA research is whether people stay in AA because they’re the type of people who will stick with a program no matter what it is and who would have stuck with it even if it were of no help to them at all.”

Contrary to popular belief, spontaneous recovery (getting better without any treatment) from alcoholism is pretty common. This overlooked fact is always lurking in the background of observational studies. When this phenomenon is brought out into the open, the Dodes note that “some of AA’s success rate may simply be nature taking its course.”

The Dodes also take aim at what they call the “rehab industry,” which largely uses TSF as its treatment paradigm. Despite the dollars and lives involved, there is an embarrassing lack of peer-reviewed outcome data. The authors combed the literature and found only one observational study, from Hazelden’s flagship residential program in Center City, Minnesota (Stitchfield R & Owen P, Addict Behav 1998;23(5):669–683). The investigation was fraught with so many problems—selection bias, no control group, self-reported data, questionable statistical methods—that it’s difficult to draw any meaningful conclusions from it.

Not surprisingly, The Sober Truth has drawn fire. A book review by psychiatrist Dr. Richard Friedman in the New York Times called it “polemical and deeply flawed,” although the main objections surrounded how the Dodes conceptualize addiction and propose to treat it, not their core arguments about AA’s efficacy (http://nyti.ms/TBQog9). Book reviews have not yet appeared in the professional literature, although a nuanced and more positive assessment of the AA literature has been available for a number of years (Kaskutas LA, J Addict Dis 2009;28(2):145–157).