Review of: Agarwal AK et al, J Gen Intern Med 35(6):1647–1653

When a patient or family member is looking for a specialized drug treatment facility (SDTF), they will often turn to online review sites to get information about facilities in their area. These reviews can drive patients’ choice of facility, but they often don’t focus on the same themes as national surveys of healthcare quality. What can we learn from online reviews about what makes a good or bad facility, according to our patients?

This study evaluated online reviews of all the SDTFs in Pennsylvania that were registered with the Substance Abuse and Mental Health Services Administration (SAMHSA) as of 2016. The reviews were posted to Google or Yelp from July 2010 to August 2018. A total of 539 SDTFs were registered in 2016, but only 485 had a review posted to Google or Yelp during the study period. The types of facilities included inpatient, residential, and outpatient levels of care. There were 7,823 reviews posted during the study period, with 68% including narrative text. A bimodal distribution emerged, with 43% of reviews being 5-star (best) and 34% 1-star (worst), so the authors focused only on these extremes of the review spectrum. In order to statistically analyze the narrative reviews, the researchers used a technique called latent Dirichlet allocation (LDA). LDA generated topics or themes based on the repetition of words in the text using an automated machine learning process. Correlation coefficients (r values) were then calculated to identify which themes were most closely matched with 5-star or 1-star reviews. R values could range from 0 to 1, with larger numbers showing higher correlation.

The themes most correlated with 5-star reviews were focus on recovery (r = 0.53), helpfulness of staff (r = 0.43), compassionate care (r = 0.37), life-changing experience (r = 0.32), and professionalism (r = 0.29). For the 1-star reviews, the matching topics included long wait time (r = 0.41), poor accommodations (r = 0.26), poor phone communication (r = 0.24), lack of medications offered (r = 0.24), and lack of appointment availability (r = 0.23). The researchers pointed out that most of these themes (both positive and negative) are not included in national surveys of SDTFs such as ­SAMHSA’s annual National Survey of Substance Abuse Treatment Services.

These results reveal what patients most value in SDTFs—focusing on recovery and delivering care in a compassionate, professional manner. There are obvious limitations to this study as pointed out by an editorial (Chen KM and Fiellin DA, J Gen Intern Med 35(6):1633–1634), most concerning of which is data fidelity. There’s no way to know if a review was posted by an actual patient of the facility or if it was a fake review. There were also very few reviews per facility—an average of 15.3 per facility, or about 2 reviews per facility per year—and 54 facilities had no reviews at all.

CATR’s Take
Most of the themes identified in online reviews of addiction facilities are not reflected in national standardized surveys, so patient-centered outcomes need to be included in our appraisal of addiction facilities. This analysis gives voice to what matters most to patients’ experiences in SDTFs and may help guide topics for inclusion in standardized national surveys.