As a physician, you’re expected to practice evidence-based medicine. But how can anyone keep up with the latest research? While there are lots of secondary sources of information (including The Carlat Report), reading original research articles allows you to reach your own conclusions about each study. But it can also be daunting.

In this article I’ll discuss a focused approach for identifying and evaluating research most relevant to your practice.

Step 1: Decide What to Read

Scores of new papers appear every day, and no one can read them all. Many clinicians’ eyes glaze over at the thought of reading journal articles, so I recommend that you focus on articles relevant to your own clinical cases. This primes your mind for new information and helps with recall.

If you start with a concise clinical question about a real patient, online sources like PubMed’s Clinical Queries page (www.ncbi.nlm.nih.gov/pubmed/clinical) can make it easy to find relevant articles. Search engines like Google can also guide you toward primary literature, but be aware that some search results are heavy in promotional content and its associated biases.

Once you’ve identified a paper, just skimming the abstract doesn’t cut it. There’s no way to evaluate a study’s caliber—or how well the results apply to your patients—without reading the actual paper. Worse yet, glancing over abstracts can lure you into accepting authors’ sometimes biased conclusions at face value.

 

Step 2: Get your Hands on It

You can always plunk down cash, but getting full articles is expensive if you don’t have a system.

If you work at a hospital or university, you probably have access to a medical library where mainstream journal articles are available for free, while less-common publications can be ordered. Some medical libraries “lend” journal articles to physicians in their communities even if they aren’t university-affiliated.

PubMed’s search page (www.pubmed.gov) has direct links to many free articles, and the National Library of Medicine has a page dedicated to finding full-text resources (http://1.usa.gov/1q3bvnw).

Another excellent resource is Google Scholar (scholar.google.com), which scours the Internet for PDF versions of full-text articles, and is a powerful search engine in its own right, comparable to PubMed. Finally, many professional organizations offer online access to their journals as a benefit of membership.

Step 3: Understand the Design

Once you’ve selected an article, you’ll need to identify the study design. For an in-depth review of study designs, check out Clinical Epidemiology: The Essentials by Robert and Suzanne Fletcher (5^th edition, Lippincott Williams Wilkins;2012).

There are many variations and hybrids, so take a close look at the “methods” section of papers you read. Most published research in psychiatry can be categorized as one of the following types:

• Case reports: Someone writes up an interesting case they’ve seen. Case reports generate hypotheses but don’t test them. They are highly susceptible to bias, in part because they often describe the joint occurrence of uncommon events. Case reports rarely describe treatment failures. By definition, they only describe one patient (ie, “N=1”), so case reports almost never provide a basis for altering clinical practice.


• Case series: Someone writes up a small number of similar cases. Case series have no control group and don’t test hypotheses, and they suffer the same susceptibility to bias as case reports. However, they reveal patterns among similar patients, and may lead to new hypotheses or suggestions for managing unusual or refractory conditions.


• Case-control studies: Researchers select cases with versus without a particular outcome, then ask subjects about prior exposures. For example, people with or without a current diagnosis of schizophrenia may be asked about exposure to cannabis. Case-control studies are highly susceptible to recall bias. They give an estimate of risk called the odds ratio.


• Cohort studies: Groups of people are followed prospectively to see how many people either with or without a particular exposure develop an outcome of interest. For example, people who do and don’t smoke cannabis are followed up after 10 years to see how many in each group developed schizophrenia.
  
  Cohort studies allow calculation of relative risk, but they are prone to misclassification and susceptibility bias. Cohort studies can also evaluate non-randomized treatment effects, for instance, whether cannabis smokers who take antidepressants may be more likely to develop schizophrenia than those who do not.
  
  
  Some cohort studies obtain information from registries of patient data—for example, all members of an HMO; all VA patients; or all individuals born in Denmark between 1980 and 1989. These are helpful because they deal with “real-world” patients, although they are not randomized. Electronic medical records (EMRs) make registry studies much more feasible.


• Randomized controlled trials: Subjects are carefully selected and then randomized to treatment or placebo groups. RCTs evaluate the efficacy of treatment in the short-term, but they are costly.
  
  Some RCTs are “open-label,” meaning that subjects and researchers know what’s being given, while “double-blind” trials mean no one knows who’s receiving treatment and who’s receiving a placebo. Blinding can be difficult to accomplish, for example, in studies where one treatment arm receives psychotherapy. All RCTs are subject to selection bias.


• Systematic review: This is a review of research designed to answer a specific clinical question, for example, “what is the most effective approach for treating psychotic depression?” The Cochrane Collaboration (www.cochrane.org) produces many high-quality systematic reviews. Systematic reviews impose strict inclusion criteria on the studies they analyze, but publication bias poses a major problem.


• Meta-analysis: Researchers use statistical methods to produce a weighted average of treatment effect sizes derived from multiple studies. The weight accorded to each study depends on sample size and quality. Some systematic reviews incorporate meta-analytic methods. Again, publication bias can strongly influence results.

Step 4: Identify Biases

Bias refers to anything that systematically and unexpectedly influences research results. It affects all research, not just the studies carried out by the pharmaceutical industry. There are many classifications of bias, and study designs differ in their susceptibility to each.

Understanding how bias affects internal validity—meaning your confidence that a study accurately identifies cause-and-effect relationships—is indispensable to critically appraising research. (See “Investigating Bias in Research” in this issue for more about bias.)

Step 5: Think About Random Error

Pure chance can throw off study results and render them invalid. In general, effect size and study power determine how likely the results may have arisen purely from chance. Appendectomy for acute appendicitis, for instance, has such a large effect size that its benefit is almost certainly not due to chance.

A larger number of subjects—otherwise known as more power—reduces the role of chance in clinical trials, but introduces more heterogeneity into the population.

Researchers use statistical methods to determine the likelihood their results arose from chance. If that probability falls below an arbitrary threshold, such as p<.05 in most RCTs, the results are said to be statistically significant. Statistical methods go beyond the scope of this article, but Clinical Epidemiology: The Essentials provides an excellent introduction.

Step 6: Appraise the Study

Just because results are touted as statistically significant doesn’t mean they’ll help you help your patients. Validity refers to how legitimate the results of the study actually are, and can be evaluated by the FRISBEE mnemonic, created by Duke University’s residency program. The last E, “equivalence,” may be the most important, as it pertains to external validity, or how well study results will generalize to patients in your practice (Xiong GL & Adams J, Curr Psychiatry 2007;6(12):96).

Although FRISBEE was developed for appraising RCTs, similar concepts can be applied to other study designs. Appraisal worksheets for treatment and diagnostic studies are available on Duke University’s website (bit.ly/1rMWqGc).