Clinical Orthopaedics and Related Research®
© The Association of Bone and Joint Surgeons® 2012
10.1007/s11999-012-2703-y

Editorial: Let’s Talk About Level IV: The Bones of a Good Retrospective Case Series

Seth S. Leopold 
(1)
Clinical Orthopaedics and Related Research, 1600 Spruce Street, Philadelphia, PA 19103, USA
 
 
Seth S. Leopold
Published online: 21 November 2012
Without Abstract
The author certifies that he has no funding or commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.
All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request.
“A good retrospective case series.” An oxymoron? Not necessarily.
The retrospective case series, or Level IV study, has always been, and remains today, the most common research design in our literature. When I first looked at this topic more than 10 years ago, nearly four in five orthopaedic studies were retrospective [3]. Although it is trendy in evidence-based medicine circles to turn one’s nose up at the humble case series, there is no avoiding them, and, in fact, a majority of the 100 most-commonly cited orthopaedic articles are Level IV studies [2].
As the Editor-in-Chief of Clinical Orthopaedics and Related Research ®, I would love to see the journal populated with randomized trials and meta-analyses. But I also know that —at least for the foreseeable future—I will see many Level IV studies for every randomized trial that crosses my desk. The retrospective reports that rise to the top likely will share several common elements.

They Will Start with Interesting, Specific Questions

Retrospective studies can be good conversation starters. They can serve as “proof of concept” before embarking on more convincing (and more expensive) prospective studies. New topics almost always make their first appearances in the literature as retrospective reports. Focusing on (or introducing) a controversy is another potential use of these reports, as has been suggested before [1]. However the questions must be interesting and the authors must articulate very specific endpoints to start a good conversation. The paper’s methods then must be tooled specifically to answer those questions. Statements like “We report outcomes on…” or “We present the results of…” are too general to be useful.

They Will Recognize that Level IV Research Cannot Drive the Adoption of an Approach or an Implant, but a Retrospective Series of Failures Might Lead Surgeons to Abandon One

“We did 10 of these, and seven worked” is not convincing in the absence of a control group. “We did 10 of these, and seven failed” might very well be, especially if the high risk of failure is a new finding.

They Will Articulate Clearly What, and Who, was Treated

How did the treating physician decide to use the approach in question? What were the diagnostic criteria, and which patients with that diagnosis got the treatment in question? As important, which patients did not? Selection bias can be a critical weakness in Level IV therapeutic studies. Clear diagnoses and indications for treatment are critical to help readers understand whether selection bias influenced the findings.

They Will Deal with the Three Biggest Problems of Level IV Studies: Bias, Bias, and Bias

Selection bias is not the only problem with these papers. Other frequent offenders include:
  • Transfer bias—loss to follow-up
  • Co-treatment bias—application of additional treatments, such as other drugs or therapies, to the patients getting the new approach, and
  • Assessor bias—having an interested party evaluate the patients (such as the operating surgeon), or failing to use a suitably validated outcomes tool to quantify the results.
Level IV reports can be useful. Many, if not most, of the important advances in our specialty began as retrospective case series. However, they must be written, read, and published with care.
References
1.
Brand RA, Lotke PA. Editorial – Scientific reporting: how to focus the “good ole boy paper”. Clin Orthop Relat Res. 2011;469:607–608.PubMedCrossRef
2.
Lefaivre KA, Shadgan B, O’Brien PJ. 100 most cited articles in orthopaedic surgery. Clin Orthop Relat Res. 2011;469:1487–1497.PubMedCrossRef
3.
Leopold SS, Warme WJ, Fritz Braunlich E, Shott S. Association between funding source and study outcome in orthopaedic research. Clin Orthop Relat Res. 2003;415:293–301.PubMedCrossRef