Skip to main content
SpringerLink
Log in

The fragility of statistical significance in distal femur fractures: systematic review of randomized controlled trials

  • Original Article
  • Published:
European Journal of Orthopaedic Surgery & Traumatology Aims and scope Submit manuscript

Abstract

Purpose

The purpose of this study was to apply both the fragility index (FI) and fragility quotient (FQ) to evaluate the degree of statistical fragility in the distal femur fracture (DFF) literature. We hypothesized that the dichotomous outcomes within the DFF literature are statistically fragile.

Methods

Using preferred reporting items for systematic reviews and meta-analyses, we performed a PubMed search for distal femur fractures clinical trials from 2000 to 2022 reporting dichotomous outcomes. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each fragility index by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ.

Results

Of the 4258 articles screened, 92 met the search criteria, with eleven RCTs included for analysis. Ninety eight outcome events with 25 significant (P < 0.05) outcomes and 73 nonsignificant (P > 0.05) outcomes were identified. The overall FI and FQ for all 98 outcomes were 5 (IQR 4–6) and 0.130 (IQR 0.087–0.174), respectively. Three studies (33.3%) reported loss to follow (LTF) greater than 5.

Conclusions

The randomized controlled trials in the peer-reviewed distal femur fracture literature may not be as robust as previously thought, as incorporating statistical analyses solely on a P value threshold is misleading. Standardized reporting of the P value, FI and FQ can help the clinician reliably draw conclusions based on the fragility of outcome measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Ahmed W, Fowler RA, McCredie VA (2016) Does sample size matter when interpreting the fragility index? Crit Care Med 44:e1142–e1143. https://doi.org/10.1097/CCM.0000000000001976

    Article  PubMed  Google Scholar 

  2. Beltran MJ, Gary JL, Collinge CA (2015) Management of distal femur fractures with modern plates and nails: state of the art. J Orthop Trauma 29:165–172. https://doi.org/10.1097/BOT.0000000000000302

    Article  PubMed  Google Scholar 

  3. Butt MS, Krikler SJ, Ali MS (1996) Displaced fractures of the distal femur in elderly patients. Operative versus non-operative treatment. J Bone Joint Surg Br 78:110–114

    Article  CAS  PubMed  Google Scholar 

  4. Carroll AH, Rigor P, Wright MA, Murthi AM (2022) Fragility of randomized controlled trials on treatment of proximal humeral fracture. JSES 31:1610–1616. https://doi.org/10.1016/j.jse.2022.01.141

    Article  Google Scholar 

  5. Checketts JX, Scott JT, Meyer C, Horn J, Jones J, Vassar M (2018) The robustness of trials that guide evidence-based orthopaedic surgery. J Bone Joint Surg Am 100:e85. https://doi.org/10.2106/JBJS.17.01039

    Article  PubMed  Google Scholar 

  6. Constant M, Trofa DP, Saltzman BM, Ahmad CS, Li X, Parisien RL (2021) The fragility of statistical significance in patellofemoral instability research: a systematic review. Am J Sports Med. https://doi.org/10.1177/03635465211039202

    Article  PubMed  Google Scholar 

  7. Court-Brown CM, Caesar B (2006) Epidemiology of adult fractures: a review. Injury 37:691–697. https://doi.org/10.1016/j.injury.2006.04.130

    Article  PubMed  Google Scholar 

  8. Doyle TR, Davey MS, Hurley ET (2022) Statistical findings reported in randomized control trials for the management of acute achilles tendon ruptures are at high risk of fragility: a systematic review. J ISAKOS 7:72–81. https://doi.org/10.1016/j.jisako.2022.04.003

    Article  PubMed  Google Scholar 

  9. Ehlinger M, Ducrot G, Adam P, Bonnomet F (2009) Distal femur fractures. Surgical techniques and a review of the literature. Orthop Traumatol Surg Res 99:353–360. https://doi.org/10.1016/j.otsr.2012.10.014

    Article  Google Scholar 

  10. Evaniew N, Files C, Smith C, Bhandari M, Ghert M, Walsh M, Devereaux PJ, Guyatt G (2015) The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J 15:2188–2197. https://doi.org/10.1016/j.spinee.2015.06.004

    Article  PubMed  Google Scholar 

  11. Fackler NP, Karasavvidis T, Ehlers CB, Callan KT, Lai WC, Parisien RL, Wang D (2022) The statistical fragility of operative vs nonoperative management for Achilles tendon rupture: a systematic review of comparative studies. Foot Ankle Int 43:1331–1339. https://doi.org/10.1177/10711007221108078

    Article  PubMed  PubMed Central  Google Scholar 

  12. Fackler NP, Ehlers CB, Callan KT, Amirhekmat A, Smith EJ, Parisien RL, Wang D (2022) Statistical fragility of single-row versus double-row anchoring for rotator cuff repair: a systematic review of comparative studies. Orthop J Sports Med 10(5):23259671221093390. https://doi.org/10.1177/23259671221093391

    Article  PubMed  PubMed Central  Google Scholar 

  13. Feinstein AR (1990) The unit fragility index: an additional appraisal of “statistical significance” for a contrast of two proportions. J Clin Epidemiol 43:201–209. https://doi.org/10.1016/0895-4356(90)90186-s

    Article  CAS  PubMed  Google Scholar 

  14. Forrester LA, Jang E, Lawson MM, Capi A, Tyler WK (2020) Statistical fragility of surgical and procedural clinical trials in orthopaedic oncology. J Am Acad Orthop Surg Glob Res Rev 4(e19):00152. https://doi.org/10.5435/JAAOSGlobal-D-19-00152

    Article  Google Scholar 

  15. Forrester LA, McCormick KL, Bonsignore-Opp L, Tedesco LJ, Baranek ES, Jang ES, Tyler WK (2021) Statistical fragility of surgical clinical trials in orthopaedic trauma. J Am Acad Orthop Surg Glob Res Rev 5(e20):00197. https://doi.org/10.5435/JAAOSGlobal-D-20-00197

    Article  Google Scholar 

  16. Gangavalli AK, Nwachuku CO (2016) Management of distal femur fractures in adults: an overview of options. Orthop Clin North Am 47:85–96. https://doi.org/10.1016/j.ocl.2015.08.011

    Article  PubMed  Google Scholar 

  17. Go CC, Maldonado DR, Go BC, Annin S, Saks BR, Jimenez AE, Domb BG (2022) The fragility index of total hip arthroplasty randomized control trials: a systematic review. J Am Acad Orthop Surg 30:e741–e750. https://doi.org/10.5435/JAAOS-D-21-00489

    Article  PubMed  Google Scholar 

  18. Ioannidis JPA (2005) Contradicted and initially stronger effects in highly cited clinical research. JAMA 294:218. https://doi.org/10.1001/jama.294.2.218

    Article  CAS  PubMed  Google Scholar 

  19. Khan M, Evaniew N, Gichuru M, Habib A, Ayeni OR, Bedi A, Walsh M, Devereaux PJ, Bhandari M (2017) The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey. Am J Sports Med 45:2164–2170. https://doi.org/10.1177/0363546516674469

    Article  PubMed  Google Scholar 

  20. Khormaee S, Choe J, Ruzbarsky JJ, Agarwal KN, Blanco JS, Doyle SM, Dodwell ER (2018) The fragility of statistically significant results in pediatric orthopaedic randomized controlled trials as quantified by the fragility index: a systematic review. J Pediatr Orthop 38:e418–e423. https://doi.org/10.1097/BPO.0000000000001201

    Article  PubMed  Google Scholar 

  21. Kubiak EN, Beebe MJ, North K, Hitchcock R, Porter MQ (2013) Early weight bearing after lower extremity fractures in adults. J Am Acad Orthop Surg 21:727–738. https://doi.org/10.5435/JAAOS-21-12-727

    Article  PubMed  Google Scholar 

  22. McCormick KL, Tedesco LJ, Swindell HW, Forrester LA, Jobin CM, Levine WN (2021) Statistical fragility of randomized clinical trials in shoulder arthroplasty. J Shoulder Elbow Surg 30:1787–1793. https://doi.org/10.1016/j.jse.2020.10.028

    Article  PubMed  Google Scholar 

  23. Megafu M, Megafu E (2022) The fragility of statistical findings in distal radius fractures: a systematic review of randomized controlled trials. Injury 53:3352–3356. https://doi.org/10.1016/j.injury.2022.07.017

    Article  PubMed  Google Scholar 

  24. Parisien RL, Trofa DP, Dashe J, Cronin PK, Curry EJ, Fu FH, Li X (2019) Statistical fragility and the role of P values in the sports medicine literature. J Am Acad Orthop Surg 27:e324–e329. https://doi.org/10.5435/JAAOS-D-17-00636

    Article  PubMed  Google Scholar 

  25. Parisien RL, Dashe J, Cronin PK, Bhandari M, Tornetta P 3rd (2019) Statistical significance in trauma research: Too unstable to trust? J Orthop Trauma 33:e466–e470. https://doi.org/10.1097/BOT.0000000000001595

    Article  PubMed  Google Scholar 

  26. Parisien R, Trofa D, Cronin P, Dashe J, Curry E, Eichinger JK, Levine W, Tornetta P 3rd, Li X (2021) Comparative studies in the shoulder literature lack statistical robustness: a fragility analysis. Arthrosc Sports Med Rehabilit 3:e1899–e1904. https://doi.org/10.1016/j.asmr.2021.08.017

    Article  Google Scholar 

  27. Parisien R, Danford N, Jarin IJ, Li X, Trofa DP, Vosseller JT (2021) The fragility of statistical findings in Achilles tendon injury research: a systematic review. J Am Acad of Orthop Surg Glob Res Rev. https://doi.org/10.5435/JAAOSGlobal-D-21-00018

    Article  Google Scholar 

  28. Parisien RL, Ehlers C, Cusano A, Tornetta P 3rd, Li X, Wang D (2021) The Statistical fragility of platelet-rich plasma in rotator cuff surgery: a systematic review and meta-analysis. Am J Sports Med 49:3437–3442. https://doi.org/10.1177/0363546521989976

    Article  PubMed  Google Scholar 

  29. Parisien RL, Constant M, Saltzman BM, Popkin CA, Ahmad CS, Li X, Trofa DP (2021) The fragility of statistical significance in cartilage restoration of the knee: a systematic review of randomized controlled trials. Cartilage 3:147S-155S. https://doi.org/10.1177/19476035211012458

    Article  Google Scholar 

  30. Parisien RL, Trofa DP, O’Connor M, Knapp B, Curry EJ, Tornetta P 3rd, Lynch TS, Li X (2021) The fragility of significance in the hip arthroscopy literature: a systematic review. JBJS Open Access 6(e21):00035. https://doi.org/10.2106/JBJS.OA.21.00035

    Article  Google Scholar 

  31. Pocock S (1985) Current issues in the design and interpretation of clinical trials. Br Med J (Clin Res Ed) 290:39–42. https://doi.org/10.1136/bmj.290.6461.39

    Article  CAS  PubMed  Google Scholar 

  32. Salazar BP, Babian AR, DeBaun MR, Githens MF, Chavez GA, Goodnough LH, Gardner MJ, Bishop JA (2021) Distal femur replacement versus surgical fixation for the treatment of geriatric distal femur fractures: a systematic review. J Orthop Trauma 35:2–9. https://doi.org/10.1097/BOT.0000000000001867

    Article  PubMed  Google Scholar 

  33. Son DW, Kim HS, Choi WY (2018) Risk factors for knee stiffness in distal femoral fractures. J Korean Fract Soc 31:123–131. https://doi.org/10.12671/jkfs.2018.31.4.123

    Article  Google Scholar 

  34. Sterne JA, Davey Smith G (2001) Sifting the evidence-what’s wrong with significance tests? BMJ 322:226–231. https://doi.org/10.1136/bmj.322.7280.226

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Strage KE, Parry JA, Mauffrey C (2021) Standardizing statistics and data reporting in orthopaedic research. Eur J Orthop Surg Traumatol 31(1):1–6. https://doi.org/10.1007/s00590-020-02843-8 (Epub 2021 Jan 3 PMID: 33389055)

    Article  PubMed  Google Scholar 

  36. Strage KE, Stacey SC, Mauffrey C, Parry JA (2022) The interobserver reliability of clinical relevance in orthopaedic research. Eur J Orthop Surg Traumatol. https://doi.org/10.1007/s00590-022-03346-4 (Epub ahead of print. PMID: 35922640)

    Article  PubMed  Google Scholar 

  37. Thomson AB, Driver R, Kregor PJ, Obremskey WT (2008) Long-Term functional outcomes after intra-articular distal femur fractures: ORIF versus retrograde intramedullary nailing. Orthopedics 31:748–750. https://doi.org/10.3928/01477447-20080801-33

    Article  PubMed  Google Scholar 

  38. von Keudell A, Shoji K, Nasr M, Lucas R, Dolan R, Weaver MJ (2016) Treatment Options for Distal Femur Fractures. J Orthop Trauma 30(Suppl 2):S25–S27. https://doi.org/10.1097/BOT.0000000000000621

    Article  Google Scholar 

Download references

Funding

Robert Parisien, MD has received Grant support, hospitality, financial or material support from Anthrex Inc. Robert Parisien services committee roles and editorial roles on the following boards: American Orthopaedic Society for Sports Medicine, American Orthopaedic Society for Sports Medicine, Arthroscopy Association of North America, Arthroscopy Association of North America, Society of Military Orthopaedic Surgeons, Journal of Cartilage and Joint Preservation, Arthroscopy, Arthroscopy, Sports Medicine and Rehabilitation and Journal of Sport Rehabilitation. Paul Tornetta III, MD has received publishing royalties, financial and material from Wolters Kluwer Health—Lippincott Williams and Wilkins. Paul Tornetta III, MD also receives royalties from Smith & Nephew. Paul Tornetta III, MD is currently on the editorial and governing board for the Journal of Orthopedic Trauma.

Author information

Authors and Affiliations

  1. Kirksville College of Osteopathic Medicine, A.T. Still University, Kirksville, MO, USA

    Michael Megafu

  2. University of Minnesota Medical School, Twin Cities Campus, Minneapolis, MN, USA

    Hassan Mian

  3. York College, City University of New York, Jamaica, NY, USA

    Emmanuel Megafu

  4. Drexel University College of Medicine, Philadelphia, PA, USA

    Sulabh Singhal

  5. University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA

    Alexander Lee

  6. Department of Orthopedic Surgery, Mount Sinai Hospital, New York, NY, USA

    Richawna Cassie & Robert Parisien

  7. Department of Orthopedic Surgery, Boston University School of Medicine, Boston, MA, USA

    Paul Tornetta III

Authors
  1. Michael Megafu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hassan Mian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emmanuel Megafu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sulabh Singhal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexander Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Richawna Cassie
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Paul Tornetta III
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Robert Parisien
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Megafu.

Ethics declarations

Conflict of interest

Michael Megafu MPH, Hassan Mian MBA, Emmanuel Megafu BA, Sulabh Singhal BS, Alexander Lee BS, Richawna Cassie MA have no relevant financial or non-financial interests to disclose. The authors did not receive support from any organization for the submitted work.

Human or animal rights

No animals or humans were used in this study and the Institutional Review Board was not notified due to the nature of the article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Megafu, M., Mian, H., Megafu, E. et al. The fragility of statistical significance in distal femur fractures: systematic review of randomized controlled trials. Eur J Orthop Surg Traumatol 33, 2411–2418 (2023). https://doi.org/10.1007/s00590-022-03452-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00590-022-03452-3

Keywords

Search

Navigation