Advertisement

Journal of General Internal Medicine

, Volume 34, Issue 7, pp 1236–1243 | Cite as

The Fragility Index in a Cohort of HIV/AIDS Randomized Controlled Trials

  • Cole WayantEmail author
  • Chase Meyer
  • Rebecca Gupton
  • Mousumi Som
  • Damon Baker
  • Matt Vassar
Original Research

Abstract

HIV/AIDS is associated with significant morbidity, mortality, and financial burden. For these reasons, robust clinical evidence is critical. We aim to investigate the fragility index, fragility quotient, and risk of bias of clinical trial endpoints in HIV medicine. The fragility index represents the minimum amount of trial endpoint “nonevents” changed to “events” in one trial arm required to nullify statistical significance. The fragility quotient contextualized the fragility index by dividing the index by the total trial sample size. We selected eligible trials from the Department of Health and Human Services guideline for the use of antiretroviral agents in HIV-1-infected adults and adolescents. We calculated the fragility index and fragility quotient for all included trials. The Cochrane “risk of bias” Tool 2.0 was used to evaluate the likelihood and sources of bias in the included trials. Thirty-nine RCTs were included for our analysis of fragility. Thirty-six were included for our analysis of the risk of bias. The median fragility index was 5. Three RCTs were at high risk of bias, all due to the selection of the endpoint or statistical test. Twenty had some concerns for risk of bias. The analyzed HIV medicine RCT endpoints were fragile, overall. This indicates that a median of 5 patients across all included studies would nullify the statistical significance of the endpoints. Furthermore, we found evidence that concerns for bias are present at a high rate.

KEY WORDS

HIV acquired immunodeficiency syndrome randomized controlled trials fragility index fragility quotient 

Notes

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Supplementary material

11606_2019_4928_MOESM1_ESM.docx (20 kb)
Supplementary Table 1 (DOCX 20 kb)

References

  1. 1.
    Basic Statistics | HIV Basics | HIV/AIDS | CDC. https://www.cdc.gov/hiv/basics/statistics.html. Accessed 2 August 2017.
  2. 2.
    Gebo KA, Fleishman JA, Conviser R, et al. Contemporary costs of HIV healthcare in the HAART era. AIDS. 2010;24(17):2705–2715.CrossRefGoogle Scholar
  3. 3.
    Schackman BR, Gebo KA, Walensky RP, et al. The lifetime cost of current human immunodeficiency virus care in the United States. Med Care. 2006;44(11):990–997.CrossRefGoogle Scholar
  4. 4.
    Panel on Clinical Practices for Treatment of HIV Infection. Guidelines for the Use of Antiretroviral Agents in HIV-1-Infected Adults and Adolescents. PsycEXTRA Dataset.  https://doi.org/10.1037/e373632004-001
  5. 5.
    Sibbald B, Roland M. Understanding controlled trials. Why are randomised controlled trials important? BMJ. 1998;316(7126):201.CrossRefGoogle Scholar
  6. 6.
    Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol. 2014;67(6):622–628.CrossRefGoogle Scholar
  7. 7.
    Ahmed W, Fowler RA, McCredie VA. Does Sample Size Matter When Interpreting the Fragility Index? Crit Care Med. 2016;44(11):e1142-e1143.CrossRefGoogle Scholar
  8. 8.
    Strategies for Management of Antiretroviral Therapy (SMART) Study Group, El-Sadr WM, Lundgren JD, et al. CD4+ count-guided interruption of antiretroviral treatment. N Engl J Med. 2006;355(22):2283–2296.CrossRefGoogle Scholar
  9. 9.
    Bunupuradah T, Chetchotisakd P, Ananworanich J, et al. A randomized comparison of second-line lopinavir/ritonavir monotherapy versus tenofovir/lamivudine/lopinavir/ritonavir in patients failing NNRTI regimens: the HIV STAR study. Antivir Ther. 2012;17(7):1351–1361.CrossRefGoogle Scholar
  10. 10.
    Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64(4):395–400.CrossRefGoogle Scholar
  11. 11.
    Suresh K, Chandrashekara S. Sample size estimation and power analysis for clinical research studies. J Hum Reprod Sci. 2012;5(1):7–13.CrossRefGoogle Scholar
  12. 12.
    Button KS, Ioannidis JPA, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14(5):365–376.CrossRefGoogle Scholar
  13. 13.
    Gelman A. Post-hoc Power Using Observed Estimate of Effect Size is too Noisy to be Useful. Ann Surg. 2018.  https://doi.org/10.1097/SLA.0000000000003089.
  14. 14.
    Ridgeon EE, Young PJ, Bellomo R, Mucchetti M, Lembo R, Landoni G. The Fragility Index in Multicenter Randomized Controlled Critical Care Trials. Crit Care Med. 2016;44(7):1278–1284.CrossRefGoogle Scholar
  15. 15.
    Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol. 2014;67(6):622–628.CrossRefGoogle Scholar
  16. 16.
    Shochet LR, Kerr PG, Polkinghorne KR. The fragility of significant results underscores the need of larger randomized controlled trials in nephrology. Kidney Int. 2017.  https://doi.org/10.1016/j.kint.2017.05.011.
  17. 17.
    Evaniew N, Files C, Smith C, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J. 2015;15(10):2188–2197.CrossRefGoogle Scholar
  18. 18.
    Khan M, Evaniew N, Gichuru M, et al. The Fragility of Statistically Significant Findings From Randomized Trials in Sports Surgery: A Systematic Survey. Am J Sports Med. 2017;45(9):2164–2170.CrossRefGoogle Scholar
  19. 19.
    Matics TJ, Khan N, Jani P, Kane JM. The Fragility Index in a Cohort of Pediatric Randomized Controlled Trials. J Clin Med Res. 2017;6(8).  https://doi.org/10.3390/jcm6080079.
  20. 20.
    Shen Y, Cheng X, Zhang W. The fragility of randomized controlled trials in intracranial hemorrhage. Neurosurg Rev. 2017.  https://doi.org/10.1007/s10143-017-0870-8
  21. 21.
    Bowers A, Meyer C, Tritz D, et al. Assessing quality of randomized trials supporting guidelines for laparoscopic and endoscopic surgery. J Surg Res. 2018;224:233–239.CrossRefGoogle Scholar
  22. 22.
    Carter RE, McKie PM, Storlie CB. The Fragility Index: a P-value in sheep’s clothing? Eur Heart J. 2017;38(5):346–348.Google Scholar
  23. 23.
    Benjamin DJ, Berger JO, Johannesson M, et al. Redefine statistical significance. Nat Hum Behav. 2017;2(1):6–10.CrossRefGoogle Scholar
  24. 24.
    Wayant C, Scott J, Vassar M. Evaluation of Lowering the P Value Threshold for Statistical Significance From .05 to .005 in Previously Published Randomized Clinical Trials in Major Medical Journals. JAMA. 2018;320(17):1813–1815.CrossRefGoogle Scholar
  25. 25.
    Lakens D, Adolfi FG, Albers CJ, et al. Justify your alpha. Nat Hum Behav. 2018;2(3):168–171.CrossRefGoogle Scholar
  26. 26.
    Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of p-hacking in science. PLoS Biol. 2015;13(3):e1002106.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2019

Authors and Affiliations

  • Cole Wayant
    • 1
    Email author
  • Chase Meyer
    • 1
    • 2
  • Rebecca Gupton
    • 3
  • Mousumi Som
    • 3
  • Damon Baker
    • 3
  • Matt Vassar
    • 1
    • 2
  1. 1.Department of Biomedical SciencesOklahoma State University Center for Health SciencesTulsaUSA
  2. 2.Department of Psychiatry and Behavioral SciencesOklahoma State University Center for Health SciencesTulsaUSA
  3. 3.Internal MedicineOklahoma State University Medical CenterTulsaUSA

Personalised recommendations