Skip to main content
Log in

The fragility of randomized controlled trials in intracranial hemorrhage

  • Review
  • Published:
Neurosurgical Review Aims and scope Submit manuscript

Abstract

Fragility of randomized controlled trials (RCTs) has been evaluated using a novel metric called fragility index (FI), which measures how many events the statistical significance of a dichotomous outcome depends on. This study aimed to evaluate the fragility of RCTs in intracranial hemorrhage. Literature search (PubMed/Embase) identified all RCTs of intracranial hemorrhage since 2006. The overall distribution of FI was evaluated. Subgroup and spearman correlation analyses were made to explore potential factors that may affect FI value. All the included RCTs were divided into two groups (positive and negative trials) according to the statistical significance of selected outcomes. Finally, 47 positive and 51 negative trials were included. Both the median FI ([2; IQR, 1–4] vs. [6; IQR, 4–9], p < 0.001) and the proportion of trials with FI ≤1 (2 vs. 18, p < 0.001) in positive trials were smaller than negative trials. In subgroup comparison within positive trials, sample size ([165; IQR, 87–200] vs. [83; IQR, 60–120], p = 0.015) and number of events ([35; IQR, 20–72] vs. [24; IQR, 11–32], p = 0.015) were higher in subgroup with FI >1 than the subgroup with FI ≤1. Weak positive correlations were found between FI and sample size and number of events. In the field of intracranial hemorrhage, trials reporting significant conclusions often depend on a small number of events. Compared to sample size, this phenomenon is more likely to be affected by statistical approach and trial methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Bath PM, Gray LJ, Collier T, Pocock S, Carpenter J (2007) Can we improve the statistical analysis of stroke trials? Statistical reanalysis of functional outcomes in stroke trials. Stroke 38:1911–1915. doi:10.1161/STROKEAHA.106.474080

    Article  PubMed  Google Scholar 

  2. Chakkera HA, Schold JD, Kaplan B (2016) P value: significance is not all black and white. Transplantation 100:1607–1609. doi:10.1097/TP.0000000000001331

    Article  PubMed  Google Scholar 

  3. Evaniew N, Files C, Smith C, Bhandari M, Ghert M, Walsh M, Devereaux PJ, Guyatt G (2015) The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. The spine journal : official journal of the North American Spine Society 15:2188–2197. doi:10.1016/j.spinee.2015.06.004

    Article  Google Scholar 

  4. Li JY, Yuan LX, Zhang GM, Zhou L, Gao Y, Li QB, Chen C (2016) Activating blood circulation to remove stasis treatment of hypertensive intracerebral hemorrhage: a multi-center prospective randomized open-label blinded-endpoint trial. Chinese Journal of Integrative Medicine 22:328–334

    Article  PubMed  Google Scholar 

  5. Pocock SJ, Stone GW (2016) The primary outcome is positive—is that good enough? N Engl J Med 375:971–979. doi:10.1056/NEJMra1601511

    Article  PubMed  Google Scholar 

  6. Ridgeon EE, Young PJ, Bellomo R, Mucchetti M, Lembo R, Landoni G (2016) The fragility index in multicenter randomized controlled critical care trials. Crit Care Med 44:1278–1284. doi:10.1097/CCM.0000000000001670

    Article  PubMed  Google Scholar 

  7. Shepherd BE, Shaw PA, Dodd LE (2012) Using audit information to adjust parameter estimates for data errors in clinical trials. Clin Trials 9:721–729. doi:10.1177/1740774512450100

    Article  PubMed  PubMed Central  Google Scholar 

  8. Thiese MS, Ronna B, Ott U (2016) P value interpretations and considerations. J Thorac Dis 8:E928-E931. doi:10.21037/jtd.2016.08.16

  9. Thomas LE, Pencina MJ (2016) Do not over (P) value your research article. JAMA cardiology 1:1055. doi:10.1001/jamacardio.2016.3827

    Article  PubMed  Google Scholar 

  10. Walsh M, Srinathan SK, McAuley DF, Mrkobrada M, Levine O, Ribic C, Molnar AO, Dattani ND, Burke A, Guyatt G, Thabane L, Walter SD, Pogue J, Devereaux PJ (2014) The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol 67:622–628. doi:10.1016/j.jclinepi.2013.10.019

    Article  PubMed  Google Scholar 

  11. Woods KL, Fletcher S, Roffe C, Haider Y (1992) Intravenous magnesium sulphate in suspected acute myocardial infarction: results of the second Leicester Intravenous Magnesium Intervention Trial (LIMIT-2). Lancet (London) 339:1553-1558

Download references

Acknowledgements

We thank editor Monica Liu of JECCM for her help in revision.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yanfei Shen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This is a systematic review and ethics committee approval was waived for this paper.

Informed consent

None.

Electronic supplementary material

ESM 1

(PDF 509 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, Y., Cheng, X. & Zhang, W. The fragility of randomized controlled trials in intracranial hemorrhage. Neurosurg Rev 42, 9–14 (2019). https://doi.org/10.1007/s10143-017-0870-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10143-017-0870-8

Keywords

Navigation