Abstract
Fragility of randomized controlled trials (RCTs) has been evaluated using a novel metric called fragility index (FI), which measures how many events the statistical significance of a dichotomous outcome depends on. This study aimed to evaluate the fragility of RCTs in intracranial hemorrhage. Literature search (PubMed/Embase) identified all RCTs of intracranial hemorrhage since 2006. The overall distribution of FI was evaluated. Subgroup and spearman correlation analyses were made to explore potential factors that may affect FI value. All the included RCTs were divided into two groups (positive and negative trials) according to the statistical significance of selected outcomes. Finally, 47 positive and 51 negative trials were included. Both the median FI ([2; IQR, 1–4] vs. [6; IQR, 4–9], p < 0.001) and the proportion of trials with FI ≤1 (2 vs. 18, p < 0.001) in positive trials were smaller than negative trials. In subgroup comparison within positive trials, sample size ([165; IQR, 87–200] vs. [83; IQR, 60–120], p = 0.015) and number of events ([35; IQR, 20–72] vs. [24; IQR, 11–32], p = 0.015) were higher in subgroup with FI >1 than the subgroup with FI ≤1. Weak positive correlations were found between FI and sample size and number of events. In the field of intracranial hemorrhage, trials reporting significant conclusions often depend on a small number of events. Compared to sample size, this phenomenon is more likely to be affected by statistical approach and trial methodology.
Similar content being viewed by others
References
Bath PM, Gray LJ, Collier T, Pocock S, Carpenter J (2007) Can we improve the statistical analysis of stroke trials? Statistical reanalysis of functional outcomes in stroke trials. Stroke 38:1911–1915. doi:10.1161/STROKEAHA.106.474080
Chakkera HA, Schold JD, Kaplan B (2016) P value: significance is not all black and white. Transplantation 100:1607–1609. doi:10.1097/TP.0000000000001331
Evaniew N, Files C, Smith C, Bhandari M, Ghert M, Walsh M, Devereaux PJ, Guyatt G (2015) The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. The spine journal : official journal of the North American Spine Society 15:2188–2197. doi:10.1016/j.spinee.2015.06.004
Li JY, Yuan LX, Zhang GM, Zhou L, Gao Y, Li QB, Chen C (2016) Activating blood circulation to remove stasis treatment of hypertensive intracerebral hemorrhage: a multi-center prospective randomized open-label blinded-endpoint trial. Chinese Journal of Integrative Medicine 22:328–334
Pocock SJ, Stone GW (2016) The primary outcome is positive—is that good enough? N Engl J Med 375:971–979. doi:10.1056/NEJMra1601511
Ridgeon EE, Young PJ, Bellomo R, Mucchetti M, Lembo R, Landoni G (2016) The fragility index in multicenter randomized controlled critical care trials. Crit Care Med 44:1278–1284. doi:10.1097/CCM.0000000000001670
Shepherd BE, Shaw PA, Dodd LE (2012) Using audit information to adjust parameter estimates for data errors in clinical trials. Clin Trials 9:721–729. doi:10.1177/1740774512450100
Thiese MS, Ronna B, Ott U (2016) P value interpretations and considerations. J Thorac Dis 8:E928-E931. doi:10.21037/jtd.2016.08.16
Thomas LE, Pencina MJ (2016) Do not over (P) value your research article. JAMA cardiology 1:1055. doi:10.1001/jamacardio.2016.3827
Walsh M, Srinathan SK, McAuley DF, Mrkobrada M, Levine O, Ribic C, Molnar AO, Dattani ND, Burke A, Guyatt G, Thabane L, Walter SD, Pogue J, Devereaux PJ (2014) The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol 67:622–628. doi:10.1016/j.jclinepi.2013.10.019
Woods KL, Fletcher S, Roffe C, Haider Y (1992) Intravenous magnesium sulphate in suspected acute myocardial infarction: results of the second Leicester Intravenous Magnesium Intervention Trial (LIMIT-2). Lancet (London) 339:1553-1558
Acknowledgements
We thank editor Monica Liu of JECCM for her help in revision.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This is a systematic review and ethics committee approval was waived for this paper.
Informed consent
None.
Electronic supplementary material
ESM 1
(PDF 509 kb)
Rights and permissions
About this article
Cite this article
Shen, Y., Cheng, X. & Zhang, W. The fragility of randomized controlled trials in intracranial hemorrhage. Neurosurg Rev 42, 9–14 (2019). https://doi.org/10.1007/s10143-017-0870-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10143-017-0870-8