Avoid common mistakes on your manuscript.
Sir,
We truly appreciate that Saikia & Thakuria have found the time during this devastating pandemic to respond to our manuscript. It is a pleasure for us to respond to their comments to the editor in this issue of Paediatric Surgery International [1], which provides us with the valuable opportunity to point out some issues that may not have become clear enough in our initial contribution [2].
We feel that transparency is crucial within the scientific discourse [3,4,5,6], and we agree that the search strategy should be readily available. The list of included meta-analyses and the extracted data [7] have been put into a repository and cited in reference 35 of our initial contribution, so that the readership is able to double-check and reproduce our results.
The included meta-analyses have purposefully not been limited to studies with a 1:1 allocation ratio. The fragility index has been extended beyond this limitation before [8,9,10], and we have thus extended its applicability as well. Let us assume that we would have limited our analysis [2] to meta-analyses with these primary study properties: the underlying numbers would have possibly changed slightly, but it would not have changed our conclusion.
We may use the example by Potter [11] to clarify this: in a hypothetical trial A 1 of 100 patients experienced a negative event in the treatment group and 9 of 100 patients did so in the control group, which resulted in a relative risk of 0.11 in the treatment group compared to the control group with P = 0.02. In another hypothetical trial B, 200 of 4000 patients in the treatment group and 250 of 4000 patients in the control group experienced a negative event, resulting in a relative risk of 0.8, also with P = 0.02. The fragility index of the hypothetical trial A is 1, but the hypothetical trial B has a fragility index of 9. If we plot the consonance curves of the P-values of both studies using R’s [12] concurve-package (version 2.7.7) [13] (Fig. 1), it becomes obvious that hypothetical trial A is farther from the null effect of a relative risk of 1 compared to hypothetical trial B. Due to the much smaller sample size—reflected in the Fig. 1 by the wider confidence intervals—the very same P value of 0.02 in both studies can only occur due to a stronger effect. This relevant clinical difference between both trials is not captured by the fragility index, which considers the more relevant trial to be more “fragile” due to smaller number of events necessary to overturn the statistical significance.
We are aware that this is a rather technical argumentation, but as the recent controversy around posthoc-power [14] has shown, statistical details matter. There is an intrinsic fallacy regarding the concept of fragility in our specialty. Research in paediatric surgery is limited by the small numbers of major congenital malformations, the rarity of events [15]. Thus, paediatric surgery will always be penalised by the fragility index due to the diseases it deals with, which can be demonstrated by the example of severe isolated congenital diaphragmatic hernia: a recent randomised controlled trial for this highly effective treatment [16, 17] randomised 20 patients to foetal endoscopic tracheal occlusion and 21 patients to standard postnatal care. In the intention-to-treat-analysis, the survival rate at the age of six months with fetoscopic treatment was 50% (10/20), but 4.8% (1/21) in the control group (P = 0.0036) [18]. The trial would still be considered fragile, because it has a fragility index of 3, despite a massive clinical difference of a tenfold relative risk of death in the control group compared to foetal endoscopic tracheal occlusion.
We therefore thank Saikia & Thakuria for the opportunity to emphasise this highly relevant point of penalisation of small studies by the fragility index for paediatric surgeons with a relevant example. Therefore, our criticism of the fragility index was based on its statistical properties, whose inadequacies have been exposed by simulation studies before [11, 19, 20]. We therefore stand by our conclusion that the fragility index should be interpreted critically and may not be appropriate for paediatric surgery.
References
Saikia P, Thakuria B (2021) Fragility index of meta-analyses in paediatric surgery. Pediatr Surg Int. https://doi.org/10.1007/s00383-021-04918-x
Schröder A, Muensterer OJ, Oetzmann von Sochaczewski C (2021) Meta-analyses in paediatric surgery are often fragile: implications and consequences. Pediatr Surg Int 37:363–367. https://doi.org/10.1007/s00383-020-04827-5
Nuber M, Lindner A, Baumgart J et al (2020) Sex represents a relevant interaction in Sprague-Dawley rats: the example of oesophageal length*. All Life 13:448–455. https://doi.org/10.1080/26895293.2020.1806118
Schröder A, Gerin A, Firth GB et al (2019) A systematic review of necrotising fasciitis in children from its first description in 1930 to 2018. BMC Infect Dis 19:317. https://doi.org/10.1186/s12879-019-3941-3
Oetzmann von Sochaczewski C, Tagkalos E, Lindner A et al (2019) Esophageal biomechanics revisited: a tale of tenacity, anastomoses, and suture bite lengths in swine. Ann Thorac Surg 107:1670–1677. https://doi.org/10.1016/j.athoracsur.2018.12.009
Baumgart J, Deigendesch N, Lindner A et al (2020) Using multidimensional scaling in model choice for congenital oesophageal atresia: similarity analysis of human autopsy organ weights with those from a comparative assessment of Aachen Minipig and Pietrain piglets. Lab Anim 54:576–587. https://doi.org/10.1177/0023677220902184
Schröder A, Muensterer OJ, Oetzmann von Sochaczewski C (2020) Meta-analyses in paediatric surgery are often fragile: implications and consequences. https://zenodo.org/record/4275919
Bertaggia L, Baiardo Redaelli M, Lembo R et al (2019) The Fragility Index in peri-operative randomised trials that reported significant mortality effects in adults. Anaesthesia 74:1057–1060. https://doi.org/10.1111/anae.14656
Grammatikopoulou MG, Nigdelis MP, Theodoridis X et al (2021) How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials. BMJ Nutr. https://doi.org/10.1136/bmjnph-2020-000188
Xing A, Chu H, Lin L (2020) Fragility index of network meta-analysis with application to smoking cessation data. J Clin Epidemiol 127:29–39. https://doi.org/10.1016/j.jclinepi.2020.07.003
Potter GE (2020) Dismantling the Fragility index: a demonstration of statistical reasoning. Stat Med 39:3720–3731. https://doi.org/10.1002/sim.8689
R Core Team (2019) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna. https://r-project.org
Rafi Z, Vigotsky AD, Caldwell A (2020) concurve: computes & plots compatibility (confidence), surprisal, & likelihood distributions. https://CRAN.r-project.org/package=concurve
Schröder A, Oetzmann von Sochaczewski C (2020) On the Difference between a-priori and observed statistical power—a comment on “statistical power and sample size calculations: a primer for pediatric surgeons.” J Pediatr Surg 55:203–205. https://doi.org/10.1016/j.jpedsurg.2019.08.054
Curry JI, Reeves B, Stringer MD (2003) Randomized controlled trials in pediatric surgery: could we do better? J Pediatr Surg 38:556–559. https://doi.org/10.1053/jpsu.2003.50121
Harrison MR (2011) Tracheal occlusion works. Fetal Diagn Ther 29:78–79. https://doi.org/10.1159/000314618
Al-Maary J, Eastwood MP, Russo FM et al (2016) Fetal tracheal occlusion for severe pulmonary hypoplasia in isolated congenital diaphragmatic hernia: a systematic review and meta-analysis of survival. Ann Surg 264:929–933. https://doi.org/10.1097/SLA.0000000000001675
Ruano R, Yoshisaki CT, da Silva MM et al (2012) A randomized controlled trial of fetal endoscopic tracheal occlusion versus postnatal management of severe isolated congenital diaphragmatic hernia. Ultrasound Obstet Gynecol 39:20–27. https://doi.org/10.1002/uog.10142
Carter RE, McKie PM, Storlie CB (2017) The fragility index: a P-value in sheep’s clothing? Eur Heart J 38:346–348. https://doi.org/10.1093/eurheartj/ehw495
Reito A, Raittio L, Helminen O (2019) Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery. PeerJ 7:e6813. https://doi.org/10.7717/peerj.6813
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Schröder, A., Muensterer, O.J. & Oetzmann von Sochaczewski, C. The fragility index may not be ideal for paediatric surgical conditions: the example of foetal endoscopic tracheal occlusion. Pediatr Surg Int 37, 967–969 (2021). https://doi.org/10.1007/s00383-021-04926-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00383-021-04926-x