Skip to main content

Advertisement

SpringerLink
Log in

Fragility of statistically significant findings from randomized trials in comparing laparoscopic versus robotic abdominopelvic surgeries

  • Review Article
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Utility of robotic over laparoscopic approach has been an area of debate across all surgical specialties over the past decade. The fragility index (FI) is a metric that evaluates the frailty of randomized controlled trials (RCTs) findings by altering the status of patients from an event to non-event until significance is lost. This study aims to evaluate the robustness of RCTs comparing laparoscopic and robotic abdominopelvic surgeries through the FI.

Methods

A search was conducted in MEDLINE and EMBASE for RCTs with dichotomous outcomes comparing laparoscopic and robot-assisted surgery in general surgery, gynecology, and urology. The FI and reverse fragility Index (RFI) metrics were used to assess the strength of findings reported by RCTs, and bivariate correlation was conducted to analyze relationships between FI and trial characteristics.

Results

A total of 21 RCTs were included, with a median sample size of 89 participants (Interquartile range [IQR] 62–126). The median FI was 2 (IQR 0–15) and median RFI 5.5 (IQR 4–8.5). The median FI was 3 (IQR 1–15) for general surgery (n = 7), 2 (0.5–3.5) for gynecology (n = 4), and 0 (IQR 0–8.5) for urology RCTs (n = 4). Correlation was found between increasing FI and decreasing p-value, but not sample size, number of outcome events, journal impact factor, loss to follow-up, or risk of bias.

Conclusion

RCTs comparing laparoscopic and robotic abdominal surgery did not prove to be very robust. While possible advantages of robotic surgery may be emphasized, it remains novel and requires further concrete RCT data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Wang T, Wang Q, Wang S (2019) A meta-analysis of robot assisted laparoscopic radical prostatectomy versus laparoscopic radical prostatectomy. Open Med 14:485. https://doi.org/10.1515/MED-2019-0052

    Article  Google Scholar 

  2. Marchand G, Taher Masoud A, Ware K et al (2021) Systematic review and meta-analysis of all randomized controlled trials comparing gynecologic laparoscopic procedures with and without robotic assistance. European J Obstetrics Gynecol Rep Biol 265:30–38. https://doi.org/10.1016/J.EJOGRB.2021.07.038

    Article  Google Scholar 

  3. Xiong J, Nunes QM, Tan C et al (2013) Comparison of short-term clinical outcomes between robotic and laparoscopic gastrectomy for gastric cancer: a meta-analysis of 2495 patients. J Laparoendosc Adv Surg Tech A 23:965–976. https://doi.org/10.1089/LAP.2013.0279

    Article  PubMed  Google Scholar 

  4. Hyun MH, Lee CH, Kim HJ et al (2013) Systematic review and meta-analysis of robotic surgery compared with conventional laparoscopic and open resections for gastric carcinoma. Br J Surg 100:1566–1578. https://doi.org/10.1002/BJS.9242

    Article  CAS  PubMed  Google Scholar 

  5. Köckerling F (2014) Robotic vs Standard Laparoscopic Technique–What is Better? Front Surg. https://doi.org/10.3389/FSURG.2014.00015

    Article  PubMed  PubMed Central  Google Scholar 

  6. Camarillo DB, Krummel TM, Salisbury JK (2004) Robotic technology in surgery: past, present, and future. Am J Surg 188:2–15. https://doi.org/10.1016/J.AMJSURG.2004.08.025

    Article  Google Scholar 

  7. Sodergren MH, Darzi A (2012) Robotic cancer surgery. Br J Surg 100:3–4. https://doi.org/10.1002/BJS.8972

    Article  PubMed  Google Scholar 

  8. Wilson EB, Wilson EB (2009) The evolution of robotic general surgery. Scand J Surg 98:125–129. https://doi.org/10.1177/145749690909800208

    Article  CAS  PubMed  Google Scholar 

  9. Song SH, Kim KS (2014) Current Status of robot-assisted laparoscopic surgery in pediatric urology. Korean J Urol 55:499. https://doi.org/10.4111/KJU.2014.55.8.499

    Article  PubMed  PubMed Central  Google Scholar 

  10. Baccaglini W, Medina L, Azhar RA, Sotelo RJ (2019) Complications of robotic surgery in urological diseases: are we using standardized methodology to report complications? Curr Opin Urol 29:19–24. https://doi.org/10.1097/MOU.0000000000000568

    Article  PubMed  Google Scholar 

  11. Sinha R, Sanjay M, Rupa B, Kumari S (2015) Robotic surgery in gynecology. J Minim Access Surg 11:50. https://doi.org/10.4103/0972-9941.147690

    Article  PubMed  PubMed Central  Google Scholar 

  12. Azawi NH, Rohrsted M, Poulsen J et al (2019) Robotic versus laparoscopic urological surgery: incidence of reoperation and complications. Scand J Urol 53:56–61. https://doi.org/10.1080/21681805.2019.1588918

    Article  PubMed  Google Scholar 

  13. Dhanani NH, Olavarria OA, Bernardi K et al (2021) The evidence behind robot-assisted abdominopelvic surgery : A systematic review. Ann Intern Med 174:1110–1117. https://doi.org/10.7326/M20-7006

    Article  PubMed  Google Scholar 

  14. Farrokhyar F, Karanicolas PJ, Thoma A et al (2010) Randomized controlled trials of surgical interventions. Ann Surg 251:400–416. https://doi.org/10.1097/SLA.0B013E3181CF863D

    Article  Google Scholar 

  15. McCulloch P, Taylor I, Sasako M et al (2002) Randomised trials in surgery: problems and possible solutions. BMJ 324:1448–1451. https://doi.org/10.1136/BMJ.324.7351.1448

    Article  PubMed  PubMed Central  Google Scholar 

  16. Walsh M, Srinathan SK, McAuley DF et al (2014) The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol 67:622–628. https://doi.org/10.1016/J.JCLINEPI.2013.10.019

    Article  PubMed  Google Scholar 

  17. Guyatt GH, Oxman AD, Kunz R et al (2011) GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol 64:395–400. https://doi.org/10.1016/J.JCLINEPI.2010.09.012

    Article  PubMed  Google Scholar 

  18. Higgins JPT, Altman DG, Gøtzsche PC et al (2011) The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. https://doi.org/10.1136/BMJ.D5928

    Article  PubMed  PubMed Central  Google Scholar 

  19. Khan MS, Fonarow GC, Friede T et al (2020) Application of the reverse fragility index to statistically nonsignificant randomized clinical trial results. JAMA Netw Open 3:e2012469–e2012469. https://doi.org/10.1001/JAMANETWORKOPEN.2020.12469

    Article  PubMed  PubMed Central  Google Scholar 

  20. Costa TN, Abdalla RZ, Tustumi F et al (2022) Robotic-assisted compared with laparoscopic incisional hernia repair following oncologic surgery: short- and long-term outcomes of a randomized controlled trial. J Robot Surg. https://doi.org/10.1007/S11701-022-01403-Y

    Article  PubMed  Google Scholar 

  21. Ojima T, Nakamura M, Hayata K et al (2021) Short-term outcomes of robotic gastrectomy vs laparoscopic gastrectomy for patients with gastric cancer: a randomized clinical trial. JAMA Surg 156:954–963. https://doi.org/10.1001/JAMASURG.2021.3182

    Article  PubMed  Google Scholar 

  22. Lu J, Zheng CH, Xu B, bin, et al (2021) Assessment of robotic versus laparoscopic distal gastrectomy for gastric cancer: a randomized controlled trial. Ann Surg 273:858–867. https://doi.org/10.1097/SLA.0000000000004466

    Article  PubMed  Google Scholar 

  23. Dhanani NH, Olavarria OA, Holihan JL et al (2021) Robotic versus laparoscopic ventral hernia repair: one-year results from a prospective, multicenter, blinded randomized controlled trial. Ann Surg 273:1076–1080. https://doi.org/10.1097/SLA.0000000000004795

    Article  PubMed  Google Scholar 

  24. Petro CC, Zolin S, Krpata D et al (2021) Patient-reported outcomes of robotic vs laparoscopic ventral hernia repair with intraperitoneal mesh: the prove-it randomized clinical trial. JAMA Surg 156:22–29. https://doi.org/10.1001/JAMASURG.2020.4569

    Article  PubMed  Google Scholar 

  25. Silay MS, Danacioglu O, Ozel K et al (2020) Laparoscopy versus robotic-assisted pyeloplasty in children: preliminary results of a pilot prospective randomized controlled trial. World J Urol 38:1841–1848. https://doi.org/10.1007/S00345-019-02910-8

    Article  PubMed  Google Scholar 

  26. Narducci F, Bogart E, Hebert T et al (2020) Severe perioperative morbidity after robot-assisted versus conventional laparoscopy in gynecologic oncology: Results of the randomized ROBOGYN-1004 trial. Gynecol Oncol 158:382–389. https://doi.org/10.1016/J.YGYNO.2020.05.010

    Article  PubMed  Google Scholar 

  27. Park JS, Kang H, Park SY et al (2019) Long-term oncologic after robotic versus laparoscopic right colectomy: a prospective randomized study. Surg Endosc 33:2975–2981. https://doi.org/10.1007/S00464-018-6563-8

    Article  PubMed  Google Scholar 

  28. Illiano E, Ditonno P, Giannitsas K et al (2019) Robot-assisted vs laparoscopic sacrocolpopexy for high-stage pelvic organ prolapse: a prospective, randomized, single-center study. Urology 134:116–123. https://doi.org/10.1016/J.UROLOGY.2019.07.043

    Article  PubMed  Google Scholar 

  29. Silva SE, A, de Carvalho JPM, Anton C et al (2018) Introduction of robotic surgery for endometrial cancer into a Brazilian cancer service: a randomized trial evaluating perioperative clinical outcomes and costs. Clinics. https://doi.org/10.6061/CLINICS/2017/E522S

    Article  PubMed  PubMed Central  Google Scholar 

  30. Porpiglia F, Fiori C, Bertolo R et al (2018) Five-year outcomes for a prospective randomised controlled trial comparing laparoscopic and robot-assisted radical prostatectomy. Eur Urol Focus 4:80–86. https://doi.org/10.1016/J.EUF.2016.11.007

    Article  PubMed  Google Scholar 

  31. Tolstrup R, Funder JA, Lundbech L et al (2018) Perioperative pain after robot-assisted versus laparoscopic rectal resection. Int J Colorectal Dis 33:285–289. https://doi.org/10.1007/S00384-017-2943-0

    Article  PubMed  Google Scholar 

  32. Deimling TA, Eldridge JL, Riley KA et al (2017) Randomized controlled trial comparing operative times between standard and robot-assisted laparoscopic hysterectomy. Int J Gynaecol Obstet 136:64–69. https://doi.org/10.1002/IJGO.12001

    Article  PubMed  Google Scholar 

  33. Pietrabissa A, Pugliese L, Vinci A et al (2016) Short-term outcomes of single-site robotic cholecystectomy versus four-port laparoscopic cholecystectomy: a prospective, randomized, double-blind trial. Surg Endosc 30:3089–3097. https://doi.org/10.1007/S00464-015-4601-3

    Article  PubMed  Google Scholar 

  34. Mueller ER, Kenton K, Anger JT et al (2016) Cosmetic appearance of port-site scars 1 year after laparoscopic versus robotic sacrocolpopexy: a supplementary study of the access clinical trial. J Minim Invasive Gynecol 23:917–921. https://doi.org/10.1016/J.JMIG.2016.05.001

    Article  PubMed  PubMed Central  Google Scholar 

  35. Mäenpää MM, Nieminen K, Tomás EI et al (2016) Robotic-assisted vs traditional laparoscopic surgery for endometrial cancer: a randomized controlled trial. Am J Obstet Gynecol 215:588.e1-588.e7. https://doi.org/10.1016/J.AJOG.2016.06.005

    Article  PubMed  Google Scholar 

  36. Porpiglia F, Morra I, Lucci Chiarissi M et al (2013) Randomised controlled trial comparing laparoscopic and robot-assisted radical prostatectomy. Eur Urol 63:120–121. https://doi.org/10.1016/J.EURURO.2012.07.007

    Article  Google Scholar 

  37. Park JS, Choi GS, Park SY et al (2012) Randomized clinical trial of robot-assisted versus standard laparoscopic right colectomy. Br J Surg 99:1219–1226. https://doi.org/10.1002/BJS.8841

    Article  CAS  PubMed  Google Scholar 

  38. Asimakopoulos AD, Pereira Fraga CT, Annino F et al (2011) Randomized comparison between laparoscopic and robot-assisted nerve-sparing radical prostatectomy. J Sex Med 8:1503–1512. https://doi.org/10.1111/J.1743-6109.2011.02215.X

    Article  PubMed  Google Scholar 

  39. Müller-Stich BP, Reiter MA, Mehrabi A et al (2009) No relevant difference in quality of life and functional outcome at 12 months’ follow-up-a randomised controlled trial comparing robot-assisted versus conventional laparoscopic Nissen fundoplication. Langenbecks Arch Surg 394:441–446. https://doi.org/10.1007/S00423-008-0446-8

    Article  PubMed  Google Scholar 

  40. Baik SH, Ko YT, Kang CM et al (2008) Robotic tumor-specific mesorectal excision of rectal cancer: short-term outcome of a pilot randomized trial. Surg Endosc 22:1601–1608. https://doi.org/10.1007/S00464-008-9752-Z

    Article  CAS  PubMed  Google Scholar 

  41. Ruzbarsky JJ, Khormaee S, Daluiski A (2019) The fragility index in hand surgery randomized controlled trials. J Hand Surg Am 44:698.e1-698.e7. https://doi.org/10.1016/J.JHSA.2018.10.005

    Article  PubMed  Google Scholar 

  42. Grolleau F, Collins GS, Smarandache A et al (2019) The fragility and reliability of conclusions of anesthesia and critical care randomized trials with statistically significant findings: a systematic review. Crit Care Med 47:456–462. https://doi.org/10.1097/CCM.0000000000003527

    Article  PubMed  Google Scholar 

  43. Muthu S, Ramakrishnan E (2021) Fragility analysis of statistically significant outcomes of randomized control trials in spine surgery. Spine 46:198–208. https://doi.org/10.1097/BRS.0000000000003645

    Article  PubMed  Google Scholar 

  44. Khan M, Evaniew N, Gichuru M et al (2017) The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey. Am J Sports Med 45:2164–2170. https://doi.org/10.1177/0363546516674469/ASSET/IMAGES/LARGE/10.1177_0363546516674469-FIG2.JPEG

    Article  PubMed  Google Scholar 

  45. Forrester LA, McCormick KL, Bonsignore-Opp L et al (2021) Statistical fragility of surgical clinical trials in orthopaedic trauma. J Am Acad Orthop Surg Glob Res Rev. https://doi.org/10.5435/JAAOSGLOBAL-D-20-00197

    Article  PubMed  PubMed Central  Google Scholar 

  46. Mazzinari G, Ball L, Serpa Neto A et al (2018) The fragility of statistically significant findings in randomised controlled anaesthesiology trials: systematic review of the medical literature. Br J Anaesth 120:935–941. https://doi.org/10.1016/J.BJA.2018.01.012

    Article  CAS  PubMed  Google Scholar 

  47. Li A, Javidan AP, Liu E et al (2022) Assessment of the reverse fragility index in vascular surgery randomized controlled trials with statistically nonsignificant primary outcomes. J Vasc Surg 75:e188. https://doi.org/10.1016/j.jvs.2022.03.366

    Article  Google Scholar 

  48. Kyriakides PW, Schultz BJ, Egol K, Leucht P (2021) The fragility and reverse fragility indices of proximal humerus fracture randomized controlled trials: a systematic review. Eur J Trauma Emerg Surg. https://doi.org/10.1007/S00068-021-01684-2

    Article  PubMed  Google Scholar 

  49. Ridgeon EE, Young PJ, Bellomo R et al (2016) The fragility index in multicenter randomized controlled critical care trials. Crit Care Med 44:1278–1284. https://doi.org/10.1097/CCM.0000000000001670

    Article  PubMed  Google Scholar 

  50. Evaniew N, Files C, Smith C et al (2015) The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J 15:2188–2197. https://doi.org/10.1016/J.SPINEE.2015.06.004

    Article  PubMed  Google Scholar 

  51. Noel CW, McMullen C, Yao C et al (2018) The fragility of statistically significant findings from randomized trials in head and neck surgery. Laryngoscope 128:2094–2100. https://doi.org/10.1002/LARY.27183

    Article  PubMed  Google Scholar 

  52. Goerke K, Parke M, Horn J et al (2020) Are results from randomized trials in anesthesiology robust or fragile? An analysis using the fragility index. Int J Evid Based Healthc 18:116–124. https://doi.org/10.1097/XEB.0000000000000200

    Article  PubMed  Google Scholar 

  53. Maeso S, Reza M, Mayol JA et al (2010) Efficacy of the Da Vinci surgical system in abdominal surgery compared with that of laparoscopy: a systematic review and meta-analysis. Ann Surg 252:254–262. https://doi.org/10.1097/SLA.0B013E3181E6239E

    Article  PubMed  Google Scholar 

  54. Pietrabissa A, Vinci A, Pugliese L, Peri A (2013) Robotic surgery: current controversies and future expectations. Cirugía Española (English Edition) 91:67–71. https://doi.org/10.1016/J.CIRENG.2012.07.002

    Article  Google Scholar 

  55. Atal I, Porcher R, Boutron I, Ravaud P (2019) The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol 111:32–40. https://doi.org/10.1016/J.JCLINEPI.2019.03.012

    Article  PubMed  Google Scholar 

  56. Schröder A, Muensterer OJ, Oetzmann von Sochaczewski C (2021) Meta-analyses in paediatric surgery are often fragile: implications and consequences. Pediatr Surg Int 37:363–367. https://doi.org/10.1007/S00383-020-04827-5

    Article  PubMed  PubMed Central  Google Scholar 

  57. Pascoal E, Liu M, Lin L, Luketic L (2022) The fragility of statistically significant results in gynaecologic surgery: a systematic review. J Obstet Gynaecol Can 44:508–514. https://doi.org/10.1016/j.jogc.2021.11.016

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of General Surgery, McMaster University, Hamilton, ON, Canada

    Yung Lee, Yasith Samarasinghe, Lucy H. Chen, Akithma Hapugall, Tyler McKechnie, Aristithes Doumouras & Dennis Hong

  2. Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA

    Yung Lee

  3. Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada

    Audrey Jong

  4. Division of Vascular Surgery, University of Toronto, Toronto, ON, Canada

    Arshia Javidan

  5. Department of Health Research Methods and Evidence, McMaster University, Hamilton, ON, Canada

    Tyler McKechnie

  6. Division of General Surgery, St. Joseph’s Healthcare, 50 Charlton Avenue East, Hamilton, ON, L8N 4A6, Canada

    Dennis Hong

Authors
  1. Yung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yasith Samarasinghe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lucy H. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Audrey Jong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Akithma Hapugall
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Arshia Javidan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tyler McKechnie
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Aristithes Doumouras
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Dennis Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dennis Hong.

Ethics declarations

Disclosure

All authors, Dr. Dennis Hong, Dr. Yung Lee, Yasith Samarasinghe, Lucy H. Chen, Audrey Jong, Akithma Hapugall, Dr. Arshia Javidan, Dr. Tyler McKechnie, and Dr. Aristithes Doumouras, have no conflicts of interest to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 102 KB)

Appendices

Appendix

Appendix 1 Search strategy

  1. 1.

    Robotic.mp

  2. 2.

    Laparocscopic.mp

  3. 3.

    Laparoscopy.mp

  4. 4.

    Randomized trial.mp

  5. 5.

    Randomised trial.mp

  6. 6.

    RCT.mp

  7. 7.

    Randomized controlled trial.mp

  8. 8.

    Randomised controlled trial.mp

  9. 9.

    Robot-assisted.mp

  10. 10.

    Robotic-assisted.mp

  11. 11.

    1 or 9 or 10

  12. 12.

    2 or 3

  13. 13.

    4 or 5 or 6 or 7 or 8

  14. 14.

    11 and 12 and 13

  15. 15.

    Remove duplicates from 14

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, Y., Samarasinghe, Y., Chen, L.H. et al. Fragility of statistically significant findings from randomized trials in comparing laparoscopic versus robotic abdominopelvic surgeries. Surg Endosc 37, 4270–4278 (2023). https://doi.org/10.1007/s00464-023-10063-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-023-10063-4

Keywords

Search

Navigation