Skip to main content

Advertisement

SpringerLink
Log in

Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review

  • Original Article
  • Published:
Pediatric Surgery International Aims and scope Submit manuscript

Abstract

Purpose

The aim of this study was to identify (1) the type of skill evaluation methods and (2) how the effect of training was evaluated in simulation-based training (SBT) in pediatric surgery.

Methods

Databases of PubMed, Cochrane Library, and Web of Science were searched for articles published from January 2000 to January 2017. Search concepts of Medical Subject Heading terms were “surgery,” “pediatrics,” “simulation,” and “training, evaluation.”

Results

Of 5858 publications identified, 43 were included. Twenty papers described simulators as assessment tools used to evaluate technical skills. Reviewers differentiated between experts and trainees using a scoring system (45%) and/or a checklist (25%). Simulators as training tools were described in 23 papers. While the training’s effectiveness was measured using performance assessment scales (52%) and/or surveys (43%), no study investigated the improvement of the clinical outcomes after SBT.

Conclusion

Scoring, time, and motion analysis methods were used for the evaluation of basic techniques of laparoscopic skills. Only a few SBT in pediatric surgery have definite goals with clinical effect. Future research needs to demonstrate the educational effect of simulators as assessment or training tools on SBT in pediatric surgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Dawe SR, Pena GN, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ et al (2014) Systematic review of skills transfer after surgical simulation-based training. Br J Surg 101:1063–1076. https://doi.org/10.1002/bjs.9482

    Article  CAS  PubMed  Google Scholar 

  2. Cheng A, Lang TR, Starr SR, Pusic M, Cook DA (2014) Technology-enhanced simulation and pediatric education: a meta-analysis. Pediatrics 133:e1313–e1323. https://doi.org/10.1542/peds.2013-2139

    Article  PubMed  Google Scholar 

  3. Barsness KA (2015) Trends in technical and team simulations: challenging the status Quo of surgical training. Semin Pediatr Surg 24:130–133. https://doi.org/10.1053/j.sempedsurg.2015.02.011

    Article  PubMed  Google Scholar 

  4. Patel EA, Aydın A, Desai A, Dasgupta P, Ahmed K (2018) Current status of simulation-based training in pediatric surgery: a systematic review. J Pediatr Surg. https://doi.org/10.1016/j.jpedsurg.2018.11.019

    Article  PubMed  Google Scholar 

  5. Accreditation Council for Graduate Medical Education (2016) ACGME program requirements for graduate medical education in the subspecialties of pediatrics. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320S_pediatric_subs_2016.pdf. Accessed 11 Feb 2019

  6. The Japanese Society of Pediatric Surgeons Website. https://www.jsps.gr.jp/about/rule_contents/rules. Accessed 11 Feb 2019

  7. Kirkpatrick D (1996) Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick’s four-level model. Train Dev 50:54–59

    Google Scholar 

  8. Azzie G, Gerstle JT, Nasr A, Lasko D, Green J, Henao O et al (2011) Development and validation of a pediatric laparoscopic surgery simulator. J Pediatr Surg 46:897–903. https://doi.org/10.1016/j.jpedsurg.2011.02.026

    Article  PubMed  Google Scholar 

  9. Nasr A, Gerstle JT, Carrillo B, Azzie G (2013) The pediatric laparoscopic surgery (PLS) simulator: methodology and results of further validation. J Pediatr Surg 48:2075–2077. https://doi.org/10.1016/j.jpedsurg.2013.01.039

    Article  PubMed  Google Scholar 

  10. Hamilton JM, Kahol K, Vankipuram M, Ashby A, Notrica DM, Ferrara JJ (2011) Toward effective pediatric minimally invasive surgical simulation. J Pediatr Surg 46:138–144. https://doi.org/10.1016/j.jpedsurg.2010.09.078

    Article  PubMed  Google Scholar 

  11. Nasr A, Carrillo B, Gerstle JT, Azzie G (2014) Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills. J Pediatr Surg 49:791–794. https://doi.org/10.1016/j.jpedsurg.2014.02.063

    Article  PubMed  Google Scholar 

  12. Harada K, Takazawa S, Tsukuda Y, Ishimaru T, Sugita N, Iwanaka T et al (2014) Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model. Minim Invasive Ther Allied Technol 24:226–232. https://doi.org/10.3109/13645706.2014.996161

    Article  Google Scholar 

  13. Takazawa S, Ishimaru T, Harada K, Deie K, Fujishiro J, Sugita N et al (2016) Pediatric thoracoscopic surgical simulation using a rapid-prototyped chest model and motion sensors can better identify skilled surgeons than a conventional box trainer. J Laparoendosc Adv Surg Tech 26:740–747. https://doi.org/10.1089/lap.2016.0131

    Article  Google Scholar 

  14. Retrosi G, Cundy T, Haddad M, Clarke S (2015) Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator. J Laparoendosc Adv Surg Tech 25:944–950. https://doi.org/10.1089/lap.2015.0069

    Article  Google Scholar 

  15. Trudeau MO, Carrillo B, Nasr A, Gerstle JT, Azzie G (2017) Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator. J Laparoendosc Adv Surg Tech 27:441–446. https://doi.org/10.1089/lap.2016.0516

    Article  Google Scholar 

  16. Herbert GL, Cundy TP, Singh P, Retrosi G, Sodergren MH, Azzie G et al (2015) Validation of a pediatric single-port laparoscopic surgery simulator. J Pediatr Surg 50:1762–1766. https://doi.org/10.1016/j.jpedsurg.2015.03.057

    Article  PubMed  Google Scholar 

  17. Shepherd G, Delft D, Truck J, Kubiak R, Ashour K, Grant H (2015) A simple scoring system to train surgeons in basic laparoscopic skills. Pediatr Surg Int 32:245–252. https://doi.org/10.1007/s00383-015-3841-6

    Article  PubMed  Google Scholar 

  18. Ieiri S, Ishii H, Souzaki R, Uemura M, Tomikawa M, Matsuoka N et al (2013) Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication. Pediatr Surg Int 29:501–504. https://doi.org/10.1007/s00383-013-3276-x

    Article  PubMed  Google Scholar 

  19. Jimbo T, Ieiri S, Obata S, Uemura M, Souzaki R, Matsuoka N et al (2016) A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system. Surg Endosc 31:1688–1696. https://doi.org/10.1007/s00464-016-5159-4

    Article  PubMed  Google Scholar 

  20. Usón-Casaús J, Pérez-Merino EM, Rivera-Barreno R, Rodríguez-Alarcón CA, Sánchez-Margallo FM (2014) Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training. J Laparoendosc Adv Surg Tech 24:280–285. https://doi.org/10.1089/lap.2013.0358

    Article  Google Scholar 

  21. Obata S, Ieiri S, Uemura M, Jimbo T, Souzaki R, Matsuoka N et al (2015) An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair. J Laparoendosc Adv Surg Tech 25:775–781. https://doi.org/10.1089/lap.2014.0259

    Article  Google Scholar 

  22. Barsness KA, Rooney DM, Davis LM, Chin AC (2014) Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator. J Pediatr Surg 49:29–33. https://doi.org/10.1016/j.jpedsurg.2013.09.069

    Article  PubMed  Google Scholar 

  23. Takazawa S, Ishimaru T, Harada K, Tsukuda Y, Sugita N, Mitsuishi M et al (2015) Video-based skill assessment of endoscopic suturing in a pediatric chest model and a box trainer. J Laparoendosc Adv Surg Tech 25:445–453. https://doi.org/10.1089/lap.2014.0269

    Article  Google Scholar 

  24. Maricic MA, Bailez MM, Rodriguez SP (2016) Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair. J Pediatr Surg 51:1429–1435. https://doi.org/10.1016/j.jpedsurg.2016.04.018

    Article  PubMed  Google Scholar 

  25. Deie K, Ishimaru T, Takazawa S, Harada K, Sugita N, Mitsuishi M et al (2017) Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model. J Laparoendosc Adv Surg Tech 27:76–81. https://doi.org/10.1089/lap.2016.0214

    Article  Google Scholar 

  26. Barber SR, Kozin ED, Dedmon M, Lin BM, Lee K, Sinha S et al (2016) 3D-printed pediatric endoscopic ear surgery simulator for surgical training. Int J Pediatr Otorhinolaryngol 90:113–118. https://doi.org/10.1016/j.ijporl.2016.08.027

    Article  PubMed  Google Scholar 

  27. Weinstock P, Rehder R, Prabhu SP, Forbes PW, Roussin CJ, Cohen AR (2017) Creation of a novel simulator for minimally invasive neurosurgery: fusion of 3D printing and special effects. J Neurosurg Pediatr 20:1–9. https://doi.org/10.3171/2017.1.PEDS16568

    Article  PubMed  Google Scholar 

  28. Nakajima K, Wasa M, Takiguchi S, Taniguchi E, Soh H, Ohashi S et al (2003) A modular laparoscopic training program for pediatric surgeons. JSLS 7:33–37

    PubMed  PubMed Central  Google Scholar 

  29. Ieiri S, Nakatsuji T, Higashi M, Akiyoshi J, Uemura M, Konishi K et al (2010) Effectiveness of basic endoscopic surgical skill training for pediatric surgeons. Pediatr Surg Int 26:947–954. https://doi.org/10.1007/s00383-010-2665-7

    Article  PubMed  Google Scholar 

  30. Pérez-Duarte FJ, Díaz-Güemes I, Sánchez-Hurtado MA, Cano Novillo I, Berchi García FJ, García Vázquez A et al (2012) Design and validation of a training model on paediatric and neonatal surgery. Cir Pediatr 25:121–125

    PubMed  Google Scholar 

  31. Jimbo T, Ieiri S, Obata S, Uemura M, Souzaki R, Matsuoka N et al (2015) Effectiveness of short-term endoscopic surgical skill training for young pediatric surgeons: a validation study using the laparoscopic fundoplication simulator. Pediatr Surg Int 31:963–969. https://doi.org/10.1007/s00383-015-3776-y

    Article  PubMed  Google Scholar 

  32. Deutsh ES (2008) High-fidelity patient simulation mannequins to facilitate aerodigestive endoscopy training. Arch Otolaryngol Head Neck Surg 134:625–629. https://doi.org/10.1001/archotol.134.6.625

    Article  Google Scholar 

  33. Binstadt E, Donner S, Nelson J, Flottemesch T, Hegarty C (2008) Simulator training improves fiber-optic intubation proficiency among emergency medicine residents. Acad Emerg Med 15:1211–1214. https://doi.org/10.1111/j.1553-2712.2008.00199.x

    Article  PubMed  Google Scholar 

  34. Jabbour N, Reihsen T, Sweet RM, Sidman JD (2011) Psychomotor skills training in pediatric airway endoscopy simulation. Otolaryngol Head Neck Surg 145:43–50. https://doi.org/10.1177/0194599811403379

    Article  PubMed  Google Scholar 

  35. Griffin GR, Hoesli R, Thorne MC (2017) Validity and efficacy of a pediatric airway foreign body training course in resident education. Ann Otol Rhinol Laryngol 120:635–640. https://doi.org/10.1177/000348941112001002

    Article  Google Scholar 

  36. Deutsch ES, Christenson T, Curry J, Hossain J, PhD Zur K et al (2009) Multimodality education for airway endoscopy skill development. Ann Otol Rhinol Laryngol 118:81–86

    Article  PubMed  Google Scholar 

  37. Dabbas N, Muktar Z, Ade-Ajayi N (2009) GABBY: an ex vivo model for learning and refining the technique of preformed silo application in the management of gastroschisis. Afr J Paediatr Surg 6:73–76. https://doi.org/10.4103/0189-6725.54766

    Article  PubMed  Google Scholar 

  38. Tugnoli G, Ribaldi S, Casali M, Calderale SM, Coletti M, Villani S et al (2007) The education of the trauma surgeon: the "trauma surgery course" as advanced didactic tool. Ann Ital Chir 78:39–44

    PubMed  Google Scholar 

  39. Lehner M, Heimberg E, Hoffmann F, Heinzel O, Kirschner H-J, Heinrich M (2017) Evaluation of a pilot project to introduce simulation-based team training to pediatric surgery trauma room care. Int J Pediatr 2017:1–6. https://doi.org/10.1155/2017/9732316

    Article  Google Scholar 

  40. Nishisaki A Scrattish L Boulet J Kalsi M Maltese M Castner T et al (2008) Effect of recent refresher training on in situ simulated pediatric tracheal intubation psychomotor skill performance. In: Henriksen K Battles JB Keyes MA et al (eds) Advances in patient safety: new directions and alternative approaches (vol. 3: performance and tools). Agency for Healthcare Research and Quality (US), Rockville

  41. Reid J, Stone K, Brown J, Caglar D, Kobayashi A, Lewis-Newby M et al (2012) The Simulation Team Assessment Tool (STAT): development, reliability and validation. Resuscitation 83:879–886. https://doi.org/10.1016/j.resuscitation.2011.12.012

    Article  PubMed  Google Scholar 

  42. Nishisaki A, Hales R, Biagas K, Cheifetz I, Corriveau C, Garber N et al (2009) A multi-institutional high-fidelity simulation “boot camp” orientation and training program for firstyear pediatric critical care fellows. Pediatr Crit Care Med 10:157–162. https://doi.org/10.1097/PCC.0b013e3181956d29

    Article  PubMed  Google Scholar 

  43. Cheng A, Goldman RD, Aish MA, Kissoon N (2010) A simulation-based acute care curriculum for pediatric emergency medicine fellowship training programs. Pediatr Emerg Care 26:475–476. https://doi.org/10.1097/PEC.0b013e3181e5841b

    Article  PubMed  Google Scholar 

  44. Stone K, Reid J, Caglar D, Christensen A, Strelitz B, Zhou L, Quan L (2014) Increasing pediatric resident simulated resuscitation performance: a standardized simulation-based curriculum. Resuscitation 85:1099–1105. https://doi.org/10.1016/j.resuscitation.2014.05.005

    Article  PubMed  Google Scholar 

  45. Atamanyuk I, Ghez O, Saeed I, Lane M, Hall J, Jackson T et al (2013) Impact of an open-chest extracorporeal membrane oxygenation model for in situ simulated team training: a pilot study. Interact Cardiovasc Thorac Surg 18:17–20. https://doi.org/10.1093/icvts/ivt437

    Article  PubMed  PubMed Central  Google Scholar 

  46. Allan CK, Pigula F, Bacha EA, Emani S, Fynn-Thompson F, Thiagarajan RR et al (2013) An extracorporeal membrane oxygenation cannulation curriculum featuring a novel integrated skills trainer leads to improved performance among pediatric cardiac surgery trainees. Simul Healthc 8:221–228. https://doi.org/10.1097/SIH.0b013e31828b4179

    Article  PubMed  Google Scholar 

  47. Brydges R, Farhat WA, El-Hout Y, Dubrowski A (2010) Pediatric urology training: performance-based assessment using the fundamentals of laparoscopic surgery. J Surg Res 161:240–245. https://doi.org/10.1016/j.jss.2008.12.041

    Article  PubMed  Google Scholar 

  48. Soltani T, Hidas G, Kelly MS, Kaplan A, Selby B, Billimek J et al (2016) Endoscopic correction of vesicoureteral reflux simulator curriculum as an effective teaching tool: pilot study. J Pediatr Urol 12:45.e1–45.e6. https://doi.org/10.1016/j.jpurol.2015.06.017

    Article  Google Scholar 

  49. Peeters SHP, Akkermans J, Slaghekke F, Bustraan J, Lopriore E, Haak MC et al (2015) Simulator training in fetoscopic laser surgery for twin-twin transfusion syndrome: a pilot randomized controlled trial. Ultrasound Obstet Gynecol 46:319–326. https://doi.org/10.1002/uog.14916

    Article  CAS  PubMed  Google Scholar 

  50. Yoo SJY, Spray T, Austion EH, Yun TJ, van Arsdell GS (2017) Hands-on surgical training of congenital heart surgery using 3-dimensional print models. J Thorac Cardiovasc Surg 153:1530–1540. https://doi.org/10.1016/j.jtcvs.2016.12.054

    Article  PubMed  Google Scholar 

  51. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR, For the International CBME Collaborators (2010) The role of assessment in competency-based medical education. Med Teach 32:676–682. https://doi.org/10.3109/0142159X.2010.500704

    Article  PubMed  Google Scholar 

  52. Fried GM, Feldman LS (2007) Objective assessment of technical performance. World J Surg 32:156–160. https://doi.org/10.1007/s00268-007-9143-y

    Article  Google Scholar 

  53. Jones DB, Schwaitzberg SD (2019) Operative endoscopic and minimally invasive surgery, 1st edn. CRC Press, Boca Raton, pp 184–187

    Google Scholar 

  54. Farcas MA, Trudeau MO, Nasr A, Gerstle JT, Carrillo B, Azzie G (2016) Analysis of motion in laparoscopy: the deconstruction of an intra- corporeal suturing task. Surg Endosc 31:3130–3139. https://doi.org/10.1007/s00464-016-5337-4

    Article  PubMed  Google Scholar 

  55. Cecilio-Fernandes D, Cnossen F, Jaarsma DADC, Tio RA (2018) Avoiding surgical skill decay a systematic review on the spacing of training sessions. J Surg Educ 75:471–480. https://doi.org/10.1016/j.jsurg.2017.08.002

    Article  PubMed  Google Scholar 

  56. Cox T, Seymour N, Stefanidis D (2015) Moving the needle: simulation’s impact on patient outcomes. Surg Clin N Am 95:827–838. https://doi.org/10.1016/j.suc.2015.03.005

    Article  PubMed  Google Scholar 

  57. Zendejas B, Cook DA, Bingener J, Huebner M, Dunn WF, Sarr MG et al (2011) Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair. Ann Surg 254:502–509. https://doi.org/10.1097/SLA.0b013e31822c6994 (discussion 509–11)

    Article  PubMed  Google Scholar 

  58. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D et al (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113

    Article  PubMed  Google Scholar 

  59. Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the Global Operative Assessment of Laparoscopic Skills-Groin Hernia (GOALS-GH). Am J Surg 201:54–61. https://doi.org/10.1016/j.amjsurg.2010.09.006

    Article  PubMed  Google Scholar 

  60. Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I et al (2015) Technical skills assessment toolbox. Ann Surg 261:251–262. https://doi.org/10.1097/SLA.0000000000000520

    Article  PubMed  Google Scholar 

  61. Oue T, Kubota A, Okuyama H, Kawahara H (2005) Laparoscopic percutaneous extraperitoneal closure (LPEC) method for the exploration and treatment of inguinal hernia in girls. Pediatr Surg Int 21:964–968. https://doi.org/10.1007/s00383-005-1556-9

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research received no specific grant from any funding agency in the public and commercial sectors.

Funding

This study was not funded by any grant.

Author information

Authors and Affiliations

  1. Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan

    Shinichiro Yokoyama, Kenichi Mizunuma, Yo Kurashima, Yusuke Watanabe, Tomoko Mizota, Saseem Poudel, Toshiaki Shichinohe & Satoshi Hirano

  2. Clinical Simulation Center, Hokkaido University, Sapporo, Japan

    Yo Kurashima

  3. Department of Surgery, National Hospital Organization Hakodate Hospital, Hakodate, Japan

    Tomoko Mizota

  4. Department of General Surgery, Steel Memorial Muroran Hospital, Muroran, Japan

    Saseem Poudel

  5. Medical Library, Hokkaido University, Sapporo, Japan

    Takanori Kikuchi

  6. St. Luke’s International University Library, Tokyo, Japan

    Fujimi Kawai

  7. Department of Gastroenterological Surgery II, Hokkaido University Faculty of Medicine, Kita 15, Nishi 7, Kita-ku, Sapporo, 060-8638, Japan

    Yo Kurashima

Authors
  1. Shinichiro Yokoyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenichi Mizunuma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yo Kurashima
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yusuke Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tomoko Mizota
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Saseem Poudel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Takanori Kikuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Fujimi Kawai
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Toshiaki Shichinohe
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Satoshi Hirano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yo Kurashima.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest for this study.

Ethical approval

Not applicable, since the study is a systematic review.

Informed consent

Not applicable, since the study is a systematic review.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yokoyama, S., Mizunuma, K., Kurashima, Y. et al. Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review. Pediatr Surg Int 35, 1085–1094 (2019). https://doi.org/10.1007/s00383-019-04539-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00383-019-04539-5

Keywords

Search

Navigation