Pediatric Surgery International

, Volume 35, Issue 10, pp 1085–1094 | Cite as

Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review

  • Shinichiro Yokoyama
  • Kenichi Mizunuma
  • Yo KurashimaEmail author
  • Yusuke Watanabe
  • Tomoko Mizota
  • Saseem Poudel
  • Takanori Kikuchi
  • Fujimi Kawai
  • Toshiaki Shichinohe
  • Satoshi Hirano
Original Article



The aim of this study was to identify (1) the type of skill evaluation methods and (2) how the effect of training was evaluated in simulation-based training (SBT) in pediatric surgery.


Databases of PubMed, Cochrane Library, and Web of Science were searched for articles published from January 2000 to January 2017. Search concepts of Medical Subject Heading terms were “surgery,” “pediatrics,” “simulation,” and “training, evaluation.”


Of 5858 publications identified, 43 were included. Twenty papers described simulators as assessment tools used to evaluate technical skills. Reviewers differentiated between experts and trainees using a scoring system (45%) and/or a checklist (25%). Simulators as training tools were described in 23 papers. While the training’s effectiveness was measured using performance assessment scales (52%) and/or surveys (43%), no study investigated the improvement of the clinical outcomes after SBT.


Scoring, time, and motion analysis methods were used for the evaluation of basic techniques of laparoscopic skills. Only a few SBT in pediatric surgery have definite goals with clinical effect. Future research needs to demonstrate the educational effect of simulators as assessment or training tools on SBT in pediatric surgery.


Surgical education Evaluation Competency Training impact 



This research received no specific grant from any funding agency in the public and commercial sectors.


This study was not funded by any grant.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest for this study.

Ethical approval

Not applicable, since the study is a systematic review.

Informed consent

Not applicable, since the study is a systematic review.


  1. 1.
    Dawe SR, Pena GN, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ et al (2014) Systematic review of skills transfer after surgical simulation-based training. Br J Surg 101:1063–1076. CrossRefPubMedGoogle Scholar
  2. 2.
    Cheng A, Lang TR, Starr SR, Pusic M, Cook DA (2014) Technology-enhanced simulation and pediatric education: a meta-analysis. Pediatrics 133:e1313–e1323. CrossRefPubMedGoogle Scholar
  3. 3.
    Barsness KA (2015) Trends in technical and team simulations: challenging the status Quo of surgical training. Semin Pediatr Surg 24:130–133. CrossRefPubMedGoogle Scholar
  4. 4.
    Patel EA, Aydın A, Desai A, Dasgupta P, Ahmed K (2018) Current status of simulation-based training in pediatric surgery: a systematic review. J Pediatr Surg. CrossRefPubMedGoogle Scholar
  5. 5.
    Accreditation Council for Graduate Medical Education (2016) ACGME program requirements for graduate medical education in the subspecialties of pediatrics. Accessed 11 Feb 2019
  6. 6.
    The Japanese Society of Pediatric Surgeons Website. Accessed 11 Feb 2019
  7. 7.
    Kirkpatrick D (1996) Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick’s four-level model. Train Dev 50:54–59Google Scholar
  8. 8.
    Azzie G, Gerstle JT, Nasr A, Lasko D, Green J, Henao O et al (2011) Development and validation of a pediatric laparoscopic surgery simulator. J Pediatr Surg 46:897–903. CrossRefPubMedGoogle Scholar
  9. 9.
    Nasr A, Gerstle JT, Carrillo B, Azzie G (2013) The pediatric laparoscopic surgery (PLS) simulator: methodology and results of further validation. J Pediatr Surg 48:2075–2077. CrossRefPubMedGoogle Scholar
  10. 10.
    Hamilton JM, Kahol K, Vankipuram M, Ashby A, Notrica DM, Ferrara JJ (2011) Toward effective pediatric minimally invasive surgical simulation. J Pediatr Surg 46:138–144. CrossRefPubMedGoogle Scholar
  11. 11.
    Nasr A, Carrillo B, Gerstle JT, Azzie G (2014) Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills. J Pediatr Surg 49:791–794. CrossRefPubMedGoogle Scholar
  12. 12.
    Harada K, Takazawa S, Tsukuda Y, Ishimaru T, Sugita N, Iwanaka T et al (2014) Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model. Minim Invasive Ther Allied Technol 24:226–232. CrossRefGoogle Scholar
  13. 13.
    Takazawa S, Ishimaru T, Harada K, Deie K, Fujishiro J, Sugita N et al (2016) Pediatric thoracoscopic surgical simulation using a rapid-prototyped chest model and motion sensors can better identify skilled surgeons than a conventional box trainer. J Laparoendosc Adv Surg Tech 26:740–747. CrossRefGoogle Scholar
  14. 14.
    Retrosi G, Cundy T, Haddad M, Clarke S (2015) Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator. J Laparoendosc Adv Surg Tech 25:944–950. CrossRefGoogle Scholar
  15. 15.
    Trudeau MO, Carrillo B, Nasr A, Gerstle JT, Azzie G (2017) Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator. J Laparoendosc Adv Surg Tech 27:441–446. CrossRefGoogle Scholar
  16. 16.
    Herbert GL, Cundy TP, Singh P, Retrosi G, Sodergren MH, Azzie G et al (2015) Validation of a pediatric single-port laparoscopic surgery simulator. J Pediatr Surg 50:1762–1766. CrossRefPubMedGoogle Scholar
  17. 17.
    Shepherd G, Delft D, Truck J, Kubiak R, Ashour K, Grant H (2015) A simple scoring system to train surgeons in basic laparoscopic skills. Pediatr Surg Int 32:245–252. CrossRefPubMedGoogle Scholar
  18. 18.
    Ieiri S, Ishii H, Souzaki R, Uemura M, Tomikawa M, Matsuoka N et al (2013) Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication. Pediatr Surg Int 29:501–504. CrossRefPubMedGoogle Scholar
  19. 19.
    Jimbo T, Ieiri S, Obata S, Uemura M, Souzaki R, Matsuoka N et al (2016) A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system. Surg Endosc 31:1688–1696. CrossRefPubMedGoogle Scholar
  20. 20.
    Usón-Casaús J, Pérez-Merino EM, Rivera-Barreno R, Rodríguez-Alarcón CA, Sánchez-Margallo FM (2014) Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training. J Laparoendosc Adv Surg Tech 24:280–285. CrossRefGoogle Scholar
  21. 21.
    Obata S, Ieiri S, Uemura M, Jimbo T, Souzaki R, Matsuoka N et al (2015) An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair. J Laparoendosc Adv Surg Tech 25:775–781. CrossRefGoogle Scholar
  22. 22.
    Barsness KA, Rooney DM, Davis LM, Chin AC (2014) Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator. J Pediatr Surg 49:29–33. CrossRefPubMedGoogle Scholar
  23. 23.
    Takazawa S, Ishimaru T, Harada K, Tsukuda Y, Sugita N, Mitsuishi M et al (2015) Video-based skill assessment of endoscopic suturing in a pediatric chest model and a box trainer. J Laparoendosc Adv Surg Tech 25:445–453. CrossRefGoogle Scholar
  24. 24.
    Maricic MA, Bailez MM, Rodriguez SP (2016) Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair. J Pediatr Surg 51:1429–1435. CrossRefPubMedGoogle Scholar
  25. 25.
    Deie K, Ishimaru T, Takazawa S, Harada K, Sugita N, Mitsuishi M et al (2017) Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model. J Laparoendosc Adv Surg Tech 27:76–81. CrossRefGoogle Scholar
  26. 26.
    Barber SR, Kozin ED, Dedmon M, Lin BM, Lee K, Sinha S et al (2016) 3D-printed pediatric endoscopic ear surgery simulator for surgical training. Int J Pediatr Otorhinolaryngol 90:113–118. CrossRefPubMedGoogle Scholar
  27. 27.
    Weinstock P, Rehder R, Prabhu SP, Forbes PW, Roussin CJ, Cohen AR (2017) Creation of a novel simulator for minimally invasive neurosurgery: fusion of 3D printing and special effects. J Neurosurg Pediatr 20:1–9. CrossRefPubMedGoogle Scholar
  28. 28.
    Nakajima K, Wasa M, Takiguchi S, Taniguchi E, Soh H, Ohashi S et al (2003) A modular laparoscopic training program for pediatric surgeons. JSLS 7:33–37PubMedPubMedCentralGoogle Scholar
  29. 29.
    Ieiri S, Nakatsuji T, Higashi M, Akiyoshi J, Uemura M, Konishi K et al (2010) Effectiveness of basic endoscopic surgical skill training for pediatric surgeons. Pediatr Surg Int 26:947–954. CrossRefGoogle Scholar
  30. 30.
    Pérez-Duarte FJ, Díaz-Güemes I, Sánchez-Hurtado MA, Cano Novillo I, Berchi García FJ, García Vázquez A et al (2012) Design and validation of a training model on paediatric and neonatal surgery. Cir Pediatr 25:121–125PubMedGoogle Scholar
  31. 31.
    Jimbo T, Ieiri S, Obata S, Uemura M, Souzaki R, Matsuoka N et al (2015) Effectiveness of short-term endoscopic surgical skill training for young pediatric surgeons: a validation study using the laparoscopic fundoplication simulator. Pediatr Surg Int 31:963–969. CrossRefGoogle Scholar
  32. 32.
    Deutsh ES (2008) High-fidelity patient simulation mannequins to facilitate aerodigestive endoscopy training. Arch Otolaryngol Head Neck Surg 134:625–629. CrossRefGoogle Scholar
  33. 33.
    Binstadt E, Donner S, Nelson J, Flottemesch T, Hegarty C (2008) Simulator training improves fiber-optic intubation proficiency among emergency medicine residents. Acad Emerg Med 15:1211–1214. CrossRefPubMedGoogle Scholar
  34. 34.
    Jabbour N, Reihsen T, Sweet RM, Sidman JD (2011) Psychomotor skills training in pediatric airway endoscopy simulation. Otolaryngol Head Neck Surg 145:43–50. CrossRefPubMedGoogle Scholar
  35. 35.
    Griffin GR, Hoesli R, Thorne MC (2017) Validity and efficacy of a pediatric airway foreign body training course in resident education. Ann Otol Rhinol Laryngol 120:635–640. CrossRefGoogle Scholar
  36. 36.
    Deutsch ES, Christenson T, Curry J, Hossain J, PhD Zur K et al (2009) Multimodality education for airway endoscopy skill development. Ann Otol Rhinol Laryngol 118:81–86CrossRefGoogle Scholar
  37. 37.
    Dabbas N, Muktar Z, Ade-Ajayi N (2009) GABBY: an ex vivo model for learning and refining the technique of preformed silo application in the management of gastroschisis. Afr J Paediatr Surg 6:73–76. CrossRefPubMedGoogle Scholar
  38. 38.
    Tugnoli G, Ribaldi S, Casali M, Calderale SM, Coletti M, Villani S et al (2007) The education of the trauma surgeon: the "trauma surgery course" as advanced didactic tool. Ann Ital Chir 78:39–44PubMedGoogle Scholar
  39. 39.
    Lehner M, Heimberg E, Hoffmann F, Heinzel O, Kirschner H-J, Heinrich M (2017) Evaluation of a pilot project to introduce simulation-based team training to pediatric surgery trauma room care. Int J Pediatr 2017:1–6. CrossRefGoogle Scholar
  40. 40.
    Nishisaki A Scrattish L Boulet J Kalsi M Maltese M Castner T et al (2008) Effect of recent refresher training on in situ simulated pediatric tracheal intubation psychomotor skill performance. In: Henriksen K Battles JB Keyes MA et al (eds) Advances in patient safety: new directions and alternative approaches (vol. 3: performance and tools). Agency for Healthcare Research and Quality (US), RockvilleGoogle Scholar
  41. 41.
    Reid J, Stone K, Brown J, Caglar D, Kobayashi A, Lewis-Newby M et al (2012) The Simulation Team Assessment Tool (STAT): development, reliability and validation. Resuscitation 83:879–886. CrossRefPubMedGoogle Scholar
  42. 42.
    Nishisaki A, Hales R, Biagas K, Cheifetz I, Corriveau C, Garber N et al (2009) A multi-institutional high-fidelity simulation “boot camp” orientation and training program for firstyear pediatric critical care fellows. Pediatr Crit Care Med 10:157–162. CrossRefPubMedGoogle Scholar
  43. 43.
    Cheng A, Goldman RD, Aish MA, Kissoon N (2010) A simulation-based acute care curriculum for pediatric emergency medicine fellowship training programs. Pediatr Emerg Care 26:475–476. CrossRefPubMedGoogle Scholar
  44. 44.
    Stone K, Reid J, Caglar D, Christensen A, Strelitz B, Zhou L, Quan L (2014) Increasing pediatric resident simulated resuscitation performance: a standardized simulation-based curriculum. Resuscitation 85:1099–1105. CrossRefPubMedGoogle Scholar
  45. 45.
    Atamanyuk I, Ghez O, Saeed I, Lane M, Hall J, Jackson T et al (2013) Impact of an open-chest extracorporeal membrane oxygenation model for in situ simulated team training: a pilot study. Interact Cardiovasc Thorac Surg 18:17–20. CrossRefPubMedPubMedCentralGoogle Scholar
  46. 46.
    Allan CK, Pigula F, Bacha EA, Emani S, Fynn-Thompson F, Thiagarajan RR et al (2013) An extracorporeal membrane oxygenation cannulation curriculum featuring a novel integrated skills trainer leads to improved performance among pediatric cardiac surgery trainees. Simul Healthc 8:221–228. CrossRefPubMedGoogle Scholar
  47. 47.
    Brydges R, Farhat WA, El-Hout Y, Dubrowski A (2010) Pediatric urology training: performance-based assessment using the fundamentals of laparoscopic surgery. J Surg Res 161:240–245. CrossRefPubMedGoogle Scholar
  48. 48.
    Soltani T, Hidas G, Kelly MS, Kaplan A, Selby B, Billimek J et al (2016) Endoscopic correction of vesicoureteral reflux simulator curriculum as an effective teaching tool: pilot study. J Pediatr Urol 12:45.e1–45.e6. CrossRefGoogle Scholar
  49. 49.
    Peeters SHP, Akkermans J, Slaghekke F, Bustraan J, Lopriore E, Haak MC et al (2015) Simulator training in fetoscopic laser surgery for twin-twin transfusion syndrome: a pilot randomized controlled trial. Ultrasound Obstet Gynecol 46:319–326. CrossRefPubMedGoogle Scholar
  50. 50.
    Yoo SJY, Spray T, Austion EH, Yun TJ, van Arsdell GS (2017) Hands-on surgical training of congenital heart surgery using 3-dimensional print models. J Thorac Cardiovasc Surg 153:1530–1540. CrossRefPubMedGoogle Scholar
  51. 51.
    Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR, For the International CBME Collaborators (2010) The role of assessment in competency-based medical education. Med Teach 32:676–682. CrossRefPubMedGoogle Scholar
  52. 52.
    Fried GM, Feldman LS (2007) Objective assessment of technical performance. World J Surg 32:156–160. CrossRefGoogle Scholar
  53. 53.
    Jones DB, Schwaitzberg SD (2019) Operative endoscopic and minimally invasive surgery, 1st edn. CRC Press, Boca Raton, pp 184–187Google Scholar
  54. 54.
    Farcas MA, Trudeau MO, Nasr A, Gerstle JT, Carrillo B, Azzie G (2016) Analysis of motion in laparoscopy: the deconstruction of an intra- corporeal suturing task. Surg Endosc 31:3130–3139. CrossRefPubMedGoogle Scholar
  55. 55.
    Cecilio-Fernandes D, Cnossen F, Jaarsma DADC, Tio RA (2018) Avoiding surgical skill decay a systematic review on the spacing of training sessions. J Surg Educ 75:471–480. CrossRefPubMedGoogle Scholar
  56. 56.
    Cox T, Seymour N, Stefanidis D (2015) Moving the needle: simulation’s impact on patient outcomes. Surg Clin N Am 95:827–838. CrossRefPubMedGoogle Scholar
  57. 57.
    Zendejas B, Cook DA, Bingener J, Huebner M, Dunn WF, Sarr MG et al (2011) Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair. Ann Surg 254:502–509. (discussion 509–11) CrossRefPubMedGoogle Scholar
  58. 58.
    Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D et al (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113CrossRefGoogle Scholar
  59. 59.
    Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the Global Operative Assessment of Laparoscopic Skills-Groin Hernia (GOALS-GH). Am J Surg 201:54–61. CrossRefPubMedGoogle Scholar
  60. 60.
    Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I et al (2015) Technical skills assessment toolbox. Ann Surg 261:251–262. CrossRefPubMedGoogle Scholar
  61. 61.
    Oue T, Kubota A, Okuyama H, Kawahara H (2005) Laparoscopic percutaneous extraperitoneal closure (LPEC) method for the exploration and treatment of inguinal hernia in girls. Pediatr Surg Int 21:964–968. CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Shinichiro Yokoyama
    • 1
  • Kenichi Mizunuma
    • 1
  • Yo Kurashima
    • 1
    • 2
    • 7
    Email author
  • Yusuke Watanabe
    • 1
  • Tomoko Mizota
    • 1
    • 3
  • Saseem Poudel
    • 1
    • 4
  • Takanori Kikuchi
    • 5
  • Fujimi Kawai
    • 6
  • Toshiaki Shichinohe
    • 1
  • Satoshi Hirano
    • 1
  1. 1.Department of Gastrointestinal Surgery IIHokkaido University Graduate School of MedicineSapporoJapan
  2. 2.Clinical Simulation CenterHokkaido UniversitySapporoJapan
  3. 3.Department of SurgeryNational Hospital Organization Hakodate HospitalHakodateJapan
  4. 4.Department of General SurgerySteel Memorial Muroran HospitalMuroranJapan
  5. 5.Medical LibraryHokkaido UniversitySapporoJapan
  6. 6.St. Luke’s International University LibraryTokyoJapan
  7. 7.Department of Gastroenterological Surgery IIHokkaido University Faculty of MedicineSapporoJapan

Personalised recommendations