Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review
The aim of this study was to identify (1) the type of skill evaluation methods and (2) how the effect of training was evaluated in simulation-based training (SBT) in pediatric surgery.
Databases of PubMed, Cochrane Library, and Web of Science were searched for articles published from January 2000 to January 2017. Search concepts of Medical Subject Heading terms were “surgery,” “pediatrics,” “simulation,” and “training, evaluation.”
Of 5858 publications identified, 43 were included. Twenty papers described simulators as assessment tools used to evaluate technical skills. Reviewers differentiated between experts and trainees using a scoring system (45%) and/or a checklist (25%). Simulators as training tools were described in 23 papers. While the training’s effectiveness was measured using performance assessment scales (52%) and/or surveys (43%), no study investigated the improvement of the clinical outcomes after SBT.
Scoring, time, and motion analysis methods were used for the evaluation of basic techniques of laparoscopic skills. Only a few SBT in pediatric surgery have definite goals with clinical effect. Future research needs to demonstrate the educational effect of simulators as assessment or training tools on SBT in pediatric surgery.
KeywordsSurgical education Evaluation Competency Training impact
This research received no specific grant from any funding agency in the public and commercial sectors.
This study was not funded by any grant.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest for this study.
Not applicable, since the study is a systematic review.
Not applicable, since the study is a systematic review.
- 5.Accreditation Council for Graduate Medical Education (2016) ACGME program requirements for graduate medical education in the subspecialties of pediatrics. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320S_pediatric_subs_2016.pdf. Accessed 11 Feb 2019
- 6.The Japanese Society of Pediatric Surgeons Website. https://www.jsps.gr.jp/about/rule_contents/rules. Accessed 11 Feb 2019
- 7.Kirkpatrick D (1996) Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick’s four-level model. Train Dev 50:54–59Google Scholar
- 13.Takazawa S, Ishimaru T, Harada K, Deie K, Fujishiro J, Sugita N et al (2016) Pediatric thoracoscopic surgical simulation using a rapid-prototyped chest model and motion sensors can better identify skilled surgeons than a conventional box trainer. J Laparoendosc Adv Surg Tech 26:740–747. https://doi.org/10.1089/lap.2016.0131 CrossRefGoogle Scholar
- 18.Ieiri S, Ishii H, Souzaki R, Uemura M, Tomikawa M, Matsuoka N et al (2013) Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication. Pediatr Surg Int 29:501–504. https://doi.org/10.1007/s00383-013-3276-x CrossRefPubMedGoogle Scholar
- 21.Obata S, Ieiri S, Uemura M, Jimbo T, Souzaki R, Matsuoka N et al (2015) An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair. J Laparoendosc Adv Surg Tech 25:775–781. https://doi.org/10.1089/lap.2014.0259 CrossRefGoogle Scholar
- 24.Maricic MA, Bailez MM, Rodriguez SP (2016) Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair. J Pediatr Surg 51:1429–1435. https://doi.org/10.1016/j.jpedsurg.2016.04.018 CrossRefPubMedGoogle Scholar
- 25.Deie K, Ishimaru T, Takazawa S, Harada K, Sugita N, Mitsuishi M et al (2017) Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model. J Laparoendosc Adv Surg Tech 27:76–81. https://doi.org/10.1089/lap.2016.0214 CrossRefGoogle Scholar
- 31.Jimbo T, Ieiri S, Obata S, Uemura M, Souzaki R, Matsuoka N et al (2015) Effectiveness of short-term endoscopic surgical skill training for young pediatric surgeons: a validation study using the laparoscopic fundoplication simulator. Pediatr Surg Int 31:963–969. https://doi.org/10.1007/s00383-015-3776-y CrossRefGoogle Scholar
- 40.Nishisaki A Scrattish L Boulet J Kalsi M Maltese M Castner T et al (2008) Effect of recent refresher training on in situ simulated pediatric tracheal intubation psychomotor skill performance. In: Henriksen K Battles JB Keyes MA et al (eds) Advances in patient safety: new directions and alternative approaches (vol. 3: performance and tools). Agency for Healthcare Research and Quality (US), RockvilleGoogle Scholar
- 41.Reid J, Stone K, Brown J, Caglar D, Kobayashi A, Lewis-Newby M et al (2012) The Simulation Team Assessment Tool (STAT): development, reliability and validation. Resuscitation 83:879–886. https://doi.org/10.1016/j.resuscitation.2011.12.012 CrossRefPubMedGoogle Scholar
- 42.Nishisaki A, Hales R, Biagas K, Cheifetz I, Corriveau C, Garber N et al (2009) A multi-institutional high-fidelity simulation “boot camp” orientation and training program for firstyear pediatric critical care fellows. Pediatr Crit Care Med 10:157–162. https://doi.org/10.1097/PCC.0b013e3181956d29 CrossRefPubMedGoogle Scholar
- 44.Stone K, Reid J, Caglar D, Christensen A, Strelitz B, Zhou L, Quan L (2014) Increasing pediatric resident simulated resuscitation performance: a standardized simulation-based curriculum. Resuscitation 85:1099–1105. https://doi.org/10.1016/j.resuscitation.2014.05.005 CrossRefPubMedGoogle Scholar
- 45.Atamanyuk I, Ghez O, Saeed I, Lane M, Hall J, Jackson T et al (2013) Impact of an open-chest extracorporeal membrane oxygenation model for in situ simulated team training: a pilot study. Interact Cardiovasc Thorac Surg 18:17–20. https://doi.org/10.1093/icvts/ivt437 CrossRefPubMedPubMedCentralGoogle Scholar
- 46.Allan CK, Pigula F, Bacha EA, Emani S, Fynn-Thompson F, Thiagarajan RR et al (2013) An extracorporeal membrane oxygenation cannulation curriculum featuring a novel integrated skills trainer leads to improved performance among pediatric cardiac surgery trainees. Simul Healthc 8:221–228. https://doi.org/10.1097/SIH.0b013e31828b4179 CrossRefPubMedGoogle Scholar
- 49.Peeters SHP, Akkermans J, Slaghekke F, Bustraan J, Lopriore E, Haak MC et al (2015) Simulator training in fetoscopic laser surgery for twin-twin transfusion syndrome: a pilot randomized controlled trial. Ultrasound Obstet Gynecol 46:319–326. https://doi.org/10.1002/uog.14916 CrossRefPubMedGoogle Scholar
- 53.Jones DB, Schwaitzberg SD (2019) Operative endoscopic and minimally invasive surgery, 1st edn. CRC Press, Boca Raton, pp 184–187Google Scholar
- 59.Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the Global Operative Assessment of Laparoscopic Skills-Groin Hernia (GOALS-GH). Am J Surg 201:54–61. https://doi.org/10.1016/j.amjsurg.2010.09.006 CrossRefPubMedGoogle Scholar