Equipment-related malfunctions directly relate to one-fourth of the adverse events in the surgical theater. A serious game trains residents to recognize and respond to equipment problems in minimally invasive surgery (MIS). These include disturbed vision, gas transport, electrocautery, and pathophysiological disturbances. This randomized controlled trial explores whether game-based training improves surgical residents’ response to equipment-related problems during surgery.
Thirty-one surgical residents with no previous experience in MIS took part in a standardized basic laparoscopy training course. Fifteen residents were randomly assigned to the game-enhanced curriculum (intervention) and sixteen were assigned to the regular curriculum (control). Participants performed a MIS task in a live anesthetized pig model, during which three standardized equipment malfunction scenarios occurred. Observers recorded the problems recognized and solved, time, and participants’ technical performance.
Twenty-four participants completed the post-test (n = 12 per group). The intervention group solved more problems than the control group (59 vs. 33%, p = 0.029). The intervention group also recognized a larger proportion of problems, although this parameter was non-significant (67 vs. 42%, p = 0.14). Random effects modeling showed a significant improved game performance per participant over time.
Surgical residents, who play for only 1 h on a custom-made serious game, respond significantly better to equipment-related problems during surgery than residents trained by a standard training curriculum. These results imply that entertaining serious games can indeed be considered for use in official training for surgeons and other medical specialists.
Minimally invasive surgery (MIS) has been widely adopted in various surgical procedures, reducing overall patient morbidity whilst improving cosmetic results. However, the surgeon’s increased workload in a technology-dependent environment  increases the chance for errors to occur. Errors relating to the equipment occur frequently in the laparoscopic suite and pose a significant threat to patient safety [2–5]. A recent systematic review shows that equipment malfunctions are to be held responsible for nearly a quarter of the adverse events in the OR .
Standardized MIS training courses aim to develop knowledge and psychomotor skills and are part of surgical training in many countries [e.g., fundamentals of laparoscopic surgery  (FLS)]. Basic laparoscopic training courses focus on laparoscopic principles and dexterity training. However, they do not educate surgical trainees to deal with the laparoscopic environment or with equipment-related errors. Even experienced professionals seem to be insufficiently equipped to solve laparoscopic equipment-related problems, when they encounter them during MIS . Recent studies show that long-term knowledge preservation regarding MIS equipment after basic laparoscopy courses is poor . Additional training focused on long-term knowledge retention and dealing with non-routine events during MIS is therefore much needed.
A serious game (Dr. Game, Surgeon Trouble®) was developed to train surgical personnel in recognizing and solving equipment-related problems in MIS. Playing this specifically developed serious game is likely to improve trainees’ problem recognition and problem-solving skills in the OR. The serious game’s construct validity was established in a previous study . Serious games are digital applications that are both fun to play and supply the player skills, knowledge, or attitudes useful in reality . Both virtual reality simulators  and serious games  have proven to be effective modalities to improve surgeons’ laparoscopic dexterity and suturing skills. Serious games enhance voluntary play among trainees compared to virtual reality simulators, which make them interesting training solutions for busy professionals .
This study examines the influence of this custom-made serious game on surgical trainees’ problem recognition and problem-solving capabilities during equipment malfunctions in the laparoscopic OR. We hypothesize that trainees who follow a game-enhanced curriculum would recognize and solve more equipment-related problems than trainees who followed the regular basic laparoscopic training curriculum.
This randomized, single-blinded controlled trial was conducted at a tertiary academic center in the Netherlands. The institutional ethics committee has reviewed the study protocol and concluded that full review was unnecessary because it is not a clinical trial. The institutional animal studies review board approved the study.
Participants were residents in their first or second year of general surgical training participating in the standard basic laparoscopic training course (BLTC). They were required not to have any experience in MIS as a primary surgeon. After giving consent, participants were enrolled into either the control group (regular BLTC curriculum) or the intervention group (game-enhanced BLTC curriculum). Randomization was conducted using a sealed opaque envelope with equal probability of group allocation. Participants could not be blinded due to the nature of the intervention.
Setting: basic laparoscopic training course
The BLTC is an obligatory part of the surgical residency training curriculum in the Netherlands  and is based on FLS principles . The purpose of the course is to familiarize novice surgical trainees with laparoscopic principles, equipment, and basic dexterity. The 2-day curriculum consists of lectures on the principles of laparoscopic instrumentation, laparoscopic tower, pneumoperitoneum, electrocautery and vessel sealing, ergonomics, cholecystectomy, appendectomy and hernia repair, and technical skills training on a laparoscopic box trainer (peg transfer, cord placement, rubber band placement and cutting, and cholecystectomy on a cadaver liver), after which trainees complete a hands-on interactive training session on a live anesthetized pig model (trocar positioning, cholecystectomy, appendectomy).
The game-enhanced curriculum consisted of the regular BLTC, enhanced by two separate 30-min sessions of serious gaming, containing an estimated 10 play sessions (Fig. 1). The participants received an individual login and standardized instruction tutorial before commencing the game. The control group followed the regular BLTC curriculum. They had the opportunity to explore the laparoscopic equipment during the intervention groups’ gaming sessions.
Dr. Game, Surgeon Trouble® (WeirdBeard co., Amsterdam, The Netherlands) was designed to train surgical trainees in recognizing and responding to equipment problems of the laparoscopic tower  (Fig. 2). The game consists of a entertaining mini-game designed to attract the player’s attention. The main objective in the game is to create rows of three similar blocks—which is fun and challenging (Fig. 2A). The task, although not requiring any professional expertise, demands the player’s full attention. Meanwhile, the laparoscopic tower is virtually embedded in the gameplay, and in order to progress, one must be able to solve laparoscopic equipment-related problem scenarios. Changes in the environment (screen, sounds, values) signify equipment-related problems and malfunctions. Signals partly occur outside the player’s direct focus of attention, similar to the OR environment. The player scores extra points by timely recognizing the problem, after which he or she enters a troubleshooting mode in the game (Fig. 2B).
The player can solve the problem by selecting the correct equipment and actions, after which he or she can play again and ‘score’ again. The player receives direct feedback. This creates a continuing cycle of challenges, actions, and feedback. It can be expected that the player’s problem-solving ability in the laparoscopic operation room will improve by playing this specific serious game. Secondly, dealing with these situations in a game environment may also familiarize novice surgeons with the principles of situation awareness. The participants’ game performance is measured and stored in a database (the amount of scenarios recognized and solved, time required, and the amount of correct/incorrect actions required to solve the problem).
The game’s educational content includes problem scenarios of screen and lighting (19), gas transport and pneumoperitoneum (5), electrocautery (2), and pathophysiological disturbances related to MIS (2). Content has been previously validated by equipment specialists. The game screen relates to the camera and lighting, handling blocks to the electrosurgical unit, and the appearance of the visual field to the pneumoperitoneum. Per 3-min game session, the player encounters approximately six problem scenarios. The ‘troubleshooting mode’ (Fig. 2C) depicts a laparoscopic tower (Olympus Exera II CLV 180 light source, Olympus UHI-3 insufflator, Olympus Exera II CV-180 video processor, EndoEYE HD Video Laparoscope, and SurgMaster UES-40 electrosurgical unit (all: Olympus co., Tokyo, Japan)). The simulated MIS unit in the game corresponds to the unit used in the BLTC.
All participants performed two standardized tasks on a live anesthetized pig model as the primary surgeon, consisting of (1) searching the small bowel for a Meckel’s diverticulum and (2) performing a biopsy of the parietal peritoneum. During their procedure, they were assisted by two OR nurses and a camera navigator. Three standardized equipment problem scenarios occurred: (1) failure of the insufflation and pneumoperitoneum, (2) failure of the electrocautery unit, and (3) saturation change on the anesthesia monitor (Table 1).
Primary outcome measures were the proportion of problems recognized and solved per participant; secondary outcome measure was the time required to do so. These measures were calculated independently from each other. An independent assessor, blinded to group allocation, registered these parameters. Problem recognition was defined as the participant verbally or otherwise indicating that a problem had occurred <2 min after the onset of the “symptoms,” and problem solving was defined as solving it <2 min after problem recognition.
An experienced surgeon blinded to group allocation assessed the participants’ technical skills through an Objective Structured Assessment of Technical Skills (OSATS) form. This contains 7 items (tissue handling, movement, instrument handling, instrument knowledge, use of assistance, procedural progress, and procedural knowledge) scored on a 5-point Likert scale . The participants received a standardized instruction before the test, during which they were told that they would be judged on their technical performance only (OSATS). They were instructed to use and coach the OR personnel present as they would normally do and talk aloud in case of trouble.
The participants’ learning curves in the game-enhanced curriculum group were calculated to assess if their performance during gaming sessions improved and, thus, if learning did occur (% of problems solved per individual game session).
Prior to the trial, a pilot study was performed in which eight surgical residents with no MIS experience as primary surgeon were assessed using the set-up described above. Using an alpha of 0.05, a power of 0.80, a population standard deviation of 0.186, and an estimated effect size of 50%, the required size for each group was 12. The dropout rate was estimated at 20%.
Descriptive statistics were calculated for all variables. All data were not normally distributed and thus Mann–Whitney U tests were applied to calculate the differences in the primary and secondary outcome measures. Subgroup analyses were performed assessing the performance of both groups on the individual problem scenarios. Differences were calculated using Pearson Chi-square tests. To estimate the learning curves during the game-enhanced curriculum group’s individual game sessions, a random effects model was calculated. Analyses were performed using the IBM Statistical Package for Social Sciences version 20 (IBM corp., Armonk, NY, USA) and R version 2.15 (R Foundation for Statistical Computing, Vienna, Austria).
Thirty-one surgical residents were randomized between May 2013 and April 2015. In total, 24 completed the curriculum and the assessment according to protocol (12 per group, Fig. 3). Four participants did not complete the test because of an incident occurring at the test site, requiring it to be closed down. Three participants were removed from the analysis due to protocol violations. Disturbances during the test caused heterogeneity of surgical circumstances deviating beyond normal variability.
None of the participants had experience as primary surgeon in MIS. The game-enhanced curriculum group completed an average of 11.8 game sessions, relating to about 60 min of gameplay. There were no differences between age, gender, technical score (OSATS), postgraduate year, or experience in conventional surgery. The regular curriculum group contained slightly more residents in their 2-year preparatory training for a specialty other than general surgery (e.g., orthopedic, plastic, cardiothoracic surgery, or urology). An overview of the demographic characteristics is shown in Table 2.
Primary and secondary outcome parameters
The participants in the game-enhanced curriculum group solved a median of 59% (interquartile range (IQR) 33–67%) of the problems presented to them, compared to 33% (8–33%) in the regular curriculum group (Fig. 4, p = 0.03). Participants in the game-enhanced curriculum group recognized a median of 67% of the problems (IQR 33–92%), compared to 42% (33–67%) in the regular curriculum group (Fig. 4, p = 0.14).
The game-enhanced curriculum group recognized problems in a median of 66 s (IQR 52–85 s) vs. 80 s (68–86 s) in the regular curriculum group, p = 0.24. The game-enhanced curriculum group solved a problem in a median of 75 s (IQR 46–88 s) vs. 85 s (75–101 s) in the regular curriculum group, p = 0.14.
A subgroup analysis of problem types recognized and solved by intervention and control groups is shown in Table 3. The difference in the total number of problems solved between the intervention and control groups was statistically significant (20/36 vs. 11/35, p = 0.04). The most obvious difference between the intervention and control groups was observed in the participant’s ability to recognize (8/12 vs. 4/11, p = 0.14) and deal with insufflator malfunctions (8/12 vs. 3/11, p = 0.06).
The participants in the game-enhanced curriculum group each completed a mean of 11.8 game sessions (SD 1.7), in which they played a mean of 63.9 problem scenarios (SD 13.0). In the first four game sessions, they solved a mean of 48.1% of the problems (SD 14.5), in the second four sessions 54.5% (SD 9.9), and in the third four sessions 69.3% of the problems (SD 14.5). Their learning curve during the game sessions was estimated using a linear regression model with random intercepts (Fig. 5). This shows a 2.3% improvement in the proportion of solved cases per three-min session (p < 0.001).
This randomized controlled trial effectively demonstrates that 1 h of practice on a custom-made serious game results in an improved problem-solving performance concerning equipment-related problems in the MIS theater. The importance of training laparoscopic equipment failure scenarios is emphasized by the relatively poor performance of the control group, showing that the current laparoscopy training courses are insufficient to train pupil’s ability to recognize and respond to non-routine events in the MIS environment. Participants in the game group were not only able to solve more equipment-related problems, but also more likely to solve equipment failure problems that did not occur in their area of focus (e.g., insufflation of the pneumoperitoneum and vital parameters), indicating an improved situational awareness. This has considerable implications for both the surgical training curriculum and patient safety in the OR.
Serious gaming is an innovative training method that is currently being explored in medical pre- and postgraduate training [15–17]. A well-designed serious game appeals to the intrinsic motivation of the trainee to play, while educational content is fitted in a subtle, ‘stealthy’ fashion . Through repeated, voluntary interaction with the content, games lead to experiential learning . Although the effectiveness of serious games to enhance ‘technical’ surgical skills has been shown in previous studies [20–22], this is the first study in which a serious game is systematically assessed for its capacity in training the ability to anticipate non-routine adverse events in the surgical theater.
Although both primary outcome parameters (problems recognized and solved) show a reasonable effect size, only the latter is statistically significant. The range in participants’ ability to recognize problems was larger than the range in the ability to solve problems, which accounts for the difference in statistical significance.
Logically, a higher number of participants would have led to statistical significance in this parameter too. A trend is also seen in the time required by the participants to recognize and solve problems in favor of the intervention group. This parameter is of lesser importance to novice learners, whereas acting haphazardly is more dangerous in the OR than acting slow and consciously.
Other studies have proven the effectiveness of serious games for laparoscopic psychomotor skills training (‘technical’ skills). Jalink et al. compared the performance of surgeons and non-surgeons on a specifically developed Wii™-based serious game and a laparoscopic box trainer, finding a significant, high correlation . Badurdeen et al. found similar correlations between performance on Wii™-based entertainment games and laparoscopic box trainer scores . Youngblood et al. compared the training results of medical students in trauma management between a serious game and patient simulator, finding a significant, comparable improvement in skills in terms of a behavioral performance evaluation scale . To our knowledge, this study is the first randomized study to prove the effect of a serious game in terms of performance improvement in the surgical environment (i.e., predictive validity).
A second strength is that the serious game intervention was applied within the regular curriculum. Participants in the control group participated in the customary BLTC, which includes lectures on laparoscopic instrumentation and the laparoscopic tower. These lectures include handling specific equipment-related problem scenarios. This substantially increases the generalizability of the study results, whereas the systematic game-enhanced curriculum and the regular curriculum are compared and not merely a “trained” and a “non-trained” group.
A limitation of this study is the relatively high dropout rate (22.5%). Although 31 participants were initially recruited, only 24 completed the post-test according to the protocol, equaling the minimum required number in the power analysis. A higher inclusion number was deliberately obtained because of suspected high dropout rates based on literature . All participants failed due to logistical reasons and none refused to partake in the test. The test protocol was complex, relying heavily on the performance of the study personnel in staging the test setting and equipment failure scenarios. This led to protocol violations in four occasions. Ultimately, group sizes and baseline characteristics (technical performance, previous surgical experience, and demographic characteristics) did not differ significantly. Selection bias due to dropout therefore seems limited. Future research on situational awareness in the OR should limit the complexity of the study protocol and reliance on trained personnel.
A second limitation is that retention of learning in time was not measured. Due to the use of live animal models and participants in clinical employment in multiple teaching hospitals, repeated measurements would have posed great logistical challenges. From a practical point of view, one may state that, whereas this study proves that gameplay improves performance, continued gameplay is likely to maintain this level of skill.
The serious game used in this study is unique in the sense that the gameplay resembles a popular arcade-type animated game, in which important surgical content was embedded. Whereas most currently available medical serious games apply realistic graphical simulations to mimic reality , this animated approach has deliberately been chosen to preserve interest of the player, which is especially novel to the field. It has been shown that high graphical fidelity to the medical construct (e.g., near-perfect graphical depiction of the operation room) is not necessary to teach important medical content, as long as the game’s functional fidelity (e.g., resemblance of important ‘cues’ in the action or procedure) remains adequate . This implies that future surgical training does not necessarily have to take place in realistic e-learning or simulation modules, but could be delivered through entertaining and attractively animated videogames. Simple and compelling games are known to be fun, reinforcing, and even addictive . This aims to captivate the user and improve the interaction time.
The place of serious gaming in the surgical residency curriculum is somewhat ambiguous. Its main advantage is the ability to invoke ‘voluntary play’ by using motivational triggers such as competition and attractive gameplay . This distinguishes serious games from less challenging simulators, which are frequently left untouched by trainees, unless they are obligated . As the optimal effect of serious games is reached through the trainee’s intrinsic motivation (‘voluntary play’), ‘obligatory play’ of serious games in the surgical curricula thus seems to be a contradiction in terms. However, because non-routine events training carries clinical importance in terms of patient safety, the achievement of a minimally required level of expertise for trainees seems inevitable.
Conclusions and recommendations
This randomized controlled trial shows that surgical trainees that follow a game-enhanced curriculum have a significantly higher ability to solve equipment-related problems in the MIS theater than surgical trainees that follow the regular curriculum. Equipment failure is known to lead to procedural delays and represents a potential threat to patient safety. Future research should determine the value of ‘voluntary play’ of serious games compared to an obligated minimally required level of performance and relate these findings to the long-term retention of performance.
Zheng B, Cassera MA, Martinec DV et al (2010) Measuring mental workload during the performance of advanced laparoscopic tasks. Surg Endosc 24:45–50. doi:10.1007/s00464-011-2038-x
Weerakkody RA, Cheshire NJ, Riga C et al (2013) Surgical technology and operating-room safety failures: a systematic review of quantitative studies. BMJ Qual Saf 22:710–718. doi:10.1136/bmjqs-2012-001778
Verdaasdonk EGG, Stassen LPS, van der Elst M et al (2007) Problems with technical equipment during laparoscopic surgery. An observational study. Surg Endosc 21:275–279. doi:10.1007/s00464-006-0019-2
Courdier S, Garbin O, Hummel M et al (2009) Equipment failure: causes and consequences in endoscopic gynecologic surgery. J Minim Invasive Gynecol 16:28–33. doi:10.1016/j.jmig.2008.08.019
Buzink SN, van L L, de Hingh IH et al (2010) Risk-sensitive events during laparoscopic cholecystectomy: the influence of the integrated operating room and a preoperative checklist tool. Surg Endosc 24:1990–1995
Peters JH, Fried GM, Swanstrom LL et al (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135:21–27. doi:10.1016/S0039-6060(03)00156-9
Graafland M, Bemelman WA, Schijven MP (2014) Prospective cohort study on surgeons’ response to equipment failure in the laparoscopic environment. Surg Endosc 28:2695–2701. doi:10.1007/s00464-014-3530-x
Menezes CA, Birch DW, Vizhul A et al (2011) A deficiency in knowledge of basic principles of laparoscopy among attendees of an advanced laparoscopic surgery course. J Surg Educ 68:3–5. doi:10.1016/j.jsurg.2010.09.005
Michael DR, Chen S (2006) Serious games: games that educate, train, and inform, 1st edn. Thomson Course Technology, Boston
Grantcharov TP, Kristiansen VB, Bendix J et al (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150. doi:10.1002/bjs.4407
Rosser JC, Lynch PJ, Cuddihy L, et al. The impact of video games on training surgeons in the 21st century. Arch Surg 2007;142:181–6; discussion 186. doi:10.1001/archsurg.142.2.181
Verdaasdonk EGG, Dankelman J, Schijven MP et al (2009) Serious gaming and voluntary laparoscopic skills training: a multicenter study. Minim Invasive Ther Allied Technol 18:232–238. doi:10.1080/13645700903054046
Hamming J, Borel Rinkes IHM, Heineman E (2009) Scherp: structured curriculum for surgery for reflective professionals (Structure Curriculum Heelkunde voor Reflectieve Professionals [Dutch]). Opleidingsplan Heelkunde. 2009;08-22235_3. http://knmg.artsennet.nl
Martin JA, Regehr G, Reznick R et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278. doi:10.1046/j.1365-2168.1997.02502.x
Graafland M, Schraagen JMC, Schijven MP (2012) Systematic review of serious games for medical education and surgical skills training. Br J Surg 99:1322–1330. doi:10.1002/bjs.8819
de Wit-Zuurendonk LD, Oei SG (2011) Serious gaming in women’s health care. BJOG 118:17–21. doi:10.1111/j.1471-0528.2011.03176.x
Akl EA, Kairouz VF, Sackett KM et al (2013) Educational games for health professionals. Cochrane Database Syst Rev 3:CD006411. doi:10.1002/14651858.CD006411.pub4
Sharp L (2012) Stealth learning: unexpected learning opportunities through games. J Instr Res 1:42–48. doi:10.9743/JIR.2013.6
Kiili K (2005) Digital game-based learning: towards an experiential gaming model. Internet High Educ 8:13–24. doi:10.1016/j.iheduc.2004.12.001
Jalink MB, Goris J, Heineman E et al (2014) Construct and concurrent validity of a Nintendo Wii video game made for training basic laparoscopic skills. Surg Endosc 28:537–542. doi:10.1007/s00464-013-3199-6
Badurdeen S, Abdul-Samad O, Story G et al (2010) Nintendo Wii video-gaming ability predicts laparoscopic skill. Surg Endosc 24:1824–1828. doi:10.1007/s00464-009-0862-z
Rosenberg BH, Landsittel D, Averch TD (2005) Can video games be used to predict or improve laparoscopic skills? J Endourol 19:372–376. doi:10.1089/end.2005.19.372
Youngblood P, Harter PM, Srivastava S et al (2008) Design, development, and evaluation of an online virtual emergency department for training trauma teams. Simul Healthc 3:146–153. doi:10.1097/SIH.0b013e31817bedf7
Palter VN, Grantcharov TP (2012) Development and validation of a comprehensive curriculum to teach an advanced minimally invasive procedure: a randomized controlled trial. Ann Surg 256:25–32. doi:10.1097/SLA.0b013e318258f5aa
Alexander AL, Brunyé T, Sidman J et al (2005) From gaming to training: a review of studies on fidelity, immersion, presence, and buy-in and their effects on transfer in PC-based simulations and games. In: Proceedings from the 2005 interservice/industry training, simulation, and education conference (I/ITSEC). Arlington, pp. 1–14
Hoffman E Life, addictive game mechanics, and the truth hiding in bejeweled. http://www.gamasutra.com/blogs/ErinHoffman/20090916/85505/Life_Addictive_Game_Mechanics_And_The_Truth_Hiding_In_Bejeweled.php
Chang L, Petros J, Hess DT et al (2007) Integrating simulation into a surgical residency program: is voluntary participation effective? Surg Endosc 21:418–421. doi:10.1007/s00464-006-9051-5
The authors wish to thank the following persons for their contribution in the conduct of this study: Prof. T.M. van Gulik, Dr. E.J.M. Nieveen van Dijkum, Dr. S.M. Lagarde, Dr. M.G. Besselink, Drs. S.L. Gans, Drs. K. Ram, Drs. F. Huisman, Ms. E. Barsom, Ms. L.E. Philipszoon, Mr. W. van Riel (Academic Medical Centre, Amsterdam, The Netherlands), Dr. G. Heuff (Spaarne Hospital, Hoofddorp, The Netherlands), and Dr. S. Siregar (Leiden University Medical Centre, Leiden, The Netherlands).
Pieken in de Delta-program of the Ministry of Economic Affairs, Agriculture and Innovation, the city of Utrecht and the province of Utrecht (Grant ref. PID 101060, Patient safety project). Research grant from CBuzinez (Innovation platform from CZ Healthcare Insurance Agency). Funding agencies had no role in the design and conduct of the study, collection and interpretation of the data, or the preparation and approval of the manuscript.
Maurits Graafland, Willem A. Bemelman, and Marlies P. Schijven have no conflict of interest or financial ties to disclose.
The study protocol was reviewed and approved by the institutional research ethics committee. This study was conducted in full concordance with the Dutch law on animal experiments.
About this article
Cite this article
Graafland, M., Bemelman, W.A. & Schijven, M.P. Game-based training improves the surgeon’s situational awareness in the operation room: a randomized controlled trial. Surg Endosc 31, 4093–4101 (2017). https://doi.org/10.1007/s00464-017-5456-6