Laparoscopic surgery requires complex techniques and skills that are not employed in open surgery. The instruments provide limited haptic feedback, lack degrees of freedom, and move inverted inside the abdomen [1, 2]. Furthermore, video monitors that provide a 2-dimensional projection of the operating field impair depth perception and are moved away from the patient [3, 4]. Surgeons and residents in surgery have to master these technical skills and challenges before they can perform any laparoscopic procedure appropriately and safely.

How to teach laparoscopic surgery to residents in a safe and efficient way is the topic of many debates, conventions, and research projects [5]. Rasmussen’s model of human behavior in laparoscopic training, as described by Wentink at al. [6], identified three levels of behavior that have to be trained, namely, skill-based behavior, rule-based behavior, and knowledge-based behavior. Skill-based behavior in laparoscopy is best described as the set of technical skills that are needed in every procedure and comprises motor movements that are continuously regulated by feedback systems. Rule-based behavior is more complex and comprises specific sets of procedural steps that are performed according to stored rules. A sign serves to activate or trigger a stored rule. For example, having applied ligation clips on the cystic duct and artery in a laparoscopic cholecystectomy is the sign that triggers the rule that these structures can be transected next. Knowledge-based behavior is encountered when no rules are available, for instance, when a complication or an unexpected anatomical variation is encountered. Different plans of behavior are evaluated against the anticipated goal.

Traditionally, surgery has been taught following the master-apprentice model (MAM). In this model the surgical trainee learns to perform surgical procedures under the supervision of a qualified surgeon. The supervising surgeon instructs the trainee and, when necessary, he temporarily takes over the procedure to show a difficult step. Nowadays, it is no longer accepted that a novice learns skill-based behavior on patients as there are validated training platforms available for practicing basic skills for both conventional and laparoscopic surgery that avoid patients being exposed to early learning curves [7, 8]. Basic laparoscopic motor skills can be practiced repeatedly on box trainers, virtual reality (VR) trainers, and augmented reality (AR) trainers [9, 10]. VR trainers allow repeated practice of various exercises and record parameters such as instrument path length, collisions, and time to objectively score the trainee’s performance on these exercises. Some VR trainers are compact and use “plug and play” technology so they can be taken home for practice. However, a disadvantage of most VR trainers is the lack of haptic feedback for instrument and tissue handling [11]. Box trainers, on the other hand, provide haptic feedback and can be used for both basic skills and for procedure training using cadaver organs, but they require the use of surgical instruments and disposable materials [12].

Besides technical skills, a trainee has to acquire knowledge of the procedure itself, knowledge of the pitfalls of the procedure, and coping strategies when a problem presents itself, skills defined by Rasmussen as rule-based and knowledge-based behaviors [6]. In general, teaching these types of behavior is more complex and costly than teaching skill-based behavior. Animal model training and cadaver training are very helpful, but also resource intensive and not available on demand. Books, internet courses, and instruction videos can provide important fundamental knowledge of an illness and its surgical treatment [13]. However, the most important element in training a specific surgical procedure remains the hands-on training on a real patient with an experienced surgeon at the trainee’s side.

At the beginning of a trainee’s learning curve, it is likely that the supervising surgeon frequently takes over the procedure to demonstrate case-specific rule-based and knowledge-based behavior. A major disadvantage of this training model is that steps can be performed only once per procedure by either the trainee or the supervising surgeon. When the supervisor takes over, that part is lost to the trainee who has to wait for the next operation to perform the step himself. To minimize the frequency of this intervention by the supervisor and maximize the actual operating time for the trainee, we created a new training method called INtraoperative Video-Enhanced Surgical Training (INVEST). This method implements instruction videos to train rule-based and knowledge-based behaviors by demonstrating key elements and essential tips and tricks of the procedure step-by-step and on-demand without the need for the supervisor to demonstrate them by taking over. Therefore, these steps are preserved and can be performed by the trainee.

In the current study we investigated the effect of INVEST on the early learning curve in surgical procedure training inside the operating theater by showing short intraoperative instruction videos to surgical trainees.

Methods

Study design

This study was a randomized controlled trial with repeated measurements among trainees who were randomly assigned to a series of laparoscopic cholecystectomies utilizing either INVEST or the usual MAM. Intraindividual improvement of surgical skills was evaluated across students trained in either group. We preferred a baseline follow-up study design as it rules out a large number of confounding factors that are likely to occur when comparing separate outcomes between independent groups. Furthermore, we controlled for equal levels of surgical skills at baseline in order to avoid differences in outcome that are due to initial differences among participants.

Procedure

The INVEST instruction video was created in conformity with the guidelines for laparoscopic cholecystectomy as formulated by the Association of Surgeons of the Netherlands [14]. These guidelines are similar to the guidelines formulated by SAGES and EAES, with the addition of the importance of the Critical View of Safety (CVS) [15]. Chronologically, all the separate steps of the procedure that are described in the guidelines were clustered into seven clearly identifiable stages: (1) open introduction of the first trocar, (2) accessory trocar placement, (3) opening of peritoneal envelope, (4) creating the CVS, (5) clipping and division of cystic duct and artery, (6) retrograde cholecystectomy, and (7) gallbladder removal and closure. For each of the seven stages a 1-min video clip was created, demonstrating anatomical landmarks, key elements, and operative techniques essential to that particular phase of the procedure. Video clips were displayed on demand on a second screen next to the operative screen when the trainee was ready for the next step of the procedure. For safety reasons, neither the trainee nor the supervising surgeon was allowed to continue the procedure while the instruction video was playing. After completion of each video clip, a written summary appeared and was displayed on the accessory screen while the trainee performed the next step.

Trainee selection

Ten trainees were included in this study. All trainees were registered residents in surgery, were in the early phase of their training, and resided at the department of surgery at Leeuwarden Medical Center. Criteria for inclusion were at least 6 months of experience in open surgical techniques and the successful completion of a training course in basic surgical skills. Exclusion criteria were any hands-on experience with laparoscopic cholecystectomies and a cumulative experience of more than five cases in other laparoscopic procedures.

Trainee preparation

Since the trainees had no previous experience in laparoscopic techniques, they also had no practical experience with the basic motor skills that are unique to laparoscopic procedures. These skills have to be mastered before anyone can be safely and efficiently trained in a specific laparoscopic procedure. Therefore, before randomization, all residents scheduled for this study developed their basic laparoscopic skills on the SIMENDO laparoscopy trainer (Simendo, Rotterdam, The Netherlands). This validated VR simulator has a variety of exercises and is supplied with a proficiency-based technical skills training curriculum [16]. Successful completion of the SIMENDO curriculum indicates an adequate level of proficiency in basic laparoscopic technical skills to allow safe participation in laparoscopic procedures on humans [17]. As an additional result of the curriculum, the technical skills of all the trainees were calibrated at an equal level.

After completing the SIMENDO curriculum, residents were randomly assigned to one of the two arms of this study by drawing a sealed envelope. In both groups, each resident performed six laparoscopic cholecystectomies within 2 weeks. Residents prepared themselves for these procedures in standard fashion using textbooks, anatomy books, and online information. During the procedure itself, the control group was trained using MAM. The experimental group, in addition to being supervised by a qualified surgeon, was trained with INVEST.

Patient selection and supervision

Patients with uncomplicated symptomatic gallstone disease were selected for this study. All patients were asked to give informed consent that a resident would perform the procedure under the supervision of a qualified surgeon. Since the procedure itself did not differ between the experimental and control groups, informed consent was not needed for using INVEST.

Three dedicated laparoscopic surgeons were randomly assigned to supervise the procedures in both groups. They were conversant with the latest guidelines and approved the content of the instruction video. The supervising surgeons were not informed of the progression of the trainee in the course of the six cholecystectomies nor were they informed of previous scores. The surgeons guarded the safety and the flow of the procedure, they gave verbal instructions, and, when necessary, they temporarily took over the procedure. The time and reason for temporarily taking over the procedure was decided on the supervising surgeon’s professional autonomy.

Operating theater setup

All procedures were performed in dedicated minimally invasive surgery (MIS) suites. An MIS suite is a fully integrated operating room (OR) in which laparoscopic equipment and multiple flat-screen monitors are permanently installed to be operational on demand. In the INVEST setting, two monitors were facing the operator and the supervisor, providing an ergonomically safe posture. One monitor displayed the operative image and the other was used for the instruction video. A third monitor displayed the operative image for the scrub nurse. A research fellow who was present during the procedure played the instruction video on demand from a computer that was linked to the designated flat-screen monitor.

Objective Structured Assessment of Technical Skill (OSATS)

After each procedure, the supervising surgeon evaluated the skills of the trainee on a modified and translated version of the seven-question global rating scale that is used as part of the objective structured assessment of technical skill (OSATS) as described by Martin et al. [18] This modified OSATS global rating scale is used nationwide and is part of the mandatory digital portfolio for every resident in surgery in The Netherlands. Using a 10-point scale, it grades the trainee on seven important elements of any operation, concerning a combination of skill-based, rule-based, and knowledge-based behaviors: (1) respect for tissue, (2) time and motion, (3) instrument handling, (4) knowledge of instruments, (5) use of assistants, (6) flow of operation, and (7) knowledge of the procedure.

Trainee’s opinion questionnaire

After completion of the six cholecystectomies, the trainees in the INVEST group were invited to give their opinion of the training method via a brief questionnaire. To assess the attitude toward INVEST among participants who were not acquainted with it in practice, the instruction video was also shown to the trainees in the MAM group. These trainees were also invited to fill out this questionnaire. The questionnaire comprised seven statements that had to be rated on a 5-point Likert scale where 1 = totally disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = totally agree.

Statistical methods

To evaluate the effect of INVEST, we were interested in the improvement of skills during the training curriculum. The individual OSATS scores for each procedure are snapshots and do not represent a learning curve. Therefore, we used the OSATS score of the first procedure as a baseline and calculated the improvement from the baseline during the following procedures. For each trainee, the absolute improvement on the OSATS score was calculated for each of the seven individual items and for the complete OSATS scale. In addition, we calculated the relative improvement on the OSATS scale that estimates the maximum score each trainee was able to achieve. With this relative improvement we estimated the percentage that each trainee improved from the first OSATS score (procedure 1) toward the maximum OSATS score at follow-up (procedure 6). Relative improvement in the seven OSATS skills and overall OSATS scale was calculated as follows:

$$ {\frac{{{\text{Final \,OSATS \,score\, (procedure 6)}} - {\text{initial \,OSATS \,score\,(procedure 1)}}}}{{{ \max } . {\text{achievable \,OSATS \,score}} - {\text{initial\, OSATS \,score\, (procedure 1)}} }}} \times 100\% $$

Reliability

The reliability of the seven-item modified OSATS scale was examined with the internal consistency coefficient Cronbach’s alpha [19]. Since Cronbach’s alpha is dependent on the number of items in the scale and on the mean inter-item correlation (MIIC), one can achieve a high reliability estimate by having either many items or highly intercorrelated items (or a combination of the two) [20, 21]. According to the guidelines by Briggs and Cheek [22], the MIIC should fall in an optimal range between 0.20 and 0.50 but should not be less than 0.15 [20, 22, 23]. Therefore, taking the upper value of the range, an MIIC ≥ 0.25 seems reasonable. For the seven-item OSATS global rating scale in this study, a Cronbach’s alpha coefficient of 0.70 was minimally acceptable.

Given the small sample size, the differences in absolute and relative improvements in skills between the INVEST group and the master-apprentice group were compared with the nonparametric Wilcoxon Mann–Whitney test for ordinal data. Effect sizes were calculated only for statistically significant differences, as it makes no sense to estimate clinical relevance of a result that is based on random variation. Cohen’s effect size (ES) for independent samples was used to estimate the magnitude of these differences [24]. According to Cohen’s thresholds, an ES < 0.20 indicates a trivial difference, 0.20–0.50 a small difference, 0.50–0.80 a moderate difference, and >0.80 a large difference.

Results

Ten trainees were randomly assigned to the two arms of the study, with no dropout after inclusion. Each trainee successfully completed the basic skills training curriculum on the SIMENDO to the preset level of proficiency before randomization. There were no differences between the groups with respect to training time to acquire the proficiency level. Each resident performed six laparoscopic cholecystectomies within the set period of 2 weeks. Each procedure was evaluated by the supervising surgeon using an OSATS global rating scale. There were no technical problems with displaying the instruction video in the INVEST group.

Reliability of the OSATS global rating scale was determined. The OSATS consists of seven items, and summed scores of these items indicate the extent of performance of technical skills. A lower overall score means a poor performance while a higher score indicates good to excellent performance. The internal consistency of the seven-item OSATS overall performance scale was good and yielded a Cronbach’s alpha of 0.88.

The OSATS scores for the first procedure were not statistically different between the groups. Analysis of the absolute and relative improvements in the seven separate skills on the OSATS global rating scale indicated a statistically significant (P < 0.05) and clinically relevant (ES > 0.80) difference in skills acquisition in favor of the INVEST group for the following skills: time and motion, use of assistants, flow of operation, and knowledge of the procedure (Table 1). The absolute and relative skill improvements on the complete OSATS global rating scale were also significantly higher in the INVEST group (Table 1). The relative improvement during the six procedures is graphically displayed in Fig. 1.

Table 1 Absolute and relative improvements on the seven-item OSATS global rating scale for the separate items and the sum score in the INVEST group and the MAM group
Full size table
Fig. 1
figure 1

Relative improvement for the INVEST and MAM group during the curriculum. * indicates statistical significance

Full size image

The trainees in the INVEST group totally agreed with the statements that intraoperative video training is fun, it has a positive effect on the learning curve, and it is a uniform means of learning laparoscopic cholecystectomy. They agreed on the statements that they were allowed to do more steps of the procedure, the supervisor had to intervene less frequently during the procedure, and they would also like to have this type of training for other procedures. They were neutral on the statement that INVEST would be useful after six procedures. The answers in the MAM group were similar and not statistically different (Table 2).

Table 2 Trainees’ opinion (mean) on the INVEST curriculum
Full size table

Conclusions and discussion

This study was conducted to explore the potential benefit and the trainee acceptance of intraoperative video-enhanced surgical procedure training for laparoscopic surgery. The INVEST curriculum significantly enhanced skill development during the early learning curve for laparoscopic cholecystectomy. Trainees indicated being positive about the content and the idea of the curriculum.

In our opinion, INVEST should not be seen as a stand-alone training curriculum. It should be an integrated part of a complete laparoscopic curriculum that teaches all aspects of skill-based, rule-based, and knowledge-based behavior. Therefore, a balanced training program commences with essential basic skills training on VR and/or AR simulators. Elements of procedures should be practiced in box trainers with cadaver models [25]. Ideally, trainees should attend courses that use live animal models or human cadavers to perform specific procedures on healthy organs before they go to the operating theater to perform their first procedures on real patients with INVEST.

This study did not demonstrate a significant difference in skill acquisition for three of the seven items on the OSATS global rating scale: (1) respect for tissue, (3) instrument handling, and (4) knowledge of instruments. In our opinion, respect for tissue is part of knowledge-based behavior and cannot be trained with a video. Appreciation of tissue is individually determined and comes with experience. Instrument handling cannot be trained with a video. The SIMENDO training curriculum provided the initial training in instrument handling. Further acquisition of instrument handling skills comes with experience. Knowledge of laparoscopic instruments was not part of the INVEST video and therefore we were not surprised with these results. We are exploring the possibilities of adding this skill to the video. All the other items on the OSATS scale that did significantly improve with the INVEST curriculum were part of the training video.

A possible weakness of this study is the small group size, which makes it vulnerable for type I error. While designing this study, measures were taken to minimize this risk. First, the level of surgical and laparoscopic experience among the trainees had to be very uniform on admission. None of them had noteworthy laparoscopic experience and, before randomization, each trainee was identically prepared with the SIMENDO basic laparoscopic skills curriculum. Second, we tried to score the performance of the trainees as uniformly as possible. The OSATS global rating scale is a validated tool for evaluating technical skills in a reliable and reproducible manner [18]. Data derived from the OSATS global rating scale in this study proved to be internally consistent. In addition to comparing OSATS scores between the groups after completion of the curriculum, we calculated and compared the individual improvement in skills for each trainee from the initial to the final procedure in the curriculum. When we compare improvement, each trainee has his or her own baseline and completion scores that translate into a more reliable outcome than just comparing the completion scores between the groups.

Finally, we tried to minimize bias caused by the supervising surgeon who also evaluated the procedure and filled out the OSATS global rating scale. By using three different surgeons in random order, the surgeons were not aware of a trainee’s performance on previous procedures. The supervising surgeons could obviously not be blinded for the different arms of the study. Furthermore, comparing absolute and relative skill improvements instead of OSATS scores also corrected for potential bias caused by supervisors overrating trainees in the INVEST group. A possible way to avoid this observer bias would be the employment blinded and independent surgeons who perform the assessment postoperatively by means of procedure videos. However, to our knowledge there is no validated tool for scoring surgical skills on procedure videos. For future analyses, we are exploring the possibility of scoring procedure videos by means other than OSATS.

The outcome of this study in exploring the feasibility and potential benefit of INVEST for procedure training inside the OR is very promising. Future developments within the laparoscopic cholecystectomy video will comprise the inclusion of instrument handling and knowledge of instruments, since these skills were not addressed in the present video. In addition, we are considering employing INVEST for more advanced laparoscopic procedures.

In conclusion, we recommend INVEST for procedure training inside the OR, providing a uniform, efficient, and stimulating training environment that also appreciates patient safety. INVEST supports supervising surgeons in coaching trainees in their early learning curve after the transition from skills lab to the procedure training operating theater. It improves the early learning curve, it is easy to use in daily practice, and it costs very little extra time.