Advertisement

Effect of structural training on surgical outcomes of residents’ first operative laparoscopy: a randomized controlled trial

  • Ewa JokinenEmail author
  • Tomi S. Mikkola
  • Päivi Härkki
Open Access
Article

Abstract

Background

Gynecological surgery and resident education have changed during recent decades, thus impacting surgical training. Training on simulators must begin before operating on patients. The objective of this study was to evaluate the effect of a simple curriculum on the surgical outcome of the participants’ first operative laparoscopy.

Methods

This randomized prospective interventional study was carried out in Helsinki University Hospital and Hyvinkää Hospital. We recruited twenty junior residents in Obstetrics and Gynecology, of which half formed a control group and the rest completed the intervention with a theoretical and a practical part. The participants’ first laparoscopic salpingectomy was assessed from video recordings by using Objective Structured Assessment of Technical Skills (OSATS) forms and the Numeric Rating Scale (NRS). The surgical outcome and assessed scores were compared between the groups.

Results

We found no differences in operative time, blood loss, or complications, nor in OSATS or NRS scores. In the intervention group, participants with the weakest performances in the simulator, seemed to benefit from the training program more than the participants with the best performances (skill level elevation 29.2–31.6% vs. 21.1–23.3%, respectively). The participants with the best performances in the simulator were scored among the best in the recorded operations as well.

Conclusion

In this study, we found no difference in the surgical outcome between the groups. However, the participants with low starting levels in the simulator could elevate their skill levels more, though they did not reach the skill level of those with a high starting level. Consequently, we found elevation in skills levels in the simulator tasks, but not in the surgical outcome. Likely, our simple training program with a fixed number of repetitions was insufficient to reach a plateau in the learning curve, and thus the training program in such a curriculum should be proficiency based.

Keywords

Surgical education Resident education Virtual simulator 

During recent decades, gynecological surgery and training has experienced many radical changes, such as restrictions in working hours [1], new options in medical treatment [2, 3], and delicate office procedures [4]. For residents, this means fewer surgical cases in the operating room.

As live surgical experience is decreased during residents’ surgical rotation [1], they would benefit more from the live cases if they would be well pre-trained. Residents would be able to better focus on pathological anatomy and surgical techniques if they had acquired basic surgical ability and knowledge in a skills lab before operating on patients. More importantly, this would increase patient safety.

Both trainer boxes and virtual simulators have proven to be beneficial in surgical education [5], and now different kinds of curriculums are developed to implement them to training programs [6]. Many published training programs aim to achieve skills to perform a specific operation or task, and they include training with specific procedural modules [7, 8]. In this study, we developed a simple curriculum that contains a cognitive part and a practical part with basic skills only, and evaluated the effect of this curriculum on surgical outcomes of residents’ first laparoscopic salpingectomy.

Materials and methods

This trial was designed as randomized, interventional, blinded trial with two parallel study groups with 1:1 allocation ration. We recruited 20 residents between June 2013 and December 2016 from the Helsinki University Hospital or from Hyvinkää Hospital. Each new inexperienced resident was invited to participate, and none of them refused. As inclusion criteria, no operative laparoscopies as a primary surgeon were allowed, only diagnostic laparoscopies or clip sterilizations were permitted, as well as to assist in more advanced laparoscopies. Of the participants, ten were randomized to the intervention group, and ten to the control group by a sealed envelope manner. Randomization was done by research assistant outside our study group.

The intervention comprised a web-based theoretical course (Basics in Gynecological Laparoscopy) [9], and of a practical part with LAP Mentor virtual reality simulator (Simbionix Corporation, Cleveland, Ohio, USA). The cognitive material contains knowledge on pelvic anatomy, a laparoscopic technical part including instruments, laparoscopic unit and energy sources, operative phases, gynecological operations and complications, and a test following the actual course. With the simulator, all the nine basic skills tasks were practiced five times each, and the rehearsals were recorded for further evaluation. The nine basic skills tasks included camera manipulating, eye-hand coordination, clip application, clipping and grasping, two-handed maneuvers, cutting, electro surgery, and translocation of objects. From each skill tasks, three to four automatically stored parameters best describing the skill were used in the analysis. The chosen parameters included accuracy rate, total time used, and total path length of an instrument and were 32 in total. Both the web course and the simulator training were required to be completed within a month before the assessment. Both groups took part in similar standard clinical education including patient care in wards and clinical lectures. In the operating room, no operative laparoscopies as a first surgeon were permitted during the study.

The operation to assess the effect of intervention was salpingectomy in the right side; the left tube was alternative if the right one had already been removed. The surgery was video recorded for evaluation. The patients were scheduled to be operated because of a benign condition, commonly salpingo-oophorectomy because of an ovarian cyst or hysterectomy because of abnormal uterine bleeding. In case of salpingo-oophorectomy, first the tube was removed by the resident participating in the study, and thereafter the procedure was continued by the assisting senior. If necessary, the assisting specialist verbally instructed the resident as in any operation where the resident is the first surgeon. Demographics included the resident’s age, gender, working history, numbers of basic laparoscopic operations done, and history of video and instrument playing. The patient data were collected from the medical records and included age, body mass index, previous abdominal surgery, and indication for surgery.

The video recordings were assessed by three assessors (the authors) being blinded for the operator and the randomized group. The assessment was done by using Numeric Rating Scale (NRS) and both OSATS (Objective Structured Assessment of Technical Skills) form for Global Rating Skills (GRS) [10] and salpingectomy-specific form (OSA-LS) [11]. The last item of the OSA-LS form (extraction of the fallopian tube) was dropped off, as in most videos this was not recorded. Before assessing the videos, the assessors made an agreement of concordant use of the forms. Both OSATS and NRS scores were used as the primary outcome measures. The OSATS forms rated score 10–50 (6–30 from the GRS and 4–20 from the OSA-LS) and NRS 0–10 using expert level as a reference. The secondary outcome measures included operating time, blood loss, and direct complications. Data for operating time and direct complications were collected later in the medical records, while blood loss was visually graded in the recorded videos using stages 0 to 3.

Power calculations were based on OSATS scores. According to previous study [12], in laparoscopic salpingectomy, the difference in OSATS scores between novices and intermediate group was six points. Aiming to detect this difference and using type 1 error 0.05 and power of 0.80, the study required at least 18 participants and thus, we included 20 participants, 10 in each group.

The statistics were done by using SPSS 21.0 statistical software (Chicago, IL). The analyses were done by the first author (E.J.) with guidance by our statistician. The reliability of the three assessors was calculated by Intraclass correlation coefficient using both absolute and consistency type agreement. To investigate differences between the intervention and the control group, for continuous variables, we used the Independent samples T-test for parametric and Mann–Whitney U Test for non-parametric variables. For categorical variables, we used Pearson Chi-Square test.

The study plan was approved by the Helsinki University Hospital Ethics Committee (Dnro390/13/03/03/2012) and from the Hospital District of Helsinki and Uusimaa.

Results

Flowchart is shown in Fig. 1. One of the video recordings failed in the control group, and this participant was excluded from the study. One participant in the intervention group did not complete the training program due to unknown reason, but the recorded operation was included in the analysis.

Fig. 1

Flowchart of the participants

Patient, operator/participant, and surgery-related demographics are shown in Table 1. There were no differences between background data concerning operated patients. Participants in the control group had longer experience in general surgery, but not in gynecology, and there was no statistical difference in basic laparoscopies done. Otherwise there were no differences between the groups.

Table 1

Patient, operator/participant, and surgery-related demographics

 

Intervention group mean (standard deviation)

Control group mean (standard deviation)

p-value

Patient related

 Age (years)

52.8 (13.6)

53.6 (10.1)

0.893

 BMI (kg/m²)

25.1 (4.2)

24.1 (4.2)

0.608

 Previous abdominal surgery (n)

0.7 (0.5)

1.0 (1.0)

0.428

 Deliveries (n)

1.4 (1.1)

1.3 (1.1)

0.996

Operator related

 Age (years)

32.9 (3.0)

32.2 (4.0)

0.711

 Experience in Obst & Gyn (months)

17.5 (4.7)

18.1 (12.5)

0.898

 Experience in General Surgery (months)

3.0 (3.2)

5.9 (2.4)

0.034

 Basic laparoscopies done (n)

0.1 (0.4)

1.2 (1.7)

0.120

 Video playing (no/yes/ex)

6/1/1a

7/0/0b

0.364

 Instrument playing (no/yes/ex)

4/1/3a

6/1/0b

0.187

Surgery related

 Operative time (min)

14.6 (4.7)

12.6 (4.0)

0.349

 Bleeding (stage 0–3)

0.8 (1.0)

0.3 (0.7)

0.515

 Operated side (right/left)

8/2

6/3

0.628

 Complications (n)

0

0

 

Statistically significant values are highlighted in bold (p < 0.05)

aData of 8 participants

bData of 7 participants

In assessing the recorded operations, there were no statistical difference in OSATS or NRS scores between the control and intervention group (Fig. 2). The mean for total OSATS points was 16.4 (SD 5.3) in the intervention group and 16.2 (SD 5.7) in the control group, and the mean NRS was 2.9 (SD 1.6) and 3.1 (SD 1.7), respectively. In the reliability analysis, between the three assessors, the consistency type agreement was 0.787 (95% CI 0.729–0.835) and the absolute type agreement 0.787 (95% CI 0,729-0.834). In the operations recorded, there were no differences between the groups in operative time, bleeding, or complications (Table 1).

Fig. 2

OSATS points and NRS scores in the study groups. Line represents the median value, box 50% of the cases, and whiskers the whole range

The data recorded in the simulator were visually analyzed for defining the plateaus in the learning curves by using group means in each skill task and in the chosen parameters. Plateau was reached in two of the nine skill tasks; in cutting and in translocating of objects. In recorded data, by visually comparing individuals’ performances in the chosen 32 parameters, participants whose performances were two of the best and two of the weakest were clearly distinguishable. We evaluated their practice program thoroughly. When comparing the skill level in the beginning of the practice program, the best performed participants started 14.5% and 15.1% above and the weakest performed participants 0.7% and 16.5% below the whole group mean (Fig. 3). We found a tendency that the best performed participants elevated their skill levels 23.3% and 21.1%, while the skill levels were elevated 31.6% and 29.2% among the weakest performed participants (Fig. 4). At the end of the training program, the skill levels of the best performed participants were still higher (35.7% and 34.2%) than those of the weakest performed ones (32.6% and 25.4%) (Fig. 3). When comparing the performance in simulator with the assessment scores in the real operation among intervention group, the best participants in simulator were scored 2nd and 3rd, while the weakest performed were scored 9th and 10th .

Fig. 3

Skill levels in the beginning (a) and in the end (b) of the training program in each task. Results of only two of the best (B1 and B2) and two of the weakest (W1 and W2) are shown. The level is shown as difference in percent using the study group mean in the beginning of the program as a reference

Fig. 4

Increase in skill level in percent during the training program in each task. Results of only two of the best (B1 and 2) and two of the weakest (W1 and 2) are shown

Discussion

In our randomized study, we found no difference in the surgical outcome between intervention and control group. As an intervention, we used a combination of a theoretical web course and a training program with a virtual simulator.

In our intervention, the training program in a virtual simulator comprised of basic skills tasks only, and their repetition of five times each. We noted a broad distribution in starting levels and learning curves. In overall data, only in two tasks the learning curve plateau was reached, indicating that the intervention was insufficient. The starting levels and learning curves have been noticed to be highly variable between individuals [13, 14], and in different studies, the repetition rate to reach the expert level was broad [8, 15, 16]. In a study investigating the effect of training basic skills modules in a LapMentor virtual simulator, 10–22 repetitions in different tasks were required to reach expert criterions [17]. However, there is no fixed expert criterion, although the target level for training affects training length, skills level assessment, and training costs [18], but the expert criterion level is often determined by the experts in their own clinic. Programs with proficiency-based interventions improve OSATS scores in salpingectomy [12, 14] and in tubal occlusion [19], but operating performance was improved also with fixed training time in tubal occlusion [20, 21] and in salpingectomy [22], and with fixed training session numbers in cholecystectomy [23]. In a recent study, junior residents’ skills improved after repeating LapMentor basic skills modules five times in a single session [24]. Despite these different findings, proficiency-based training programs seem to be more reasonable than programs based on fixed repetitions or specific training time.

Interestingly, when evaluating skill levels in the simulator data, we noted that residents with low starting levels improved the performance more than residents with higher starting levels. However, at the end of the training sessions, their skill level did not reach the level of the residents with a higher starting point. According to our study, there seems to be a tendency between virtual simulator skill levels and operative skills, as the best performing participants in a simulator also had the best OSATS scores in operations, and vice versa. Both findings are in concordance with an earlier study according to which training in a virtual simulator is most valuable in the early stages of training and a better score in a virtual simulator is predictive of better surgical performance [25].

We included only basic skill tasks in our intervention, as our goal was to investigate preparedness for real operations, and we chose to use a virtual simulator. Some curricula contain a technical skills training program with basic skills in a low-fidelity model (FLS, ESGE), while others have a training program with virtual simulator containing either basic drills [23] or both basic drills and procedural modules [8]. Basic skills training has been proven to turn to better procedural skills in the operating room using in training both low- [14, 20] or high-fidelity [20, 23] models. In addition to technical drills or procedural modules, a laparoscopic curriculum should contain theoretical knowledge [26, 27]. We included a web course ‘Basics in Gynecological Laparoscopy’ [9] available for every resident in Finland through the internet as a cognitive part. As a consequence, the whole program became self-guided and easy to be trained. The contents of our theoretical course are relatively concordant to cognitive components in laparoscopic curricula published in England [6], North America [28], and Denmark [8]. Some curricula have the theoretical knowledge as a self-learning material on a separate CD-ROM, like Fundamentals of Laparoscopic Surgery (FLS) program [29] from the Society of American Gastrointestinal Endoscopic Surgeons, or on-line, like the Winners program from the European Society for Gynaecological Endoscopy (http://www.esge.com). Some curricula have lectures on similar subjects [8, 22], or have both self-learning and lectures [30].

One of the strengths of our study is assessing live operations, as the main object in resident training is better performance in the operating room and increased patient safety. Also recordings of the operations were all successful, assessors were blinded for the operator and the study group, and reliability between assessors was good. In a review of virtual simulation education in laparoscopy [31], from 9 published studies using a live human operation for assessment, only one was in gynecology. After this review was published, a live tubal occlusion was assessed after virtual reality training in two studies [19, 20]. In addition, a live salpingectomy was assessed after training basic skills with a box trainer [14] and after diverse intervention including salpingectomy in a porcine cadaver [22]. Thus, simulator training studies assessing live operations are still rather few and therefore, further studies are still needed.

Our study has limitations. First, recruitment of participants took more than three years. Fortunately, recruitment bias was avoided, as every suitable resident agreed to participate. During that time, there were no changes in traditional teaching. Nine of ten participants in the intervention group completed the rehearsal program with the simulator as instructed. All training sessions were documented and assessed. Second, the sample size is small, and thus, chance could have had an impact on the study outcome. In the control group, there were three participants whose performance without training reached the best performances in the intervention group. As expected, the basic psychomotorical skills differs between individuals. Since the study group was rather small, these three talented individuals may have affected the result to the extent that possible differences between the study groups could not be detected.

To conclude, with our simple systematic training program, we were unable to show a significant effect on residents’ first laparoscopic operation. During training sessions, the plateau in the learning curve was not reached in every task, thus a proficiency-based curriculum could be more effective. It appears that residents with lower baseline laparoscopic skills benefit most from training with a virtual simulator.

Notes

Funding

The study was supported by University of Helsinki and special governmental grant for health sciences research.

Compliance with ethical standards

Disclosures

Drs. Ewa Jokinen, Tomi S. Mikkola, and Päivi Härkki have no conflicts of interest or financial ties to disclose.

References

  1. 1.
    Elbadrawy M, Majoko F, Gasson J (2008) Impact of Calman system and recent reforms on surgical training in gynaecology. J Obstet Gynaecol 28:474–477CrossRefGoogle Scholar
  2. 2.
    Hurskainen R, Teperi J, Rissanen P, Aalto AM, Grenman S, Kivela A, Kujansuu E, Vuorma S, Yliskoski M, Paavonen J (2001) Quality of life and cost-effectiveness of levonorgestrel-releasing intrauterine system versus hysterectomy for treatment of menorrhagia: a randomised trial. Lancet 357:273–277CrossRefGoogle Scholar
  3. 3.
    Donnez J, Donnez O, Dolmans MM (2014) With the advent of selective progesterone receptor modulators, what is the place of myoma surgery in current practice? Fertil Steril 102:640–648CrossRefGoogle Scholar
  4. 4.
    Salazar CA, Isaacson K (2017) Office operative hysteroscopy: an Update. J Minim Invasive Gynecol 25:199–208Google Scholar
  5. 5.
    Vitish-Sharma P, Knowles J, Patel B (2011) Acquisition of fundamental laparoscopic skills: is a box really as good as a virtual reality trainer? Int J Surg 9:659–661CrossRefGoogle Scholar
  6. 6.
    Burden C, Fox R, Lenguerrand E, Hinshaw K, Draycott TJ, James M (2014) Curriculum development for basic gynaecological laparoscopy with comparison of expert trainee opinions; prospective cross-sectional observational study. Eur J Obstet Gynecol Reprod Biol 180:1–7CrossRefGoogle Scholar
  7. 7.
    Grantcharov TP, Reznick RK (2008) Teaching procedural skills. BMJ 336:1129–1131CrossRefGoogle Scholar
  8. 8.
    Strandbygaard J, Bjerrum F, Maagaard M, Rifbjerg Larsen C, Ottesen B, Sorensen JL (2014) A structured four-step curriculum in basic laparoscopy: development and validation. Acta Obstet Gynecol Scand 93:359–366CrossRefGoogle Scholar
  9. 9.
    Jokinen E, Mikkola TS, Harkki P (2017) Evaluation of a web course on the basics of gynecological laparoscopy in resident training. J Surg Educ 74:717–723CrossRefGoogle Scholar
  10. 10.
    Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278CrossRefGoogle Scholar
  11. 11.
    Larsen CR, Grantcharov T, Schouenborg L, Ottosen C, Soerensen JL, Ottesen B (2008) Objective assessment of surgical competence in gynaecological laparoscopy: development and validation of a procedure-specific rating scale. BJOG 115:908–916CrossRefGoogle Scholar
  12. 12.
    Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802CrossRefGoogle Scholar
  13. 13.
    McDougall EM, Corica FA, Boker JR, Sala LG, Stoliar G, Borin JF, Chu FT, Clayman RV (2006) Construct validity testing of a laparoscopic surgical simulator. J Am Coll Surg 202:779–787CrossRefGoogle Scholar
  14. 14.
    Gala R, Orejuela F, Gerten K, Lockrow E, Kilpatrick C, Chohan L, Green C, Vaught J, Goldberg A, Schaffer J (2013) Effect of validated skills simulation on operating room performance in obstetrics and gynecology residents: a randomized controlled trial. Obstet Gynecol 121:578–584CrossRefGoogle Scholar
  15. 15.
    Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A (2007) Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 246:771–779CrossRefGoogle Scholar
  16. 16.
    Bjerrum F, Sorensen JL, Thinggaard J, Strandbygaard J, Konge L (2015) Implementation of a cross-specialty training program in basic laparoscopy. JSLS 19Google Scholar
  17. 17.
    Andreatta PB, Woodrum DT, Birkmeyer JD, Yellamanchilli RK, Doherty GM, Gauger PG, Minter RM (2006) Laparoscopic skills are improved with LapMentor training: results of a randomized, double-blinded study. Ann Surg 243:854–860; discussion 860–853CrossRefGoogle Scholar
  18. 18.
    Luursema JM, Rovers MM, Alken A, Kengen B, van Goor H (2015) When experts are oceans apart: comparing expert performance values for proficiency-based laparoscopic simulator training. J Surg Educ 72:536–541CrossRefGoogle Scholar
  19. 19.
    Ahlborg L, Hedman L, Nisell H, Fellander-Tsai L, Enochsson L (2013) Simulator training and non-technical factors improve laparoscopic performance among OBGYN trainees. Acta Obstet Gynecol Scand 92:1194–1201CrossRefGoogle Scholar
  20. 20.
    Akdemir A, Sendag F, Oztekin MK (2014) Laparoscopic virtual reality simulator and box trainer in gynecology. Int J Gynaecol Obstet 125:181–185CrossRefGoogle Scholar
  21. 21.
    Banks EH, Chudnoff S, Karmin I, Wang C, Pardanani S (2007) Does a surgical simulator improve resident operative performance of laparoscopic tubal ligation? Am J Obstet Gynecol 197:541–e1CrossRefGoogle Scholar
  22. 22.
    Patel NR, Makai GE, Sloan NL, Della Badia CR (2016) Traditional versus simulation resident surgical laparoscopic salpingectomy training: a randomized controlled trial. J Minim Invasive Gynecol 23:372–377CrossRefGoogle Scholar
  23. 23.
    Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150CrossRefGoogle Scholar
  24. 24.
    Paquette J, Lemyre M, Vachon-Marceau C, Bujold E, Maheux-Lacroix S (2017) Virtual laparoscopy simulation: a promising pedagogic tool in gynecology. JSLS 21Google Scholar
  25. 25.
    Hart R, Doherty DA, Karthigasu K, Garry R (2006) The value of virtual reality-simulator training in the development of laparoscopic surgical skills. J Minim Invasive Gynecol 13:126–133CrossRefGoogle Scholar
  26. 26.
    Fried GM (2006) Lessons from the surgical experience with simulators: incorporation into training and utilization in determining competency. Gastrointest Endosc Clin N Am 16:425–434CrossRefGoogle Scholar
  27. 27.
    Stefanidis D, Heniford BT (2009) The formula for a successful laparoscopic skills curriculum. Arch Surg 144:77–82; discussion 82CrossRefGoogle Scholar
  28. 28.
    Shore EM, Lefebvre GG, Husslein H, Bjerrum F, Sorensen JL, Grantcharov TP (2015) Designing a standardized laparoscopy curriculum for gynecology residents: a delphi approach. J Grad Med Educ 7:197–202CrossRefGoogle Scholar
  29. 29.
    Ritter EM, Scott DJ (2007) Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery. Surg Innov 14:107–112CrossRefGoogle Scholar
  30. 30.
    Shore EM, Grantcharov TP, Husslein H, Shirreff L, Dedy NJ, McDermott CD, Lefebvre GG (2016) Validating a standardized laparoscopy curriculum for gynecology residents: a randomized controlled trial. Am J Obstet Gynecol 215:204–e1CrossRefGoogle Scholar
  31. 31.
    Larsen CR, Oestergaard J, Ottesen BS, Soerensen JL (2012) The efficacy of virtual reality simulation training in laparoscopy: a systematic review of randomized trials. Acta Obstet Gynecol Scand 91:1015–1028CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

OpenAccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Obstetrics and GynecologyUniversity of Helsinki and Helsinki University HospitalHelsinkiFinland
  2. 2.Helsinki University HospitalHelsinkiFinland

Personalised recommendations