Surgical Endoscopy

, Volume 29, Issue 11, pp 3392–3403

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

  • Fernando Pérez Escamirosa
  • Ricardo Manuel Ordorica Flores
  • Ignacio Oropesa García
  • Cristian Rubén Zalles Vidal
  • Arturo Minor Martínez
New Technology

Abstract

Background

The aim of this study is to present face, content, and constructs validity of the endoscopic orthogonal video system (EndoViS) training system and determines its efficiency as a training and objective assessment tool of the surgeons’ psychomotor skills.

Methods

Thirty-five surgeons and medical students participated in this study: 11 medical students, 19 residents, and 5 experts. All participants performed four basic skill tasks using conventional laparoscopic instruments and EndoViS training system. Subsequently, participants filled out a questionnaire regarding the design, realism, overall functionality, and its capabilities to train hand–eye coordination and depth perception, rated on a 5-point Likert scale. Motion data of the instruments were obtained by means of two webcams built into a laparoscopic physical trainer. To identify the surgical instruments in the images, colored markers were placed in each instrument. Thirteen motion-related metrics were used to assess laparoscopic performance of the participants. Statistical analysis of performance was made between novice, intermediate, and expert groups. Internal consistency of all metrics was analyzed with Cronbach’s α test.

Results

Overall scores about features of the EndoViS system were positives. Participants agreed with the usefulness of tasks and the training capacities of EndoViS system (score >4). Results presented significant differences in the execution of three skill tasks performed by participants. Seven metrics showed construct validity for assessment of performance with high consistency levels.

Conclusions

EndoViS training system has been successfully validated. Results showed that EndoViS was able to differentiate between participants of varying laparoscopic experience. This simulator is a useful and effective tool to objectively assess laparoscopic psychomotor skills of the surgeons.

Keywords

Laparoscopic surgery Surgical training Objective assessment Motion metrics Validation Endoscopic orthogonal video system (EndoViS

References

  1. 1.
    Aziz O, Constantinides V, Tekkis PP, Athanasiou T, Purkayastha S, Paraskeva P, Darzi AW, Heriot AG (2006) Laparoscopic versus open surgery for rectal cancer: a meta-analysis. Ann Surg Oncol 13(3):413–424CrossRefPubMedGoogle Scholar
  2. 2.
    Staudacher C, Vignali A (2010) Laparoscopic surgery for rectal cancer: the state of the art. World J Gastrointest Surg 2(9):275–282. doi:10.4240/wjgs.v2.i9.275 PubMedCentralCrossRefPubMedGoogle Scholar
  3. 3.
    Klarenbeek BR, Bergamaschi R, Veenhof AA, van der Peet DL, van den Broek WT, de Lange ES, Bemelman WA, Heres P, Lacy AM, Cuesta MA (2011) Laparoscopic versus open sigmoid resection for diverticular disease: follow-up assessment of the randomized control Sigma trial. Surg Endosc 25(4):1121–1126. doi:10.1007/s00464-010-1327-0 CrossRefPubMedGoogle Scholar
  4. 4.
    Figert PL, Park AE, Witzke DB, Schwartz RW (2001) Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 193(5):533–537CrossRefPubMedGoogle Scholar
  5. 5.
    Livingston EH, Rege RV (2004) A nationwide study of conversion from laparoscopic to open cholecystectomy. Am J Surg 188(3):205–211CrossRefPubMedGoogle Scholar
  6. 6.
    Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165(3):358–361CrossRefPubMedGoogle Scholar
  7. 7.
    Bridges M, Diamond DL (1999) The financial impact of teaching surgical residents in the operating room. Am J Surg 177(1):28–32CrossRefPubMedGoogle Scholar
  8. 8.
    Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240(3):518–525PubMedCentralCrossRefPubMedGoogle Scholar
  9. 9.
    Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA (2007) Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Stud Health Technol Inform 125:76–81PubMedGoogle Scholar
  10. 10.
    Samia H, Khan S, Lawrence J, Delaney CP (2013) Simulation and its role in training. Clin Colon Rectal Surg 26(1):47–55. doi:10.1055/s-0033-1333661 PubMedCentralCrossRefPubMedGoogle Scholar
  11. 11.
    Khan MW, Lin D, Marlow N, Altree M, Babidge W, Field J, Hewett P, Maddern G (2014) Laparoscopic skills maintenance: a randomized trial of virtual reality and box trainer simulators. J Surg Educ 71(1):79–84. doi:10.1016/j.jsurg.2013.05.009 CrossRefPubMedGoogle Scholar
  12. 12.
    Laski D, Stefaniak TJ, Makarewicz W, Proczko M, Gruca Z, Sledziński Z (2011) Structuralized box-trainer laparoscopic training significantly improves performance in complex virtual reality laparoscopic tasks. Wideochir Inne Tech Malo Inwazyjne 7(1):27–32. doi:10.5114/wiitm.2011.25666 PubMedCentralPubMedGoogle Scholar
  13. 13.
    Martinez AM, Espinoza DL (2007) Novel laparoscopic home trainer. Surg Laparosc Endosc Percutan Tech 17(4):300–302CrossRefPubMedGoogle Scholar
  14. 14.
    Martinez AM, Kalach AC, Espinoza DL (2008) Millimetric laparoscopic surgery training on a physical trainer using rats. Surg Endosc 22:246–249CrossRefPubMedGoogle Scholar
  15. 15.
    Hinata N, Iwamoto H, Morizane S, Hikita K, Yao A, Muraoka K, Honda M, Isoyama T, Sejima T, Takenaka A (2013) Dry box training with three-dimensional vision for the assistant surgeon in robot-assisted urological surgery. Int J Urol 20(10):1037–1041. doi:10.1111/iju.12101 PubMedGoogle Scholar
  16. 16.
    Verdaasdonk EG, Stassen LP, Monteny LJ, Dankelman J (2006) Validation of a new basic virtual reality simulator for training of basic endoscopic skills: the SIMENDO. Surg Endosc 20(3):511–518CrossRefPubMedGoogle Scholar
  17. 17.
    Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191(1):28–32CrossRefPubMedGoogle Scholar
  18. 18.
    Ayodeji ID, Schijven M, Jakimowicz J, Greve JW (2007) Face validation of the Simbionix LAP Mentor virtual reality training module and its applicability in the surgical curriculum. Surg Endosc 21(9):1641–1649CrossRefPubMedGoogle Scholar
  19. 19.
    Iwata N, Fujiwara M, Kodera Y, Tanaka C, Ohashi N, Nakayama G, Koike M, Nakao A (2011) Construct validity of the LapVR virtual-reality surgical simulator. Surg Endosc 25(2):423–428. doi:10.1007/s00464-010-1184-x CrossRefPubMedGoogle Scholar
  20. 20.
    Stylopoulos N, Cotin S, Maithel SK, Ottensmeye M, Jackson PG, Bardsley RS, Neumann PF, Rattner DW, Dawson SL (2004) Computer-enhanced laparoscopic training system (CELTS): bridging the gap. Surg Endosc 18(5):782–789CrossRefPubMedGoogle Scholar
  21. 21.
    Soyinka AS, Schollmeyer T, Meinhold-Heerlein I, Gopalghare DV, Hasson H, Mettler L (2008) Enhancing laparoscopic performance with the LTS3E: a computerized hybrid physical reality simulator. Fertil Steril 90(5):1988–1994. doi:10.1016/j.fertnstert.2007.08.077 CrossRefPubMedGoogle Scholar
  22. 22.
    Pellen MG, Horgan LF, Barton JR, Attwood SE (2009) Construct validity of the ProMIS laparoscopic simulator. Surg Endosc 23(1):130–139. doi:10.1007/s00464-008-0066-y CrossRefPubMedGoogle Scholar
  23. 23.
    Botden SM, Jakimowicz JJ (2009) What is going on in augmented reality simulation in laparoscopic surgery? Surg Endosc 23(8):1693–1700. doi:10.1007/s00464-008-0144-1 PubMedCentralCrossRefPubMedGoogle Scholar
  24. 24.
    Chmarra MK, Grimbergen CA, Dankelman J (2007) Systems for tracking minimally invasive surgical instruments. Minim Invasive Ther Allied Technol 16:328–340CrossRefPubMedGoogle Scholar
  25. 25.
    Oropesa I, Sánchez-González P, Chmarra MK, Lamata P, Fernández A, Sánchez-Margallo JA, Jansen FW, Dankelman J, Sánchez-Margallo FM, Gómez EJ (2013) EVA: laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment. Surg Endosc 27(3):1029–1039. doi:10.1007/s00464-012-2513-z CrossRefPubMedGoogle Scholar
  26. 26.
    Van Empel PJ, Commandeur JP, van Rijssen LB, Verdam MG, Huirne JA, Scheele F, Bonjer HJ, Jeroen Meijerink W (2013) Learning curve on the TrEndo laparoscopic simulator compared to an expert level. Surg Endosc 27(8):2934–2939. doi:10.1007/s00464-013-2859-x CrossRefPubMedGoogle Scholar
  27. 27.
    Kovac E, Azhar RA, Quirouet A, Delisle J, Anidjar M (2012) Construct validity of the LapSim virtual reality laparoscopic simulator within a urology residency program. Can Urol Assoc J 6(4):253–259. doi:10.5489/cuaj.12047 PubMedCentralCrossRefPubMedGoogle Scholar
  28. 28.
    Ritter EM, Kindelan TW, Michael C, Pimentel EA, Bowyer MW (2007) Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS). Surg Endosc 21(8):1441–1445CrossRefPubMedGoogle Scholar
  29. 29.
    Maithel S, Sierra R, Korndorffer J, Neumann P, Dawson S, Callery M, Jones D, Scott D (2006) Construct and face validity of MIST-VR, Endotower, and CELTS: are we ready for skills assessment using simulators? Surg Endosc 20(1):104–112CrossRefPubMedGoogle Scholar
  30. 30.
    Pérez F, Sossa H, Martínez R, Lorias D, Minor A (2013) Video-based tracking of laparoscopic instruments using an orthogonal webcams system. World Acad Sci Eng Technol Int J Med Health Pharm Biomed Eng 7(8):184–187Google Scholar
  31. 31.
    Cotin S, Stylopoulos N, Ottensmeyer MP, Neumann P, Rattner DW, Dawson SL (2002) Metrics for laparoscopic skills trainers: the weakest link! In: MICCAI 2002. LNCS, 2488, pp. 35–43. Springer, HeidelbergGoogle Scholar
  32. 32.
    Chmarra MK, Kolkman W, Jansen FW, Grimbergen CA, Dankelman J (2007) The influence of experience and camera holding on laparoscopic instrument movements with the TrEndo tracking system. Surg Endosc 21:2069–2075CrossRefPubMedGoogle Scholar
  33. 33.
    Hofstad EF, Våpenstad C, Chmarra MK, Langø T, Kuhry E, Mårvik R (2013) A study of psychomotor skills in minimally invasive surgery: what differentiates expert and nonexpert performance. Surg Endosc 27(3):854–863. doi:10.1007/s00464-012-2524-9 CrossRefPubMedGoogle Scholar
  34. 34.
    Oropesa I, Chmarra MK, Sánchez-González P, Lamata P, Rodrigues SP, Enciso S, Sánchez-Margallo FM, Jansen FW, Dankelman J, Gómez EJ (2013) Relevance of motion-related assessment metrics in laparoscopic surgery. Surg Innov 20(3):299–312. doi:10.1177/1553350612459808 CrossRefPubMedGoogle Scholar
  35. 35.
    Vassiliou MC, Ghitulescu GA, Feldman LS, Stanbridge D, Leffondré K, Sigman HH, Fried GM (2006) The MISTELS program to measure technical skill in laparoscopic surgery: evidence for reliability. Surg Endosc 20(5):744–747CrossRefPubMedGoogle Scholar
  36. 36.
    Rivard JD, Vergis AS, Unger BJ, Hardy KM, Andrew CG, Gillman LM, Park J (2014) Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator. Surg Endosc 28(6):1921–1928. doi:10.1007/s00464-013-3414-5 CrossRefPubMedGoogle Scholar
  37. 37.
    Arikatla VS, Sankaranarayanan G, Ahn W, Chellali A, De S, Caroline GL, Hwabejire J, DeMoya M, Schwaitzberg S, Jones DB (2013) Face and construct validation of a virtual peg transfer simulator. Surg Endosc 27(5):1721–1729. doi:10.1007/s00464-012-2664-y PubMedCentralCrossRefPubMedGoogle Scholar
  38. 38.
    Chellali A, Zhang L, Sankaranarayanan G, Arikatla VS, Ahn W, Derevianko A, Schwaitzberg SD, Jones DB, Demoya M, Cao CG (2014) Validation of the VBLaST peg transfer task: a first step toward an alternate training standard. Surg Endosc. doi:10.1007/s00464-014-3538-2 Google Scholar
  39. 39.
    Chmarra MK, de Klein S, Winter JC, Jansen FW, Dankelman J (2011) Objective classification of residents based on their psychomotor laparoscopic skills. Surg Endosc 24(5):1031–1039. doi:10.1007/s00464-009-0721-y CrossRefGoogle Scholar
  40. 40.
    Van Sickle KR, McClusky DA 3rd, Gallagher AG, Smith CD (2005) Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc 19(9):1227–1231CrossRefPubMedGoogle Scholar
  41. 41.
    Yamaguchi S, Yoshida D, Kenmotsu H, Yasunaga T, Konishi K, Ieiri S, Nakashima H, Tanoue K, Hashizume M (2011) Objective assessment of laparoscopic suturing skills using a motion-tracking system. Surg Endosc 25(3):771–775. doi:10.1007/s00464-010-1251-3 CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Fernando Pérez Escamirosa
    • 1
  • Ricardo Manuel Ordorica Flores
    • 2
  • Ignacio Oropesa García
    • 3
  • Cristian Rubén Zalles Vidal
    • 2
  • Arturo Minor Martínez
    • 1
  1. 1.Bioelectronics Section, Department of Electrical EngineeringCenter for Research and Advanced Studies of the National Polytechnic Institute of Mexico (CINVESTAV - IPN)Mexico CityMexico
  2. 2.Department of Pediatric SurgeryHospital Infantil de Mexico Federico GómezMexico CityMexico
  3. 3.Bioengineering and Telemedicine Centre (GBT), ETSI TelecommunicationUniversidad Politécnica de Madrid (UPM)MadridSpain

Personalised recommendations