Skip to main content
SpringerLink
Log in

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

  • New Technology
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The aim of this study is to present face, content, and constructs validity of the endoscopic orthogonal video system (EndoViS) training system and determines its efficiency as a training and objective assessment tool of the surgeons’ psychomotor skills.

Methods

Thirty-five surgeons and medical students participated in this study: 11 medical students, 19 residents, and 5 experts. All participants performed four basic skill tasks using conventional laparoscopic instruments and EndoViS training system. Subsequently, participants filled out a questionnaire regarding the design, realism, overall functionality, and its capabilities to train hand–eye coordination and depth perception, rated on a 5-point Likert scale. Motion data of the instruments were obtained by means of two webcams built into a laparoscopic physical trainer. To identify the surgical instruments in the images, colored markers were placed in each instrument. Thirteen motion-related metrics were used to assess laparoscopic performance of the participants. Statistical analysis of performance was made between novice, intermediate, and expert groups. Internal consistency of all metrics was analyzed with Cronbach’s α test.

Results

Overall scores about features of the EndoViS system were positives. Participants agreed with the usefulness of tasks and the training capacities of EndoViS system (score >4). Results presented significant differences in the execution of three skill tasks performed by participants. Seven metrics showed construct validity for assessment of performance with high consistency levels.

Conclusions

EndoViS training system has been successfully validated. Results showed that EndoViS was able to differentiate between participants of varying laparoscopic experience. This simulator is a useful and effective tool to objectively assess laparoscopic psychomotor skills of the surgeons.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Aziz O, Constantinides V, Tekkis PP, Athanasiou T, Purkayastha S, Paraskeva P, Darzi AW, Heriot AG (2006) Laparoscopic versus open surgery for rectal cancer: a meta-analysis. Ann Surg Oncol 13(3):413–424

    Article  PubMed  Google Scholar 

  2. Staudacher C, Vignali A (2010) Laparoscopic surgery for rectal cancer: the state of the art. World J Gastrointest Surg 2(9):275–282. doi:10.4240/wjgs.v2.i9.275

    Article  PubMed Central  PubMed  Google Scholar 

  3. Klarenbeek BR, Bergamaschi R, Veenhof AA, van der Peet DL, van den Broek WT, de Lange ES, Bemelman WA, Heres P, Lacy AM, Cuesta MA (2011) Laparoscopic versus open sigmoid resection for diverticular disease: follow-up assessment of the randomized control Sigma trial. Surg Endosc 25(4):1121–1126. doi:10.1007/s00464-010-1327-0

    Article  PubMed  Google Scholar 

  4. Figert PL, Park AE, Witzke DB, Schwartz RW (2001) Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 193(5):533–537

    Article  CAS  PubMed  Google Scholar 

  5. Livingston EH, Rege RV (2004) A nationwide study of conversion from laparoscopic to open cholecystectomy. Am J Surg 188(3):205–211

    Article  PubMed  Google Scholar 

  6. Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165(3):358–361

    Article  CAS  PubMed  Google Scholar 

  7. Bridges M, Diamond DL (1999) The financial impact of teaching surgical residents in the operating room. Am J Surg 177(1):28–32

    Article  CAS  PubMed  Google Scholar 

  8. Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240(3):518–525

    Article  PubMed Central  PubMed  Google Scholar 

  9. Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA (2007) Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Stud Health Technol Inform 125:76–81

    CAS  PubMed  Google Scholar 

  10. Samia H, Khan S, Lawrence J, Delaney CP (2013) Simulation and its role in training. Clin Colon Rectal Surg 26(1):47–55. doi:10.1055/s-0033-1333661

    Article  PubMed Central  PubMed  Google Scholar 

  11. Khan MW, Lin D, Marlow N, Altree M, Babidge W, Field J, Hewett P, Maddern G (2014) Laparoscopic skills maintenance: a randomized trial of virtual reality and box trainer simulators. J Surg Educ 71(1):79–84. doi:10.1016/j.jsurg.2013.05.009

    Article  PubMed  Google Scholar 

  12. Laski D, Stefaniak TJ, Makarewicz W, Proczko M, Gruca Z, Sledziński Z (2011) Structuralized box-trainer laparoscopic training significantly improves performance in complex virtual reality laparoscopic tasks. Wideochir Inne Tech Malo Inwazyjne 7(1):27–32. doi:10.5114/wiitm.2011.25666

    PubMed Central  PubMed  Google Scholar 

  13. Martinez AM, Espinoza DL (2007) Novel laparoscopic home trainer. Surg Laparosc Endosc Percutan Tech 17(4):300–302

    Article  PubMed  Google Scholar 

  14. Martinez AM, Kalach AC, Espinoza DL (2008) Millimetric laparoscopic surgery training on a physical trainer using rats. Surg Endosc 22:246–249

    Article  PubMed  Google Scholar 

  15. Hinata N, Iwamoto H, Morizane S, Hikita K, Yao A, Muraoka K, Honda M, Isoyama T, Sejima T, Takenaka A (2013) Dry box training with three-dimensional vision for the assistant surgeon in robot-assisted urological surgery. Int J Urol 20(10):1037–1041. doi:10.1111/iju.12101

    PubMed  Google Scholar 

  16. Verdaasdonk EG, Stassen LP, Monteny LJ, Dankelman J (2006) Validation of a new basic virtual reality simulator for training of basic endoscopic skills: the SIMENDO. Surg Endosc 20(3):511–518

    Article  CAS  PubMed  Google Scholar 

  17. Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191(1):28–32

    Article  PubMed  Google Scholar 

  18. Ayodeji ID, Schijven M, Jakimowicz J, Greve JW (2007) Face validation of the Simbionix LAP Mentor virtual reality training module and its applicability in the surgical curriculum. Surg Endosc 21(9):1641–1649

    Article  CAS  PubMed  Google Scholar 

  19. Iwata N, Fujiwara M, Kodera Y, Tanaka C, Ohashi N, Nakayama G, Koike M, Nakao A (2011) Construct validity of the LapVR virtual-reality surgical simulator. Surg Endosc 25(2):423–428. doi:10.1007/s00464-010-1184-x

    Article  PubMed  Google Scholar 

  20. Stylopoulos N, Cotin S, Maithel SK, Ottensmeye M, Jackson PG, Bardsley RS, Neumann PF, Rattner DW, Dawson SL (2004) Computer-enhanced laparoscopic training system (CELTS): bridging the gap. Surg Endosc 18(5):782–789

    Article  CAS  PubMed  Google Scholar 

  21. Soyinka AS, Schollmeyer T, Meinhold-Heerlein I, Gopalghare DV, Hasson H, Mettler L (2008) Enhancing laparoscopic performance with the LTS3E: a computerized hybrid physical reality simulator. Fertil Steril 90(5):1988–1994. doi:10.1016/j.fertnstert.2007.08.077

    Article  PubMed  Google Scholar 

  22. Pellen MG, Horgan LF, Barton JR, Attwood SE (2009) Construct validity of the ProMIS laparoscopic simulator. Surg Endosc 23(1):130–139. doi:10.1007/s00464-008-0066-y

    Article  PubMed  Google Scholar 

  23. Botden SM, Jakimowicz JJ (2009) What is going on in augmented reality simulation in laparoscopic surgery? Surg Endosc 23(8):1693–1700. doi:10.1007/s00464-008-0144-1

    Article  PubMed Central  PubMed  Google Scholar 

  24. Chmarra MK, Grimbergen CA, Dankelman J (2007) Systems for tracking minimally invasive surgical instruments. Minim Invasive Ther Allied Technol 16:328–340

    Article  CAS  PubMed  Google Scholar 

  25. Oropesa I, Sánchez-González P, Chmarra MK, Lamata P, Fernández A, Sánchez-Margallo JA, Jansen FW, Dankelman J, Sánchez-Margallo FM, Gómez EJ (2013) EVA: laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment. Surg Endosc 27(3):1029–1039. doi:10.1007/s00464-012-2513-z

    Article  PubMed  Google Scholar 

  26. Van Empel PJ, Commandeur JP, van Rijssen LB, Verdam MG, Huirne JA, Scheele F, Bonjer HJ, Jeroen Meijerink W (2013) Learning curve on the TrEndo laparoscopic simulator compared to an expert level. Surg Endosc 27(8):2934–2939. doi:10.1007/s00464-013-2859-x

    Article  PubMed  Google Scholar 

  27. Kovac E, Azhar RA, Quirouet A, Delisle J, Anidjar M (2012) Construct validity of the LapSim virtual reality laparoscopic simulator within a urology residency program. Can Urol Assoc J 6(4):253–259. doi:10.5489/cuaj.12047

    Article  PubMed Central  PubMed  Google Scholar 

  28. Ritter EM, Kindelan TW, Michael C, Pimentel EA, Bowyer MW (2007) Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS). Surg Endosc 21(8):1441–1445

    Article  CAS  PubMed  Google Scholar 

  29. Maithel S, Sierra R, Korndorffer J, Neumann P, Dawson S, Callery M, Jones D, Scott D (2006) Construct and face validity of MIST-VR, Endotower, and CELTS: are we ready for skills assessment using simulators? Surg Endosc 20(1):104–112

    Article  CAS  PubMed  Google Scholar 

  30. Pérez F, Sossa H, Martínez R, Lorias D, Minor A (2013) Video-based tracking of laparoscopic instruments using an orthogonal webcams system. World Acad Sci Eng Technol Int J Med Health Pharm Biomed Eng 7(8):184–187

    Google Scholar 

  31. Cotin S, Stylopoulos N, Ottensmeyer MP, Neumann P, Rattner DW, Dawson SL (2002) Metrics for laparoscopic skills trainers: the weakest link! In: MICCAI 2002. LNCS, 2488, pp. 35–43. Springer, Heidelberg

  32. Chmarra MK, Kolkman W, Jansen FW, Grimbergen CA, Dankelman J (2007) The influence of experience and camera holding on laparoscopic instrument movements with the TrEndo tracking system. Surg Endosc 21:2069–2075

    Article  CAS  PubMed  Google Scholar 

  33. Hofstad EF, Våpenstad C, Chmarra MK, Langø T, Kuhry E, Mårvik R (2013) A study of psychomotor skills in minimally invasive surgery: what differentiates expert and nonexpert performance. Surg Endosc 27(3):854–863. doi:10.1007/s00464-012-2524-9

    Article  PubMed  Google Scholar 

  34. Oropesa I, Chmarra MK, Sánchez-González P, Lamata P, Rodrigues SP, Enciso S, Sánchez-Margallo FM, Jansen FW, Dankelman J, Gómez EJ (2013) Relevance of motion-related assessment metrics in laparoscopic surgery. Surg Innov 20(3):299–312. doi:10.1177/1553350612459808

    Article  PubMed  Google Scholar 

  35. Vassiliou MC, Ghitulescu GA, Feldman LS, Stanbridge D, Leffondré K, Sigman HH, Fried GM (2006) The MISTELS program to measure technical skill in laparoscopic surgery: evidence for reliability. Surg Endosc 20(5):744–747

    Article  CAS  PubMed  Google Scholar 

  36. Rivard JD, Vergis AS, Unger BJ, Hardy KM, Andrew CG, Gillman LM, Park J (2014) Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator. Surg Endosc 28(6):1921–1928. doi:10.1007/s00464-013-3414-5

    Article  PubMed  Google Scholar 

  37. Arikatla VS, Sankaranarayanan G, Ahn W, Chellali A, De S, Caroline GL, Hwabejire J, DeMoya M, Schwaitzberg S, Jones DB (2013) Face and construct validation of a virtual peg transfer simulator. Surg Endosc 27(5):1721–1729. doi:10.1007/s00464-012-2664-y

    Article  PubMed Central  PubMed  Google Scholar 

  38. Chellali A, Zhang L, Sankaranarayanan G, Arikatla VS, Ahn W, Derevianko A, Schwaitzberg SD, Jones DB, Demoya M, Cao CG (2014) Validation of the VBLaST peg transfer task: a first step toward an alternate training standard. Surg Endosc. doi:10.1007/s00464-014-3538-2

    Google Scholar 

  39. Chmarra MK, de Klein S, Winter JC, Jansen FW, Dankelman J (2011) Objective classification of residents based on their psychomotor laparoscopic skills. Surg Endosc 24(5):1031–1039. doi:10.1007/s00464-009-0721-y

    Article  Google Scholar 

  40. Van Sickle KR, McClusky DA 3rd, Gallagher AG, Smith CD (2005) Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc 19(9):1227–1231

    Article  PubMed  Google Scholar 

  41. Yamaguchi S, Yoshida D, Kenmotsu H, Yasunaga T, Konishi K, Ieiri S, Nakashima H, Tanoue K, Hashizume M (2011) Objective assessment of laparoscopic suturing skills using a motion-tracking system. Surg Endosc 25(3):771–775. doi:10.1007/s00464-010-1251-3

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors want to thank all medical students, residents, and surgeons for their enthusiastic and kindly participation in all trials and staff at Hospital Infantil de Mexico for the facilities to carry out this validation study.

Disclosures

Fernando Pérez, Ricardo Ordorica, Ignacio Oropesa, Cristian Zalles, and Dr. Arturo Minor have no conflicts of interest of financial ties to disclose.

Author information

Authors and Affiliations

  1. Bioelectronics Section, Department of Electrical Engineering, Center for Research and Advanced Studies of the National Polytechnic Institute of Mexico (CINVESTAV - IPN), Av. Instituto Politécnico Nacional No. 2508, Col. San Pedro Zacatenco, 07360, Mexico City, Mexico

    Fernando Pérez Escamirosa & Arturo Minor Martínez

  2. Department of Pediatric Surgery, Hospital Infantil de Mexico Federico Gómez, Mexico City, Mexico

    Ricardo Manuel Ordorica Flores & Cristian Rubén Zalles Vidal

  3. Bioengineering and Telemedicine Centre (GBT), ETSI Telecommunication, Universidad Politécnica de Madrid (UPM), Madrid, Spain

    Ignacio Oropesa García

Authors
  1. Fernando Pérez Escamirosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ricardo Manuel Ordorica Flores
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ignacio Oropesa García
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cristian Rubén Zalles Vidal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arturo Minor Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Pérez Escamirosa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Escamirosa, F.P., Flores, R.M.O., García, I.O. et al. Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons. Surg Endosc 29, 3392–3403 (2015). https://doi.org/10.1007/s00464-014-4032-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-014-4032-6

Keywords

Search

Navigation