Objective classification of psychomotor laparoscopic skills of surgeons based on three different approaches
- 33 Downloads
The determination of surgeons’ psychomotor skills in minimally invasive surgery techniques is one of the major concerns of the programs of surgical training in several hospitals. Therefore, it is important to assess and classify objectively the level of experience of surgeons and residents during their training process. The aim of this study was to investigate three classification methods for establishing automatically the level of surgical competence of the surgeons based on their psychomotor laparoscopic skills.
A total of 43 participants, divided into an experienced surgeons group with ten experts (> 100 laparoscopic procedures performed) and non-experienced surgeons group with 24 residents and nine medical students (< 10 laparoscopic procedures performed), performed three tasks in the EndoViS training system. Motion data of the instruments were captured with a video-tracking system built into the EndoViS simulator and analyzed using 13 motion analysis parameters (MAPs). Radial basis function networks (RBFNets), K-star (K*), and random forest (RF) were used for classifying surgeons based on the MAPs’ scores of all participants. The performance of the three classifiers was examined using hold-out and leave-one-out validation techniques.
For all three tasks, the K-star method was superior in terms of accuracy and AUC in both validation techniques. The mean accuracy of the classifiers was 93.33% for K-star, 87.58% for RBFNets, and 84.85% for RF in hold-out validation, and 91.47% for K-star, 89.92% for RBFNets, and 83.72% for RF in leave-one-out cross-validation.
The three proposed methods demonstrated high performance in the classification of laparoscopic surgeons, according to their level of psychomotor skills. Together with motion analysis and three laparoscopic tasks of the Fundamental Laparoscopic Surgery Program, these classifiers provide a means for objectively classifying surgical competence of the surgeons for existing laparoscopic box trainers.
KeywordsLaparoscopic surgery Training Motion analysis Objective assessment Classification Video-based tracking
The authors want to thank all the surgeons, residents, and medical students for their enthusiastic and kindly participation in this study.
The authors declare that no grants or funding sources were received for this work.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Informed consent was obtained from all individual participants included in the study.
- 9.Bansal VK, Raveendran R, Misra MC, Bhattacharjee H, Rajan K, Krishna A, Kumar P, Kumar S (2014) A prospective randomized controlled blinded study to evaluate the effect of short-term focused training program in laparoscopy on operating room performance of surgery residents (CTRI /2012/11/003113). J Surg Educ 71:52–60. https://doi.org/10.1016/j.jsurg.2013.06.012 CrossRefPubMedGoogle Scholar
- 13.Pérez-Escamirosa F, Chousleb-Kalach A, Hernández-Baro MD, Sánchez-Margallo JA, Lorias-Espinoza D, Minor-Martínez A (2016) Construct validity of a video-tracking system based on orthogonal cameras approach for objective assessment of laparoscopic skills. Int J Comput Assist Radiol Surg 11:2283–2293. https://doi.org/10.1007/s11548-016-1388-1 CrossRefPubMedGoogle Scholar
- 14.Oropesa I, Sánchez-González P, Chmarra MK, Lamata P, Fernández A, Sánchez-Margallo JA, Jansen FW, Dankelman J, Sánchez-Margallo FM, Gómez EJ (2013) EVA: laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment. Surg Endosc 27:1029–1039. https://doi.org/10.1007/s00464-012-2513-z CrossRefPubMedGoogle Scholar
- 15.Oropesa I, Chmarra MK, Sánchez-González P, Lamata P, Rodrigues SP, Enciso S, Sánchez-Margallo FM, Jansen FW, Dankelman J, Gómez EJ (2013) Relevance of motion-related assessment metrics in laparoscopic surgery. Surg Innov 20:299–312. https://doi.org/10.1177/1553350612459808 CrossRefPubMedGoogle Scholar
- 17.Escamirosa FP, Flores RMO, García IO, Vidal CR, Martínez AM (2015) Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons. Surg Endosc Other Interv Tech 29:3392–3403. https://doi.org/10.1007/s00464-014-4032-6 CrossRefGoogle Scholar
- 20.Oropesa I, Sánchez-González P, Chmarra MK, Lamata P, Pérez-Rodríguez R, Jansen FW, Dankelman J, Gómez EJ (2014) Supervised classification of psychomotor competence in minimally invasive surgery based on instruments motion analysis. Surg Endosc 28:657–670. https://doi.org/10.1007/s00464-013-3226-7 CrossRefPubMedGoogle Scholar
- 21.Gao Y, Vedula SS, Reiley CE, Ahmidi N, Varadarajan B, Lin HC, Tao L, Zappella L, Béjar B, Yuh DD, Chen CCG, Vidal R, Khudanpur S, Hager GD (2014) JHU-ISI gesture and skill assessment working set (JIGSAWS): a surgical activity dataset for human motion modeling. In: MICCAI workshop: M2CAI, vol 3, p 3Google Scholar
- 23.Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller PA (2018) Evaluating surgical skills from kinematic data using convolutional neural networks. In: MICCAI, pp 214–221Google Scholar
- 33.Pérez F, Sossa H, Martínez R, Lorias D, Minor A (2013) Video-based tracking of laparoscopic instruments using an orthogonal webcams system. World Acad Sci Eng Technol Int J Medical Heal Biomed Bioeng Pharm Eng 7:440–443Google Scholar
- 37.Cleary JG, Cleary JG, Trigg LE (1995) K*: an instance-based learner using an entropic distance measure. in: Proceedings of 12TH international conference on machine learning, pp 108–114Google Scholar
- 38.Oshiro TM, Perez PS, Baranauskas JA (2012) How many trees in a random forest?. Springer, Berlin, pp 154–168Google Scholar