Objective assessment based on motion-related metrics and technical performance in laparoscopic suturing

  • Juan A. Sánchez-Margallo
  • Francisco M. Sánchez-Margallo
  • Ignacio Oropesa
  • Silvia Enciso
  • Enrique J. Gómez
Original Article



The aim of this study is to present the construct and concurrent validity of a motion-tracking method of laparoscopic instruments based on an optical pose tracker and determine its feasibility as an objective assessment tool of psychomotor skills during laparoscopic suturing.


A group of novice (\({<}10\) laparoscopic procedures), intermediate (11–100 laparoscopic procedures) and experienced (\({>}100\) laparoscopic procedures) surgeons performed three intracorporeal sutures on an ex vivo porcine stomach. Motion analysis metrics were recorded using the proposed tracking method, which employs an optical pose tracker to determine the laparoscopic instruments’ position. Construct validation was measured for all 10 metrics across the three groups and between pairs of groups. Concurrent validation was measured against a previously validated suturing checklist. Checklists were completed by two independent surgeons over blinded video recordings of the task.


Eighteen novices, 15 intermediates and 11 experienced surgeons took part in this study. Execution time and path length travelled by the laparoscopic dissector presented construct validity. Experienced surgeons required significantly less time (\(p<0.008\)), travelled less distance using both laparoscopic instruments (\(p<0.013\)) and made more efficient use of the work space (\(p<0.018\)) compared with novice and intermediate surgeons. Concurrent validation showed strong correlation between both the execution time and path length and the checklist score (\(r =-0.712\) and \(r=-0.731\), \(p<0.001\)).


The suturing performance was successfully assessed by the motion analysis method. Construct and concurrent validity of the motion-based assessment method has been demonstrated for the execution time and path length metrics. This study demonstrates the efficacy of the presented method for objective evaluation of psychomotor skills in laparoscopic suturing. However, this method does not take into account the quality of the suture. Thus, future works will focus on developing new methods combining motion analysis and qualitative outcome evaluation to provide a complete performance assessment to trainees.


Laparoscopic surgery Surgical training Objective assessment Motion metrics Intracorporeal suturing Suturing checklist 


  1. 1.
    Smith CD, Farrell TM, McNatt SS, Metreveli RE (2001) Assessing laparoscopic manipulative skills. Am J Surg 181(6):547–550CrossRefPubMedGoogle Scholar
  2. 2.
    van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97(7):972–987CrossRefPubMedGoogle Scholar
  3. 3.
    Fried GM, Feldman LS (2008) Objective assessment of technical performance. World J Surg 32(2):156–160CrossRefPubMedGoogle Scholar
  4. 4.
    Parsons BA, Blencowe NS, Hollowood AD, Grant JR (2011) Surgical training: the impact of changes in curriculum and experience. J Surg Educ 68(1):44–51CrossRefPubMedGoogle Scholar
  5. 5.
    Roberts KE, Bell RL, Duffy AJ (2006) Evolution of surgical skills training. World J World J Gastroenterol 12(20):3219–3224CrossRefPubMedGoogle Scholar
  6. 6.
    Ritchie WP (2004) Basic certification in surgery by the American Board of Surgery (ABS). What does it mean? Does it have value? Is it relevant? A personal opinion. Ann Surg 239(2):133–139CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Kurashima Y, Feldman LS, Kaneva PA, Fried GM, Bergman S, Demyttenaere SV, Li C, Vassiliou MC (2014) Simulation-based training improves the operative performance of totally extraperitoneal (TEP) laparoscopic inguinal hernia repair: a prospective randomized controlled trial. Surg Endosc 28(3):783–788CrossRefPubMedGoogle Scholar
  8. 8.
    Sánchez-Margallo JA, Sánchez-Margallo FM, Oropesa I, Gómez EJ (2014) Systems and technologies for objective evaluation of technical skills in laparoscopic surgery. Minim Invasive Ther Allied Technol 23(1):40–51CrossRefPubMedGoogle Scholar
  9. 9.
    Oropesa I, Sánchez-González P, Lamata P, Chmarra MK, Pagador JB, Sánchez-Margallo JA, Sánchez-Margallo FM, Gómez EJ (2011) Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. J Surg Res 175(1):e81–e95CrossRefGoogle Scholar
  10. 10.
    Vassiliou MC, Feldman LS, Fraser SA, Charlebois P, Chaudhury P, Stanbridge DD, Fried GM (2007) Evaluating intraoperative laparoscopic skill: direct observation versus blinded videotaped performances. Surg Innov 14(3):211–216CrossRefPubMedGoogle Scholar
  11. 11.
    Hasson HM (2008) Simulation training in laparoscopy using a computerized physical reality simulator. JSLS 12(4):363–367PubMedPubMedCentralGoogle Scholar
  12. 12.
    Pérez F, Ordorica RM, Oropesa I, Zalles CR, Minor A (2015) Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons. Surg Endosc 29(11):3392–3403CrossRefGoogle Scholar
  13. 13.
    Sánchez-Margallo JA, Sánchez-Margallo FM, Pagador JB, Gómez EJ, Sánchez-González P, Usón J, Moreno J (2011) Video-based assistance system for training in minimally invasive surgery. Minim Invasive Ther Allied Technol 20(4):197–205CrossRefPubMedGoogle Scholar
  14. 14.
    Oropesa I, Sánchez-González P, Chmarra MK, Lamata P, Fernández A, Sánchez-Margallo JA, Jansen FW, Dankelman J, Sánchez-Margallo FM, Gómez EJ (2013) EVA: laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment. Surg Endosc 27(3):1029–1039CrossRefPubMedGoogle Scholar
  15. 15.
    Chmarra MK, Klein S, de Winter JCF, Jansen F-W, Dankelman J (2010) Objective classification of residents based on their psychomotor laparoscopic skills. Surg Endosc 24(5):1031–1039CrossRefPubMedGoogle Scholar
  16. 16.
    Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240(3):518–528CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Mason JD, Ansell J, Warren N, Torkington J (2013) Is motion analysis a valid tool for assessing laparoscopic skill? Surg Endosc 27(5):1468–1477CrossRefPubMedGoogle Scholar
  18. 18.
    Kroeze SGC, Mayer EK, Chopra S, Aggarwal R, Darzi A, Patel A (2009) Assessment of laparoscopic suturing skills of urology residents: a Pan-European study. Eur Urol 56(5):865–872CrossRefPubMedGoogle Scholar
  19. 19.
    Yamaguchi S, Yoshida D, Kenmotsu H, Yasunaga T, Konishi K, Ieiri S, Nakashima H, Tanoue K, Hashizume M (2011) Objective assessment of laparoscopic suturing skills using a motion-tracking system. Surg Endosc 25(3):771–775CrossRefPubMedGoogle Scholar
  20. 20.
    Uemura M, Yamashita M, Tomikawa M, Obata S, Souzaki R, Ieiri S, Ohuchida K, Matsuoka N, Katayama T, Hashizume M (2015) Objective assessment of the suture ligature method for the laparoscopic intestinal anastomosis model using a new computerized system. Surg Endosc 29(2):444–452CrossRefPubMedGoogle Scholar
  21. 21.
    Sánchez-Margallo JA, Sánchez-Margallo FM, Pagador Carrasco JB, Oropesa García I, Gómez Aguilera EJ, Moreno Del Pozo J (2013) Usefulness of an optical tracking system in laparoscopic surgery for motor skills assessment. Cir Esp 92(6):421–428CrossRefPubMedGoogle Scholar
  22. 22.
    Sánchez-Margallo JA, Sánchez-Margallo FM, Pagador JB, Oropesa I, Lucas M, Gómez EJ, Moreno J (2013) Technical evaluation of a third generation optical pose tracker for motion analysis and image-guided surgery. Lect Notes Comput Sci 7761:75–82CrossRefGoogle Scholar
  23. 23.
    Enciso Sanz S, Sánchez Margallo FM, Díaz-Güemes Martín-Portugués I, Usón Gargallo J (2012) Preliminary validation of the Simulap \(\textregistered \) physical simulator and its assessment system for laparoscopic surgery. Cir Esp 90(1):38–44CrossRefPubMedGoogle Scholar
  24. 24.
    Sánchez-Margallo FM, Díaz-Güemes I, Pérez FJ, Sánchez MA, Loscertales B, Usón J (2009) Preliminary results with a training program for thoracoscopic atrial fibrillation therapy. Surg Endosc 23(8):1882–1886CrossRefPubMedGoogle Scholar
  25. 25.
    Pagador JB, Sánchez-Margallo FM, Sánchez-Peralta LF, Sánchez-Margallo JA, Moyano-Cuevas JL, Enciso-Sanz S, Usón-Gargallo J, Moreno J (2012) Decomposition and analysis of laparoscopic suturing task using tool-motion analysis (TMA): improving the objective assessment. Int J Comput Assist Radiol Surg 7(2):305–313CrossRefPubMedGoogle Scholar
  26. 26.
    Hofstad EF, Våpenstad C, Chmarra MK, Langø T, Kuhry E, Mårvik R (2013) A study of psychomotor skills in minimally invasive surgery: what differentiates expert and nonexpert performance. Surg Endosc 27(3):854–863CrossRefPubMedGoogle Scholar
  27. 27.
    Moorthy K, Munz Y, Dosis A, Bello F, Chang A, Darzi A (2004) Bimodal assessment of laparoscopic suturing skills: construct and concurrent validity. Surg Endosc 18(11):1608–1612PubMedGoogle Scholar
  28. 28.
    Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245(6):992–999CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190(1):107–113CrossRefPubMedGoogle Scholar

Copyright information

© CARS 2016

Authors and Affiliations

  • Juan A. Sánchez-Margallo
    • 1
  • Francisco M. Sánchez-Margallo
    • 2
  • Ignacio Oropesa
    • 3
    • 4
  • Silvia Enciso
    • 2
  • Enrique J. Gómez
    • 3
    • 4
  1. 1.Bioengineering and Health Technologies UnitJesús Usón Minimally Invasive Surgery CentreCáceresSpain
  2. 2.Laparoscopy UnitJesús Usón Minimally Invasive Surgery CentreCáceresSpain
  3. 3.Biomedical Engineering and Telemedicine Centre (GBT), ETSI TelecomunicaciónUniversidad Politécnica de Madrid (UPM)MadridSpain
  4. 4.Networking Research Center on BioengineeringBiomaterials and Nanomedicine (CIBER-BBN)ZaragozaSpain

Personalised recommendations