Surgical Task and Skill Classification from Eye Tracking and Tool Motion in Minimally Invasive Surgery

  • Narges Ahmidi
  • Gregory D. Hager
  • Lisa Ishii
  • Gabor Fichtinger
  • Gary L. Gallia
  • Masaru Ishii
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6363)


In the context of minimally invasive surgery, clinical risks are highly associated with surgeons’ skill in manipulating surgical tools and their knowledge of the closed anatomy. A quantitative surgical skill assessment can reduce faulty procedures and prevent some surgical risks. In this paper focusing on sinus surgery, we present two methods to identify both skill level and task type by recording motion data of surgical tools as well as the surgeon’s eye gaze location on the screen. We generate a total of 14 discrete Hidden Markov Models for seven surgical tasks at both expert and novice levels using a repeated k-fold evaluation method. The dataset contains 95 expert and 139 novice trials of surgery over a cadaver. The results reveal two insights: eye-gaze data contains skill related structures; and adding this info to the surgical tool motion data improves skill assessment by 13.2% and 5.3% for expert and novice levels, respectively. The proposed system quantifies surgeon’s skill level with an accuracy of 82.5% and surgical task type of 77.8%.


Optic Nerve Skill Level Eustachian Tube Functional Endoscopic Sinus Surgery Surgical Tool 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Padoy, N., Mateus, D., Weinland, D., Berger, M.O., Navab, N.: Workflow Monitoring based on 3D Motion Features. In: ICCV Workshop on Video-oriented Object and Event Classification, Kyoto, Japan (September 2009)Google Scholar
  2. 2.
    Lin, H.C., Shafran, I., Yuh, D., Hager, G.D.: Towards Automatic Skill Evaluation: Detection and Segmentation of Robot-Assisted Surgical Motions. Computer Aided Surgery 11(5), 220–230 (2006)CrossRefGoogle Scholar
  3. 3.
    Rosen, J., Brown, J.D., Chang, L., Sinanan, M., Hannaford, B.: Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Trans. Biomed. Eng. 53(3), 399–413 (2006)CrossRefGoogle Scholar
  4. 4.
    Munz, Y., Almoudaris, A., Moorthy, K., Dosis, A., Liddle, A., Darzi, A.: Curriculum-based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality. The American Journal of Surgery 193(6), 774–783 (2007)CrossRefGoogle Scholar
  5. 5.
    Ahmidi, N., U-Thainual, P., Vikal, S., Mousavi, P., Iordachita, I., Fichtinger, G.: A System for Performance Analysis of Surgeon Dexterity in Percutaneous Needle-based Interventions, Imaging Network Ontario. University of Toronto, Canada (2008)Google Scholar
  6. 6.
    Reiley, C., Hager, G.D.: Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5761, pp. 435–442. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Varadarajan, B., Reiley, C., Lin, H., Khudanpur, S., Hager, G.D.: Data-Derived Models for Segmentation with Application to Surgical Assessment and Training. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009, Part I. LNCS, vol. 5761, pp. 426–434. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Leong, J.J., Nicolaou, M., Atallah, L., Mylonas, G.P., Darzi, A.W., Yang, G.Z.: HMM assessment of quality of movement trajectory in laparoscopic surgery. Comput. Aided Surg. 12, 335–346 (2007)CrossRefGoogle Scholar
  9. 9.
    Nicolaou, M., James, A., Darzi, A., Yang, G.Z.: A Study of Saccade Transition for Attention Segregation and Task Strategy in Laparoscopic Surgery. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 97–104. Springer, Heidelberg (2004)Google Scholar
  10. 10.
    Martín-Loeches, M., Schacht, A., Casado, P., Hohlfeld, A., Abdel Rahman, R., Sommer, W.: Rules and heuristics during sentence comprehension: evidences from a dual-task brain potential study. Journal of Cognitive Neuroscience 21(7), 1380–1395 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Narges Ahmidi
    • 1
  • Gregory D. Hager
    • 2
  • Lisa Ishii
    • 3
  • Gabor Fichtinger
    • 1
  • Gary L. Gallia
    • 4
  • Masaru Ishii
    • 3
  1. 1.Queen’s UniversityKingstonCanada
  2. 2.Johns Hopkins UniversityBaltimore
  3. 3.Johns Hopkins Medical InstitutionsBaltimore
  4. 4.Department of NeurosurgeryJohns Hopkins University School of MedicineBaltimore

Personalised recommendations