Data-Derived Models for Segmentation with Application to Surgical Assessment and Training

  • Balakrishnan Varadarajan
  • Carol Reiley
  • Henry Lin
  • Sanjeev Khudanpur
  • Gregory Hager
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5761)


This paper addresses automatic skill assessment in robotic minimally invasive surgery. Hidden Markov models (HMMs) are developed for individual surgical gestures (or surgemes) that comprise a typical bench-top surgical training task. It is known that such HMMs can be used to recognize and segment surgemes in previously unseen trials [1]. Here, the topology of each surgeme HMM is designed in a data-driven manner, mixing trials from multiple surgeons with varying skill levels, resulting in HMM states that model skill-specific sub-gestures. The sequence of HMM states visited while performing a surgeme are therefore indicative of the surgeon’s skill level. This expectation is confirmed by the average edit distance between the state-level “transcripts” of the same surgeme performed by two surgeons with different expertise levels. Some surgemes are further shown to be more indicative of skill than others.


Hide Markov Model Linear Discriminant Analysis Recognition Accuracy Gesture Recognition Novice Surgeon 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

Supplementary material (244 KB)

Supplementary material (242 KB)

Supplementary material (89 KB)

978-3-642-04268-3_53_MOESM4_ESM.avi (60 kb)
Supplementary material (60 KB)
978-3-642-04268-3_53_MOESM5_ESM.avi (107 kb)
Supplementary material (107 KB)

Supplementary material (416 KB)

Supplementary material (410 KB)

Supplementary material (106 KB)

Supplementary material (120 KB)

Supplementary material (109 KB)

978-3-642-04268-3_53_MOESM11_ESM.avi (43 kb)
Supplementary material (43 KB)
978-3-642-04268-3_53_MOESM12_ESM.avi (21 kb)
Supplementary material (21 KB)
978-3-642-04268-3_53_MOESM13_ESM.avi (26 kb)
Supplementary material (27 KB)
978-3-642-04268-3_53_MOESM14_ESM.avi (76 kb)
Supplementary material (76 KB)


  1. 1.
    Reiley, C., Lin, H., Varadarajan, B., Khudanpur, S., Yuh, D.D., Hager, G.D.: Automatic recognition of surgical motions using statistical modeling for capturing variability. In: MMVR (2008)Google Scholar
  2. 2.
    Shuford, M.: Robotically assisted laparoscopic radical prostatectomy: a brief review of outcomes. Proc. Baylor University Medical Center 20(4), 354–356 (2007)Google Scholar
  3. 3.
    Lenihan Jr., J., Kovanda, C., Seshadri-Kreaden, U.: What is the Learning Curve for Robotic Assisted Gynecologic Surgery? J. Min. Inv. Gyn. 15(5), 589–594 (2008)CrossRefGoogle Scholar
  4. 4.
    Martin, J., Regehr, G., Reznick, R., MacRae, H., Murnaghan, J., Hutchison, C., Brown, M.: Objective structured assessment of technical skill (OSATS) for surgical residents. British Journal of Surgery 84(2), 273–278 (1997)CrossRefGoogle Scholar
  5. 5.
    Dosis, A., Bello, F., Gillies, D., Undre, S., Aggarwal, R., Darzi, A.: Laparoscopic task recognition using hidden markov models. In: MMVR (2005)Google Scholar
  6. 6.
    Richards, C., Rosen, J., Hannaford, B., Pellegrini, C., Sinanan, M.: Skills evaluation in minimally invasive surgery using force/torque signatures. Surgical Endoscopy 14, 791–798 (2000)CrossRefGoogle Scholar
  7. 7.
    Rosen, J., Solazzo, M., Hannaford, B., Sinanan, M.N.: Task decomposition of laparoscopic surgery for objective evaluation of surgical residents’ learning curve using hidden markov model. Computer Aided Surgery 7(1), 49–61 (2002)CrossRefGoogle Scholar
  8. 8.
    Lin, H.C., Shafran, I., Murphy, T.E., Okamura, A.M., Yuh, D.D., Hager, G.D.: Automatic detection and segmentation of robot-assisted surgical motions. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3749, pp. 802–810. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals of Eugenics 7, 179–188 (1936)CrossRefGoogle Scholar
  10. 10.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  11. 11.
    Varadarajan, B., Khudanpur, S., Dupoux, E.: Unsupervised learning of acoustic sub-word units. In: Proceedings of ACL 2008: HLT, Short Papers, 165–168 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Balakrishnan Varadarajan
    • 1
  • Carol Reiley
    • 2
  • Henry Lin
    • 2
  • Sanjeev Khudanpur
    • 1
    • 2
  • Gregory Hager
    • 2
  1. 1.Department of Electrical and Computer EngineeringUSA
  2. 2.Department of Computer ScienceJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations