Modeling and Online Recognition of Surgical Phases Using Hidden Markov Models

  • Tobias Blum
  • Nicolas Padoy
  • Hubertus Feußner
  • Nassir Navab
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5242)


The amount of signals that can be recorded during a surgery, like tracking data or state of instruments, is constantly growing. These signals can be used to better understand surgical workflow and to build surgical assist systems that are aware of the current state of a surgery. This is a crucial issue for designing future systems that provide context-sensitive information and user interfaces.

In this paper, Hidden Markov Models (HMM) are used to model a laparoscopic cholecystectomy. Seventeen signals, representing tool usage, from twelve surgeries are used to train the model. The use of a model merging approach is proposed to build the HMM topology and compared to other methods of initializing a HMM. The merging method allows building a model at a very fine level of detail that also reveals the workflow of a surgery in a human-understandable way. Results for detecting the current phase of a surgery and for predicting the remaining time of the procedure are presented.


Laparoscopic Cholecystectomy Hide Markov Model Dynamic Time Warping Current Phase Observation Probability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Plsek, P.: Complexity and the adoption of innovation in health care. In: Accelerating Quality Improvement in Health Care (2003)Google Scholar
  2. 2.
    Cleary, K., Chung, H.Y., Mun, S.K.: Or 2020: The operating room of the future. Laparoendoscopic and Advanced Surgical Techniques 15(5), 495–500 (2005)CrossRefGoogle Scholar
  3. 3.
    Neumuth, T., Strauß, G., Meixensberger, J., Lemke, H., Burgert, O.: Acquisition of process descriptions from surgical interventions. In: Database and Expert Systems Applications, pp. 602–611 (2006)Google Scholar
  4. 4.
    Blum, T., Padoy, N., Feußner, H., Navab, N.: Workflow mining for visualization and analysis of surgeries. In: Computer Assisted Radiology and Surgery (2008)Google Scholar
  5. 5.
    Leong, J., Nicolaou, M., Atallah, L., Mylonas, G., Darzi, A., Yang, G.-Z.: Hmm assessment of quality of movement trajectory in laparoscopic surgery. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 752–759. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Rosen, J., Brown, J., Chang, L., Sinanan, M., Hannaford, B.: Generalized Approach for Modeling Minimally Invasive Surgery as a Stochastic Process Using a Discrete Markov Model. IEEE Trans. on Biomed. Eng. 53(3), 399–413 (2006)CrossRefGoogle Scholar
  7. 7.
    Lin, H., Shafran, I., Yuh, D., Hager, G.: Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions. Computer Aided Surgery 11(5), 220–230 (2006)CrossRefGoogle Scholar
  8. 8.
    Bhatia, B., Oates, T., Xiao, Y., Hu, P.: Real-Time Identification of Operating Room State from Video. Innovative Applications of Artificial Intelligence, 1761–1766 (2007)Google Scholar
  9. 9.
    James, A., Vieira, D., Lo, B., Darzi, A., Yang, G.Z.: Eye-Gaze Driven Surgical Workflow Segmentation. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part II. LNCS, vol. 4792, pp. 110–117. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Padoy, N., Blum, T., Essa, I., Feußner, H., Berger, M.O., Navab, N.: A boosted segmentation method for surgical workflow analysis. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part I. LNCS, vol. 4791, pp. 102–109. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Padoy, N., Blum, T., Feußner, H., Berger, M.-O., Navab, N.: On-line recognition of surgical activity for monitoring in the operating room. In: Innovative Applications of Artificial Intelligence (2008)Google Scholar
  12. 12.
    Stolcke, A., Omohundro, S.: Inducing probabilistic grammars by Bayesian model merging. In: Grammatical Inference and Applications, pp. 106–118 (1994)Google Scholar
  13. 13.
    Stolcke, A., Omohundro, S.: Best-first model merging for hidden markov model induction. Technical Report TR-94-003, ICSI, Berkeley, CA (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Tobias Blum
    • 1
  • Nicolas Padoy
    • 1
    • 2
  • Hubertus Feußner
    • 3
  • Nassir Navab
    • 1
  1. 1.Computer Aided Medical Procedures (CAMP)Technische Universität MünchenGermany
  2. 2.LORIA-INRIA LorraineNancyFrance
  3. 3.Department of Surgery, Klinikum Rechts der IsarTechnische Universität MünchenGermany

Personalised recommendations