Knowledge-Driven Formalization of Laparoscopic Surgeries for Rule-Based Intraoperative Context-Aware Assistance

  • Darko Katić
  • Anna-Laura Wekerle
  • Fabian Gärtner
  • Hannes Kenngott
  • Beat Peter Müller-Stich
  • Rüdiger Dillmann
  • Stefanie Speidel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8498)


The rise of intraoperatively available information threatens to outpace our abilities to process data and thus cause informational overload. Context-aware systems, filtering information to match the current situation in the OR, will be necessary to reap all benefits of integrated and computerized surgery. To interpret surgical situations, such systems need a robust set of knowledge to make sense of intraoperative measurements. Building on our own ontology for laparoscopy, we formalized the workflow of laparoscopic adrenalectomies, cholecystectomies and pancreatic resections and developed a novel, rule-based situation interpretation algorithm based on OWL and SWRL to recognize phases of these surgeries. The system was evaluated on ground truth data from 19 manually annotated surgeries with an average recognition rate of 89%.


Laparoscopic Surgery Cognitive Surgery Context-Awareness 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Woods, D.D., Patterson, E.S., Roth, E.M.: Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis Cognition, Technology and Work 4(1), 22–36 (2002)CrossRefGoogle Scholar
  2. 2.
    Joyce, J.P., Lapinski, G.W.: A history and overview of the safety parameter display system concept. IEEE Nuclear Science NS-30 (1983)Google Scholar
  3. 3.
    Blum, T., Feußner, H., Navab, N.: Modeling and Segmentation of Surgical Workflow from Laparoscopic Video. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part III. LNCS, vol. 6363, pp. 400–407. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Cleary, K., Chung, H., Mun, S.: Or 2020: The operating room of the future. Laparoendoscopic and Advanced Surgical Techniques (2005)Google Scholar
  5. 5.
    Kranzfelder, M., Staub, C., Fiolka, A., Schneider, A., Gillen, S., Wilhelm, D., Friess, H., Knoll, A., Feussner, H.: Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg. Endosc. 27(5), 1681–1688 (2013)CrossRefGoogle Scholar
  6. 6.
    Blum, T., Padoy, N., Feussner, H., Navab, N.: Workflow mining for visualization and analysis of surgeries. Int. J. Comput. Assisted. Radiol. Surg. 3(5), 379–386 (2008)CrossRefGoogle Scholar
  7. 7.
    Bouarfa, L., Jonker, P.P., Dankelman, J.: Discovery of high-level tasks in the operating room. J. Biomed. Inform. 44(3), 455–462 (2010)CrossRefGoogle Scholar
  8. 8.
    Ahmadi, S.-A., Sielhorst, T., Stauder, R., Horn, M., Feussner, H., Navab, N.: Recovery of surgical workflow without explicit models. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 420–428. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Reiley, C., Lin, H., Varadarajan, B., Vagolgyi, B., Khudanpur, S., Yuh, D., Hager, G.: Automatic Recognition of Surgical Motions Using Statistical Modeling for Capturing Variability Medicine Meets Virtual Reality (2008)Google Scholar
  10. 10.
    Lalys, F., Riffaud, L., Morandi, X., Jannin, P.: Surgical phases detection from microscope videos by combining SVM and HMM. In: Menze, B., Langs, G., Tu, Z., Criminisi, A. (eds.) MICCAI 2010. LNCS, vol. 6533, pp. 54–62. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  11. 11.
    Padoy, N., Blum, T., Ahmadi, S.A., Feussner, H., Berger, M.O., Navab, N.: Statistical modeling and recognition of surgical workflow. Med. Image Anal. 16(3), 632–641 (2010)CrossRefGoogle Scholar
  12. 12.
    Suzuki, T., Sakurai, Y., Yoshimitsu, K., Nambu, K., Muragaki, Y.: Iseki. H.: Intraoperative multichannel audio-visual information recording and automatic surgical phase and incident detection. In: 32nd Annual International Conference of the IEEE EMBS, pp. 1190–1193 (2010)Google Scholar
  13. 13.
    Neumuth, T., Jannin, P., Schlomberg, J., Meixensberger, J., Wiedemann, P., Burgert, O.: Analysis of Surgical Intervention Populations Using Generic Surgical Process Models. International Journal of CARS (2010)Google Scholar
  14. 14.
    Burgert, O., Neumuth, T., Lempp, F., Mudunuri, R., Meixensberger, J., Strauss, G., Dietz, A., Jannin, P., Lemke, H.U.: Linking top-level ontologies and surgical workflows. Int. J. Comput. Assisted. Radiol. Surg. 1(1), 437–438 (2006)Google Scholar
  15. 15.
    Jannin, P., Morandi, X.: Surgical models for computer-assisted neurosurgery. Neuroimage 37(3), 783–791 (2007)CrossRefGoogle Scholar
  16. 16.
    Lalys, F., Bouget, B., Riffaud, R., Jannin, P.: Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int. J. CARS 8, 39–49 (2013)CrossRefGoogle Scholar
  17. 17.
    Oropesa, I., Sánchez-González, P., Lamata, P., Chmarra, M.K., Pagador, J.B., Sánchez-Margallo, J.A., Sánchez-Margallo, F.M., Gómez, E.: Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. J. Surg. Res. 171(1), 81–95 (2011)CrossRefGoogle Scholar
  18. 18.
    Adam, E.C.: Fighter cockpits of the future. In: Proceedings of 12th IEEE/AIAA Digital Avionics Systems Conference (DASC), pp. 318–323 (1993)Google Scholar
  19. 19.
    Baader, F., Calvanese, D., McGuiness, D.L., Nardi, D., Patel-Schneider, P.F.: The Description Logic Handbook: Theory, Implementation, Applications. Cambridge University Press, Cambridge (2003) ISBN 0-521-78176-0Google Scholar
  20. 20.
    SWRL: A Semantic Web Rule Language Combining OWL and RuleML W3C Member Submission (2004)Google Scholar
  21. 21.
    Katic, D., Sudra, G., Speidel, S., Castrillon-Oberndorfer, G., Eggers, G., Dillmann, R.: Knowledge-based Situation Interpretation for Context-aware Augmented Reality in Dental Implant Surgery. In: Proc. Medical Imaging and Augmented Reality (2010)Google Scholar
  22. 22.
    Katic, D., Wekerle, A.L., Görtler, J., Spengler, P., Bodenstedt, S., Röhl, S., Suwelack, S., Kenngott, H.G., Wagner, M., Mueller-Stich, B.P., Dillmann, R., Speidel, S.: Context-aware Augmented Reality in laparoscopic surgery. Comp. Med. Imag. and Graph. 37(2), 174–182 (2013)CrossRefGoogle Scholar
  23. 23.
    Neumann, B., Moeller, R.: On scene interpretation with description logics. Image and Vision Computing 26 (2008)Google Scholar
  24. 24.
    Speidel, S., Benzko, J., Sudra, G., Azad, P., Mütich, B.P., Gutt, C., Dillmann, R.: Automatic classification of minimally invasive instruments based on endoscopic image sequences. In: SPIE Medical Imaging, vol. 7261 (2009)Google Scholar
  25. 25.
    Neumuth, T., Strauß, G., Meixensberger, J., Lemke, H.U., Burgert, O.: Acquisition of process descriptions from surgical interventions. In: Bressan, S., Küng, J., Wagner, R. (eds.) DEXA 2006. LNCS, vol. 4080, pp. 602–611. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  26. 26.
    Kripalani, S., Bengtzen, R., Henderson, L., Jacobson, T.: Clinical Research in Low-Literacy Populations: Using Teach-Back to Assess Comprehension of Informed Consent and Privacy Information IRB: Ethics and Human Research (2008)Google Scholar
  27. 27.
  28. 28.
    Shearer, R., Motik, B., Horrocks, I.: HermiT: A Highly-Efficient OWL Reasoner. In: 5th Int. Workshop on OWL: Experiences and Directions (2008)Google Scholar
  29. 29.
    Neumuth, T., Jannin, P., Strauss, G., Meixensberger, J., Burgert, O.: Validation of knowledge acquisition for surgical process models. J. Am. Med. Inform. Assoc. (2009)Google Scholar
  30. 30.
    Smith, E.E., Langston, C., Nisbett, R.E.: The case for rules in reasoning. Cognitive Science 16(1), 1–40 (1992)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Darko Katić
    • 1
  • Anna-Laura Wekerle
    • 2
  • Fabian Gärtner
    • 1
  • Hannes Kenngott
    • 2
  • Beat Peter Müller-Stich
    • 2
  • Rüdiger Dillmann
    • 1
  • Stefanie Speidel
    • 1
  1. 1.Institute for Anthropomatics (IFA), Humanoids and Intelligence Systems Laboratories (HIS)Karlsruhe Institute of Technology (KIT)Germany
  2. 2.Department of General, Abdominal and Transplantation SurgeryUniversity of HeidelbergGermany

Personalised recommendations