LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition

  • Darko Katić
  • Chantal Julliard
  • Anna-Laura Wekerle
  • Hannes Kenngott
  • Beat Peter Müller-Stich
  • Rüdiger Dillmann
  • Stefanie Speidel
  • Pierre Jannin
  • Bernard Gibaud
Original Article



The rise of intraoperative information threatens to outpace our abilities to process it. Context-aware systems, filtering information to automatically adapt to the current needs of the surgeon, are necessary to fully profit from computerized surgery. To attain context awareness, representation of medical knowledge is crucial. However, most existing systems do not represent knowledge in a reusable way, hindering also reuse of data. Our purpose is therefore to make our computational models of medical knowledge sharable, extensible and interoperational with established knowledge representations in the form of the LapOntoSPM ontology. To show its usefulness, we apply it to situation interpretation, i.e., the recognition of surgical phases based on surgical activities.


Considering best practices in ontology engineering and building on our ontology for laparoscopy, we formalized the workflow of laparoscopic adrenalectomies, cholecystectomies and pancreatic resections in the framework of OntoSPM, a new standard for surgical process models. Furthermore, we provide a rule-based situation interpretation algorithm based on SQWRL to recognize surgical phases using the ontology.


The system was evaluated on ground-truth data from 19 manually annotated surgeries. The aim was to show that the phase recognition capabilities are equal to a specialized solution. The recognition rates of the new system were equal to the specialized one. However, the time needed to interpret a situation rose from 0.5 to 1.8 s on average which is still viable for practical application.


We successfully integrated medical knowledge for laparoscopic surgeries into OntoSPM, facilitating knowledge and data sharing. This is especially important for reproducibility of results and unbiased comparison of recognition algorithms. The associated recognition algorithm was adapted to the new representation without any loss of classification power. The work is an important step to standardized knowledge and data representation in the field on context awareness and thus toward unified benchmark data sets.


Context awareness Ontology Situation interpretation Knowledge sharing  



The present research was supported by the “SFB TRR 125” funded by the DFG, the ESF of Baden-Wuerttemberg and the Karlsruhe House of Young Scientists (KHYS)” and for the French partners by a French government support granted to the CominLabs excellence laboratory and managed by the National Research Agency in the “Investing for the Future” program under reference ANR–10–LABX–07–01.

Conflict of interest

Darko Katić, Chantal Julliard, Anna-Laura Wekerle, Hannes Kenngott, Beat Peter Müller-Stich, Rüdiger Dillmann, Stefanie Speidel, Pierre Jannin and Bernard Gibaud declare that they have no conflict of interest.


  1. 1.
    Woods DD, Patterson ES, Roth EM (2002) Can we ever escape from data overload? A cognitive systems diagnosis. Cognit Technol Work 4(1):22–36CrossRefGoogle Scholar
  2. 2.
    Joyce JP, Lapinski GW (1983) A history and overview of the safety parameter display system concept. IEEE Nucl Sci 30:744–749CrossRefGoogle Scholar
  3. 3.
    Kranzfelder M, Staub C, Fiolka A, Schneider A, Gillen S, Wilhelm D, Friess H, Knoll A, Feussner H (2013) Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg Endosc 27(5):1681–1688PubMedCrossRefGoogle Scholar
  4. 4.
    Lemke HU, Ratib OM, Horii SC (2005) Workflow in the operating room: a summary review of the arrowhead 2004 seminar on imaging and informatics. Int Congr Ser 1281:862–867CrossRefGoogle Scholar
  5. 5.
    Cleary K, Chung H, Mun S (2005) Or 2020: the operating room of the future. In: CARS 2005: computer assisted radiology and surgery, Proceedings of the 19th international congress and exhibition, International congress series, vol 1281, pp 832–838Google Scholar
  6. 6.
    Lalys F, Jannin P (2013) Surgical process modelling: a review. Int J Comput Assist Radiol Surg 9(3):495–511. doi: 10.1007/s11548-013-0940-5
  7. 7.
    Blum T, Padoy N, Feussner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assist Radiol Surg 3(5):379–386CrossRefGoogle Scholar
  8. 8.
    Bouarfa L, Jonker PP, Dankelman J (2010) Discovery of high-level tasks in the operating room. J Biomed Inform 44(3):455–462PubMedCrossRefGoogle Scholar
  9. 9.
    Ahmadi A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2007) Recovery of surgical workflow without explicit models. Med Image Comput Comput Assist Interv 9(Pt 1):420–428Google Scholar
  10. 10.
    Stauder R, Okur A, Peter L, Schneider A, Kranzfelder M, Feussner H, Navab N (2014) In: Random forests for phase detection in surgical workflow analysis the 5th international conference on information processing in computer-assisted interventions (IPCAI)Google Scholar
  11. 11.
    Reiley C, Lin H, Varadarajan B, Vagolgyi B, Khudanpur S, Yuh D, Hager G (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132:396–401Google Scholar
  12. 12.
    Lalys F, Riffaud L, Morandi X, Jannin P (2011) Surgical phases detection from microscope videos by combining SVM and HMM. Med Comput Vis Recognit Tech Appl Med ImagingGoogle Scholar
  13. 13.
    Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N (2010) Statistical modeling and recognition of surgical workflow. Med Image Anal 16(3):632–641PubMedCrossRefGoogle Scholar
  14. 14.
    Blum T, Feussner H, Navab N (2010) Modeling and segmentation of surgical workflow from laparoscopic video. Med Image Comput Comput Assist IntervGoogle Scholar
  15. 15.
    Suzuki T, Sakurai Y, Yoshimitsu K, Nambu K, Muragaki Y, Iseki H (2010) Intraoperative multichannel audio-visual information recording and automatic surgical phase and incident detection. In: 32nd annual international conference of the IEEE EMBS, 1190–1193Google Scholar
  16. 16.
    Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6(1):59–71PubMedPubMedCentralCrossRefGoogle Scholar
  17. 17.
    Burgert O, Neumuth T, Lempp F, Mudunuri R, Meixensberger J, Strauss G, Dietz A, Jannin P, Lemke HU (2006) Linking top-level ontologies and surgical workflows. Int J Comput Assist Radiol Surg 1(1):437–438Google Scholar
  18. 18.
    Jannin P, Morandi X (2007) Surgical models for computer-assisted neurosurgery. Neuroimage 37(3):783–791PubMedCrossRefGoogle Scholar
  19. 19.
    Neumuth T, Kaschek B, Neumuth D, Ceschia M, Meixensberger J, Strauss G, Burgert O (2010) An observation support system with an adaptive ontology-driven user interface for the modeling of complex behaviors during surgical interventions. Behav Res Methods 42(4):1049–1058Google Scholar
  20. 20.
    Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif Intell Med 51(3):147–161PubMedCrossRefGoogle Scholar
  21. 21.
    Lalys F, Bouget B, Riffaud R, Jannin P (2013) Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int J Comput Assist Radiol Surg CARS 8:39–49CrossRefGoogle Scholar
  22. 22.
    Oropesa I, Snchez-Gonzlez P, Lamata P, Chmarra MK, Pagador JB, Snchez-Margallo JA, Snchez-Margallo FM, Gmez EJ (2011) Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. J Surg Res 171(1):81–95CrossRefGoogle Scholar
  23. 23.
    Blake C, Keogh E, Merz CJ (1998) UCI repository of machine learning databases,
  24. 24.
    Katic Katic, Wekerle AL, Grtner F, Kenngott HG, Müller-Stich BP, Dillmann R, Speidel S (2014) Knowledge-driven formalization of laparoscopic surgeries for rule-based intraoperative context-aware assistance. Information processing in computer-assisted interventions, Lecture Notes in Computer Science, vol 8498, pp 158–167Google Scholar
  25. 25.
    Gibaud B, Penet C, Pierre J (2014) OntoSPM: a core ontology of surgical procedure models, SURGETICAGoogle Scholar
  26. 26.
    Grenon P, Smith B (2004) SNAP and SPAN: towards dynamic spatial ontology. Spat Cognit Compuat 1(4):69–103Google Scholar
  27. 27.
    Rosse C, Mejino JLV (2007) The foundational model of anatomy ontology. Anat Ontol Bioinf Princ Pract 6:59–117Google Scholar
  28. 28.
  29. 29.
  30. 30.
    Speidel S, Benzko J, Sudra G, Azad P, Müller-Stich BP, Gutt C, Dillmann R (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. SPIE Med Imaging 7261Google Scholar
  31. 31.
    Neumuth T, Strauss G, Meixensberger J, Lemke HU, Burgert O (2006) Acquisition of process descriptions from surgical interventions. DEXA2006Google Scholar
  32. 32.
    Xiang Z, Courtot M, Brinkman RR, Ruttenberg A, He Y (2010) OntoFox: web-based support for ontology reuse. BMC Res Notes 3:175PubMedPubMedCentralCrossRefGoogle Scholar
  33. 33.
    O’Connor MJ, Das A (2009) SQWRL: a query language for OWL OWL: experiences and directions (OWLED). In: 6th International WorkshopGoogle Scholar
  34. 34.
    Shearer R, Motik B, Horrocks I (2008) HermiT: a highly-efficient OWL reasoner. In: 5th International workshop on OWL: experiences and directionsGoogle Scholar
  35. 35.
    O’Connor MJ, Knublauch H, Tu SW, Grossof B, Dean M, Grosso WE, Musen MA (2005) Supporting Rule System Interoperability on the Semantic Web with SWRL. In: 4th International semantic web conference (ISWC) LNCS 3729:974–986Google Scholar
  36. 36.
    Horridge M, Bechhofer S (2011) The OWL API: a Java API for OWL ontologies. Semanti Web J Spec Issue Semant Web Tools Syst 2(1):11–21Google Scholar
  37. 37.
  38. 38.
    Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 16(1):72–80. doi: 10.1197/jamia.M2748 PMCID: PMC2605601
  39. 39.
    Cornet R, de Keizer N (2008) Forty years of SNOMED: a literature review. BMC Med Inform Decis Mak 8(Suppl 1):S2PubMedPubMedCentralCrossRefGoogle Scholar

Copyright information

© CARS 2015

Authors and Affiliations

  • Darko Katić
    • 1
  • Chantal Julliard
    • 1
  • Anna-Laura Wekerle
    • 1
  • Hannes Kenngott
    • 1
  • Beat Peter Müller-Stich
    • 1
  • Rüdiger Dillmann
    • 1
  • Stefanie Speidel
    • 1
  • Pierre Jannin
    • 1
  • Bernard Gibaud
    • 1
  1. 1.Karlsruhe Institute of Technology (KIT)KarlsruheGermany

Personalised recommendations