Abstract
Purpose
Computer assistance is increasingly common in surgery. However, the amount of information is bound to overload processing abilities of surgeons. We propose methods to recognize the current phase of a surgery for context-aware information filtering. The purpose is to select the most suitable subset of information for surgical situations which require special assistance.
Methods
We combine formal knowledge, represented by an ontology, and experience-based knowledge, represented by training samples, to recognize phases. For this purpose, we have developed two different methods. Firstly, we use formal knowledge about possible phase transitions to create a composition of random forests. Secondly, we propose a method based on cultural optimization to infer formal rules from experience to recognize phases.
Results
The proposed methods are compared with a purely formal knowledge-based approach using rules and a purely experience-based one using regular random forests. The comparative evaluation on laparoscopic pancreas resections and adrenalectomies employs a consistent set of quality criteria on clean and noisy input. The rule-based approaches proved best with noisefree data. The random forest-based ones were more robust in the presence of noise.
Conclusion
Formal and experience-based knowledge can be successfully combined for robust phase recognition.
Similar content being viewed by others
References
Skagestad P (1993) Thinking with machines: intelligence augmentation, evolutionary epistemology, and semiotic. J Soc Evol Syst 16(2):157–180
Woods DD, Patterson ES, Roth EM (2002) Can we ever escape from data overload? A cognitive systems diagnosis. Cogn Technol Work 4(1):22–36
Dixon B, Daly MJ, Chan H, Vescan AD, Witterick IJ, Irish JC (2013) Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc 27(2):454–461
Joyce JP, Lapinsky GW (1983) A history and overview of the safety parameter display system concept. IEEE Trans Nucl Sci 30(1):744–749
Kranzfelder M, Staub C, Fiolka A, Schneider A, Gillen S, Wilhelm D, Friess H, Knoll A, Feussner H (2013) Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg Endosc 27(5):1681–1688
Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Edwards PE, Jannin P, Peters TM, Holmes DR, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comp Med Imaging Graph 37(2):83–97
Lalys F, Jannin P (2014) Surgical process modelling: a review. Int J Comput Assist Radiol Surg 9(3):495–511. doi:10.1007/s11548-013-0940-5
Blum T, Feussner H, Navab N (2010) Modeling and segmentation of surgical workflow from laparoscopic video. In: Jiang T, Navab N, Pluim JPW, Viergever MA (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010, vol 6363. Springer, Berlin, pp 400–407
Suzuki T, Sakurai Y, Yoshimitsu K, Nambu K, Muragaki Y, Iseki H (2010) Intraoperative multichannel audio–visual information recording and automatic surgical phase and incident detection. In: IEEE EMBS, pp 1190–1193
Blum T, Padoy N, Feussner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assist Radiol Surg 3(5):379–386
Bouarfa L, Jonker PP, Dankelman J (2010) Discovery of high-level tasks in the operating room. J Biomed Inform 44(3):455–462
Ahmadi A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2007) Recovery of surgical workflow without explicit models. In: Proceedings of the MICCAI, pp 420–428
Stauder R, Okur A, Peter L, Schneider A, Kranzfelder M, Feussner H, Navab N (2014) Random forests for phase detection in surgical workflow analysis. In: Proceedings of the IPCAI
Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132:396–401
Lalys F, Riffaud L, Morandi X, Jannin P (2011) Surgical phases detection from microscope videos by combining SVM and HMM medical computer vision. Recognition techniques and applications in medical imaging
Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N (2010) Statistical modeling and recognition of surgical workflow. Med Image Anal 16(3):632–641
Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6(1):59–71
Burgert O, Neumuth T, Lempp F, Mudunuri R, Meixensberger J, Strauss G, Dietz A, Jannin P, Lemke HU (2006) Linking top-level ontologies and surgical workflows. Int J Comput Assist Radiol Surg 1(1):437–438
Jannin P, Morandi X (2007) Surgical models for computer-assisted neurosurgery. Neuroimage 37(3):783–791
Neumuth T, Kaschek B, Neumuth D, Ceschia M, Meixensberger J, Strauss G, Burgert O (2010) An observation support system with an adaptive ontology-driven user interface for the modeling of complex behaviors during surgical interventions. Behav Res Methods 42(4):1049–1058
Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif Intell Med 51(3):147–161
Lalys F, Bouget B, Riffaud R, Jannin P (2013) Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int J CARS 8:39–49
Katic D, Julliard C, Wekerle AL, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S, Jannin P, Gibaud B (2015) LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J CARS 10:1427–1434
Katic D, Wekerle AL, Grtner F, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2014) Knowledge-driven formalization of laparoscopic surgeries for rule-based intraoperative context-aware assistance. In: IPCAI
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Katic D, Wekerle AL, Gaertner F, Kenngott H, Müller-Stich BP, Dillmann E, Speidel S (2013) Ontology-based prediction of surgical events in laparoscopic surgery. In: Proceedings of the SPIE
Forestier G, Riffaud L, Jannin P (2015) Automatic phase prediction from low-level surgical activities. Int J Comput Assist Radiol Surg 20(6):833–884
Horrocks I, Patel-Schneider PF, Boley H, Tabet S, Grosof B, Dean M (2004) SWRL: a semantic web rule language combining OWL and RuleML. W3C Member Submission. https://www.w3.org/Submission/SWRL/. Accessed 25 Mar 2016
Ma H, Wang y (2009) Cultural algorithm based on particle swarm optimization for function optimization. In: Fifth international conference on natural computation
Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on NN
Kobti Z, Reynolds R, Kohler T (2004) Agent-based modeling of cultural change in swarm using cultural algorithms. In: SWARMFEST
Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 16(1):72–80
Speidel S, Benzko J, Sudra G, Azad P, Müller-Stich BP, Gutt , Dillmann R (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. In: SPIE medical imaging
Yang Y, Calmet J (2005) OntoBayes: an ontology-driven uncertainty model. In: International conference on intelligent agents, web technologies and internet commerce
Acknowledgments
The present research was supported by the “SFB TRR 125” funded by the DFG the ESF of Baden-Wuerttemberg.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical standard
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. For this type of study formal consent was not required since only secondary data, namely performed annotations without patient identifiers, were used.
Rights and permissions
About this article
Cite this article
Katić, D., Schuck, J., Wekerle, AL. et al. Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy. Int J CARS 11, 881–888 (2016). https://doi.org/10.1007/s11548-016-1379-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-016-1379-2