Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy
- 325 Downloads
Computer assistance is increasingly common in surgery. However, the amount of information is bound to overload processing abilities of surgeons. We propose methods to recognize the current phase of a surgery for context-aware information filtering. The purpose is to select the most suitable subset of information for surgical situations which require special assistance.
We combine formal knowledge, represented by an ontology, and experience-based knowledge, represented by training samples, to recognize phases. For this purpose, we have developed two different methods. Firstly, we use formal knowledge about possible phase transitions to create a composition of random forests. Secondly, we propose a method based on cultural optimization to infer formal rules from experience to recognize phases.
The proposed methods are compared with a purely formal knowledge-based approach using rules and a purely experience-based one using regular random forests. The comparative evaluation on laparoscopic pancreas resections and adrenalectomies employs a consistent set of quality criteria on clean and noisy input. The rule-based approaches proved best with noisefree data. The random forest-based ones were more robust in the presence of noise.
Formal and experience-based knowledge can be successfully combined for robust phase recognition.
KeywordsContext-awareness Cognition-guided assistance Ontology
The present research was supported by the “SFB TRR 125” funded by the DFG the ESF of Baden-Wuerttemberg.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. For this type of study formal consent was not required since only secondary data, namely performed annotations without patient identifiers, were used.
- 6.Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Edwards PE, Jannin P, Peters TM, Holmes DR, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comp Med Imaging Graph 37(2):83–97CrossRefGoogle Scholar
- 7.Lalys F, Jannin P (2014) Surgical process modelling: a review. Int J Comput Assist Radiol Surg 9(3):495–511. doi: 10.1007/s11548-013-0940-5
- 8.Blum T, Feussner H, Navab N (2010) Modeling and segmentation of surgical workflow from laparoscopic video. In: Jiang T, Navab N, Pluim JPW, Viergever MA (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010, vol 6363. Springer, Berlin, pp 400–407Google Scholar
- 9.Suzuki T, Sakurai Y, Yoshimitsu K, Nambu K, Muragaki Y, Iseki H (2010) Intraoperative multichannel audio–visual information recording and automatic surgical phase and incident detection. In: IEEE EMBS, pp 1190–1193Google Scholar
- 12.Ahmadi A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2007) Recovery of surgical workflow without explicit models. In: Proceedings of the MICCAI, pp 420–428Google Scholar
- 13.Stauder R, Okur A, Peter L, Schneider A, Kranzfelder M, Feussner H, Navab N (2014) Random forests for phase detection in surgical workflow analysis. In: Proceedings of the IPCAIGoogle Scholar
- 14.Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132:396–401Google Scholar
- 15.Lalys F, Riffaud L, Morandi X, Jannin P (2011) Surgical phases detection from microscope videos by combining SVM and HMM medical computer vision. Recognition techniques and applications in medical imagingGoogle Scholar
- 18.Burgert O, Neumuth T, Lempp F, Mudunuri R, Meixensberger J, Strauss G, Dietz A, Jannin P, Lemke HU (2006) Linking top-level ontologies and surgical workflows. Int J Comput Assist Radiol Surg 1(1):437–438Google Scholar
- 24.Katic D, Wekerle AL, Grtner F, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2014) Knowledge-driven formalization of laparoscopic surgeries for rule-based intraoperative context-aware assistance. In: IPCAIGoogle Scholar
- 26.Katic D, Wekerle AL, Gaertner F, Kenngott H, Müller-Stich BP, Dillmann E, Speidel S (2013) Ontology-based prediction of surgical events in laparoscopic surgery. In: Proceedings of the SPIEGoogle Scholar
- 28.Horrocks I, Patel-Schneider PF, Boley H, Tabet S, Grosof B, Dean M (2004) SWRL: a semantic web rule language combining OWL and RuleML. W3C Member Submission. https://www.w3.org/Submission/SWRL/. Accessed 25 Mar 2016
- 29.Ma H, Wang y (2009) Cultural algorithm based on particle swarm optimization for function optimization. In: Fifth international conference on natural computationGoogle Scholar
- 30.Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on NNGoogle Scholar
- 31.Kobti Z, Reynolds R, Kohler T (2004) Agent-based modeling of cultural change in swarm using cultural algorithms. In: SWARMFESTGoogle Scholar
- 32.Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 16(1):72–80Google Scholar
- 33.Speidel S, Benzko J, Sudra G, Azad P, Müller-Stich BP, Gutt , Dillmann R (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. In: SPIE medical imagingGoogle Scholar
- 34.Yang Y, Calmet J (2005) OntoBayes: an ontology-driven uncertainty model. In: International conference on intelligent agents, web technologies and internet commerceGoogle Scholar