Uncertainty-Encoded Augmented Reality for Robot-Assisted Partial Nephrectomy: A Phantom Study
In most robot-assisted surgical interventions, multimodal fusion of pre- and intra-operative data is highly valuable, affording the surgeon a more comprehensive understanding of the surgical scene observed through the stereo endoscopic camera. More specifically, in the case of partial nephrectomy, fusing pre-operative segmentations of kidney and tumor with the stereo endoscopic view can guide tumor localization and the identification of resection margins. However, the surgeons are often unable to reliably assess the levels of trust they can bestow on what is overlaid on the screen. In this paper, we present the proof-of-concept of an uncertainty-encoded augmented reality framework and novel visualizations of the uncertainties derived from the pre-operative CT segmentation onto the surgeon’s stereo endoscopic view. To verify its clinical potential, the proposed method is applied to an ex vivo lamb kidney. The results are contrasted to different visualization solutions based on crisp segmentation demonstrating that our method provides valuable additional information that can help the surgeon during the resection planning.
KeywordsAugmented Reality Partial Nephrectomy Stereo Match Stereo Camera Normalize Cross Correlation
Unable to display preview. Download preview PDF.
- 1.Pratt, P., Mayer, E., Vale, J., Cohen, D., Edwards, E., Darzi, A., Yang, G.-Z.: An effective visualisation and registration system for image-guided robotic partial nephrectomy. Journal of Robotic Surgery, 1–9 (2012)Google Scholar
- 7.Sinop, A.K., Grady, L.: A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm. In: ICCV, pp. 1–8 (2007)Google Scholar
- 8.Hartley, R., Zisserman, A.: Multiple view geometry in computer vision, 2nd edn. Cambridge Univ. Press (2004)Google Scholar
- 9.Pollefeys, M., Koch, R., Van Gool, L.: A simple and efficient rectification method for general motion. In: ICCV, pp. 496–501 (1999)Google Scholar
- 10.Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010)CrossRefGoogle Scholar
- 12.Röhl, S., Bodenstedt, S., Suwelack, S., Kenngott, H., Müller-Stich, B., Dillmann, R., Speidel, S.: Real-time surface reconstruction from stereo endoscopic images for intraoperative registration. In: Proc. SPIE, vol. 7964, p. 796414 (2011)Google Scholar