Efficient Multi-organ Segmentation in Multi-view Endoscopic Videos Using Pre-operative Priors
Synergistic fusion of pre-operative (pre-op) and intra- operative (intra-op) imaging data provides surgeons with invaluable insightful information that can improve their decision-making during minimally invasive robotic surgery. In this paper, we propose an efficient technique to segment multiple objects in intra-op multi-view endoscopic videos based on priors captured from pre-op data. Our approach leverages information from 3D pre-op data into the analysis of visual cues in the 2D intra-op data by formulating the problem as one of finding the 3D pose and non-rigid deformations of tissue models driven by features from 2D images. We present a closed-form solution for our formulation and demonstrate how it allows for the inclusion of laparoscopic camera motion model. Our efficient method runs in real-time on a single core CPU making it practical even for robotic surgery systems with limited computational resources. We validate the utility of our technique on ex vivo data as well as in vivo clinical data from laparoscopic partial nephrectomy surgery and demonstrate its robustness in segmenting stereo endoscopic videos.
Unable to display preview. Download preview PDF.
- 1.Bai, X., et al.: Video SnapCut: robust video object cutout using localized classifiers. ACM Trans. Graphics 28(3), 70:1–70:11 (2009)Google Scholar
- 2.Dhandra, B., et al.: Analysis of abnormality in endoscopic images using combined HSI color space and watershed segmentation. In: ICPR, vol. 4, pp. 695–698 (2006)Google Scholar
- 3.Estépar, R., et al.: Multimodality guidance in endoscopic and laparoscopic abdominal procedures. In: Intraop. Imag. Image-Guided Therapy, pp. 767–778 (2014)Google Scholar
- 4.Figueiredo, I., et al.: Variational image segmentation for endoscopic human colonic aberrant crypt foci. IEEE TMI 29(4), 998–1011 (2010)Google Scholar
- 5.Figueiredo, I., et al.: A segmentation model and application to endoscopic images. In: Image Anal. Recogn., pp. 164–171 (2012)Google Scholar
- 7.Mewes, P.W., Neumann, D., Licegevic, O., Simon, J., Juloski, A.L., Angelopoulou, E.: Automatic region-of-interest segmentation and pathology detection in magnetically guided capsule endoscopy. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011, Part III. LNCS, vol. 6893, pp. 141–148. Springer, Heidelberg (2011)CrossRefGoogle Scholar
- 11.Puerto-Souza, G., Mariottini, G.: Toward long-term and accurate augmented-reality display for minimally-invasive surgery. In: ICRA, pp. 5384–5389 (2013)Google Scholar