2D/3D Registration of a Preoperative Model with Endoscopic Video Using Colour-Consistency
Image-guided surgery needs an effective and efficient registration between 2D video images of the surgical scene and a preoperative model of a patient from 3D MRI or CT scans. Such an alignment process is difficult due to the lack of robustly trackable features on the operative surface as well as tissue deformation and specularity. In this paper, we propose a novel approach to perform the registration using PTAM camera tracking and colour-consistency. PTAM provides a set of video images with the corresponding camera positions. Registration of the 3D model to the video images can then be achieved by maximization of colour-consistency between all 2D pixels corresponding to a given 3D surface point. An improved algorithm for calculation of visible surface points is provided. It is hoped that PTAM camera tracking using a reduced set of points can be combined with colour-consistency to provide a robust registration. A ground truth simulation test bed has been developed for validating the proposed algorithm and empirical studies have shown that the approach is feasible, with ground truth simulation data providing a capture range of ±9mm/° with a TRE less than 2mm. Our intended application is robot-assisted laparoscopic prostatectomy.
KeywordsAugmented Reality Target Registration Error Camera Tracking Endoscopic Video Preoperative Model
Unable to display preview. Download preview PDF.
- 1.Bouguet, J.Y.: Complete camera calibration toolbox for Matlab, http://www.vision.caltech.edu/bouguetj/calib_doc/
- 3.Hartley, R.I., Zisserman, A.: Multiple view geometry in computer vision, 2nd edn. Cambridge University Press (2004)Google Scholar
- 5.Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: The 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), pp. 225–234 (2007)Google Scholar
- 8.Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis (2010)Google Scholar
- 9.Mountney, P., Lo, B., Thiemjarus, S., Stoyanov, D., Yang, G.-Z.: A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part II. LNCS, vol. 4792, pp. 34–41. Springer, Heidelberg (2007)CrossRefGoogle Scholar
- 11.Mourgues, F., Devernay, F., Coste-Maniere, E.: 3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: IEEE and ACM International Symposium on Augmented Reality (ISAR), p. 191 (2001)Google Scholar
- 12.Newcombe, R.A., Davison, A.J.: Live dense reconstruction with a single moving camera. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)Google Scholar
- 13.Powell, M.J.D.: The bobyqa algorithm for bound constrained optimization without derivatives. Technical Report: Department of Applied Mathematics and Theoretical Physics, Cambridge, England (2009)Google Scholar
- 14.Quartucci Forster, C.H., Tozzi, C.L.: Towards 3D reconstruction of endoscope images using shape from shading. In: Proceedings of Brazilian Symposium on Computer Graphics and Image Processing, p. 90 (2000)Google Scholar
- 16.Spoerk, J., Gendrin, C., Weber, C., Figl, M., Pawiro, S., Furtado, H., Fabri, D., Bloch, C., Bergmann, H., Gröller, E., Birkfellner, W.: High-performance gpu-based rendering for real-time, rigid 2d/3d-image registration and motion prediction in radiation oncology. Journal of Medical Physics p. epub ahead of print (2011)Google Scholar
- 19.Wang, H., Mirota, D., Ishii, M., Hager, G.: Robust motion estimation and structure recovery from endoscopic image sequences with an adaptive scale kernel consensus estimator. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–7 (2008)Google Scholar