2D/3D Registration of a Preoperative Model with Endoscopic Video Using Colour-Consistency

  • Ping-Lin Chang
  • Dongbin Chen
  • Daniel Cohen
  • Philip “Eddie” Edwards
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7264)


Image-guided surgery needs an effective and efficient registration between 2D video images of the surgical scene and a preoperative model of a patient from 3D MRI or CT scans. Such an alignment process is difficult due to the lack of robustly trackable features on the operative surface as well as tissue deformation and specularity. In this paper, we propose a novel approach to perform the registration using PTAM camera tracking and colour-consistency. PTAM provides a set of video images with the corresponding camera positions. Registration of the 3D model to the video images can then be achieved by maximization of colour-consistency between all 2D pixels corresponding to a given 3D surface point. An improved algorithm for calculation of visible surface points is provided. It is hoped that PTAM camera tracking using a reduced set of points can be combined with colour-consistency to provide a robust registration. A ground truth simulation test bed has been developed for validating the proposed algorithm and empirical studies have shown that the approach is feasible, with ground truth simulation data providing a capture range of ±9mm/° with a TRE less than 2mm. Our intended application is robot-assisted laparoscopic prostatectomy.


Augmented Reality Target Registration Error Camera Tracking Endoscopic Video Preoperative Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bouguet, J.Y.: Complete camera calibration toolbox for Matlab,
  2. 2.
    Clarkson, M.J., Rueckert, D., Hill, D.L.G., Hawkes, D.J.: Using photo-consistency to register 2D optical images of the human face to a 3D surface model. IEEE Trans. Pattern Analysis and Machine Intelligence 23(11), 1266–1280 (2001)CrossRefGoogle Scholar
  3. 3.
    Hartley, R.I., Zisserman, A.: Multiple view geometry in computer vision, 2nd edn. Cambridge University Press (2004)Google Scholar
  4. 4.
    Jannin, P., Fitzpatrick, J.M., Hawkes, D.J., Pennec, X., Shahidi, R., Vannier, M.W.: Validation of medical image processing in image-guided therapy. IEEE Transactions on Medical Imaging 21(12), 1445–1449 (2002)CrossRefGoogle Scholar
  5. 5.
    Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: The 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), pp. 225–234 (2007)Google Scholar
  6. 6.
    Lavallee, S., Sautot, P., Troccaz, J., Cinquin, P., Merloz, P.: Computer-assisted spine surgery: A technique for accurate transpedicular screw fixation using CT data and a 3-D optical localizer. Journal of Image Guided Surgery 1(1), 65–73 (1995)CrossRefGoogle Scholar
  7. 7.
    Maes, F., Vandermeulen, D., Suetens, P.: Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information. Medical Image Analysis 3(4), 373–386 (1999)CrossRefGoogle Scholar
  8. 8.
    Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis (2010)Google Scholar
  9. 9.
    Mountney, P., Lo, B., Thiemjarus, S., Stoyanov, D., Yang, G.-Z.: A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part II. LNCS, vol. 4792, pp. 34–41. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Mountney, P., Stoyanov, D., Davison, A.J., Yang, G.-Z.: Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006, Part I. LNCS, vol. 4190, pp. 347–354. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Mourgues, F., Devernay, F., Coste-Maniere, E.: 3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: IEEE and ACM International Symposium on Augmented Reality (ISAR), p. 191 (2001)Google Scholar
  12. 12.
    Newcombe, R.A., Davison, A.J.: Live dense reconstruction with a single moving camera. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2010)Google Scholar
  13. 13.
    Powell, M.J.D.: The bobyqa algorithm for bound constrained optimization without derivatives. Technical Report: Department of Applied Mathematics and Theoretical Physics, Cambridge, England (2009)Google Scholar
  14. 14.
    Quartucci Forster, C.H., Tozzi, C.L.: Towards 3D reconstruction of endoscope images using shape from shading. In: Proceedings of Brazilian Symposium on Computer Graphics and Image Processing, p. 90 (2000)Google Scholar
  15. 15.
    Salzmann, M., Fua, P.: Deformable surface 3D reconstruction from monocular images. Synthesis Lectures on Computer Vision 2(1), 1–113 (2010)CrossRefGoogle Scholar
  16. 16.
    Spoerk, J., Gendrin, C., Weber, C., Figl, M., Pawiro, S., Furtado, H., Fabri, D., Bloch, C., Bergmann, H., Gröller, E., Birkfellner, W.: High-performance gpu-based rendering for real-time, rigid 2d/3d-image registration and motion prediction in radiation oncology. Journal of Medical Physics p. epub ahead of print (2011)Google Scholar
  17. 17.
    Stoyanov, D., Darzi, A.W., Yang, G.-.Z.: Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 41–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  18. 18.
    Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  19. 19.
    Wang, H., Mirota, D., Ishii, M., Hager, G.: Robust motion estimation and structure recovery from endoscopic image sequences with an adaptive scale kernel consensus estimator. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–7 (2008)Google Scholar
  20. 20.
    Wu, C.H., Sun, Y.N., Chen, Y.C., Chang, C.C.: Endoscopic feature tracking and scale-invariant estimation of soft-tissue structures. IEICE Transactions on Information and Systems E91-D, 351–360 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ping-Lin Chang
    • 1
  • Dongbin Chen
    • 2
  • Daniel Cohen
    • 2
  • Philip “Eddie” Edwards
    • 2
  1. 1.Department of ComputingImperial CollegeLondonUnited Kingdom
  2. 2.Department of Surgery and CancerImperial CollegeLondonUnited Kingdom

Personalised recommendations