Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery

  • Danail Stoyanov
  • Marco Visentini Scarzanella
  • Philip Pratt
  • Guang-Zhong Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6361)


The recovery of 3D tissue structure and morphology during robotic assisted surgery is an important step towards accurate deployment of surgical guidance and control techniques in minimally invasive therapies. In this article, we present a novel stereo reconstruction algorithm that propagates disparity information around a set of candidate feature matches. This has the advantage of avoiding problems with specular highlights, occlusions from instruments and view dependent illumination bias. Furthermore, the algorithm can be used with any feature matching strategy allowing the propagation of depth in very disparate views. Validation is provided for a phantom model with known geometry and this data is available online in order to establish a structured validation scheme in the field. The practical value of the proposed method is further demonstrated by reconstructions on various in vivo images of robotic assisted procedures, which are also available to the community.

Additional material can be found at .


Augmented Reality Minimally Invasive Surgery Phantom Model Seed Match Medical Image Computing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Stoyanov, D., Lerotic, M., Mylonas, G., Chun, A.J., Yang, G.-Z.: Intra-operative Visualizations: Perceptual Fidelity and Human Factors. J. Display Technol. 4, 491–501 (2008)CrossRefGoogle Scholar
  2. 2.
    Taylor, R.H., Stoianovici, D.: Medical Robotics in Computer-Integrated Surgery. IEEE Trans. Robot. Autom. 19, 765–781 (2003)CrossRefGoogle Scholar
  3. 3.
    Scharstein, D., Szeliski, R.: A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. Int. J. Comput. Vision 47, 7–42 (2002)zbMATHCrossRefGoogle Scholar
  4. 4.
    Brown, M.Z., Burschka, D., Hager, G.D.: Advances in Computational Stereo. IEEE Trans. Pattern Anal. Mach. Intell. 25, 993–1008 (2003)CrossRefGoogle Scholar
  5. 5.
    Devernay, F., Mourgues, F., Coste-Maniere, E.: Towards endoscopic augmented reality for robotically assisted minimally invasive cardiac surgery. In: Medical Imaging and Augmented Reality (2001)Google Scholar
  6. 6.
    Mourgues, F., Devernay, F., Malandain, G., Coste-Manière, È.: 3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: International Symposium on Augmented Reality (2001)Google Scholar
  7. 7.
    Lau, W.W., Ramey, N.A., Corso, J., Thakor, N.V., Hager, G.D.: Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 494–501 (2004)Google Scholar
  8. 8.
    Richa, R., Poignet, P., Liu, C.: Efficient 3D Tracking for Motion Compensation in Beating Heart Surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention, vol. II, pp. 684–691 (2008)Google Scholar
  9. 9.
    Hager, G., Vagvolgyi, B., Yuh, D.: Stereoscopic Video Overlay with Deformable Registration. In: Medicine Meets Virtual Reality (2007)Google Scholar
  10. 10.
    Stoyanov, D., Darzi, A., Yang, G.-Z.: A Practical Approach Towards Accurate Dense 3D Depth Recovery for Robotic Laparoscopic Surgery. Comput. Aided Surg. 10, 199–208 (2005)CrossRefGoogle Scholar
  11. 11.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 593–600 (1994)Google Scholar
  12. 12.
    Stoyanov, D., Mylonas, G.P., Deligianni, F., Darzi, A., Yang, G.-Z.: Soft-tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 139–146 (2005)Google Scholar
  13. 13.
    Sinha, S.N., Frahm, J.-M., Pollefeys, M., Genc, Y.: Feature Tracking and Matching in Video Using Programmable Graphics Hardware. Mach. Vision and Appl. (2007)Google Scholar
  14. 14.
    Mountney, P., Lo, B.P.L., Thiemjarus, S., Stoyanov, D., Yang, G.-Z.: A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 34–41 (2007)Google Scholar
  15. 15.
    Lhuillier, M., Quan, L.: Robust dense matching using local and global geometric constraints. In: International Conference on Pattern Recognition, vol. 1, pp. 968–972 (2000)Google Scholar
  16. 16.
    Yoon, K.-J., Kweon, I.S.: Adaptive Support-Weight Approach for Correspondence Search. IEEE Trans. Pattern Anal. Mach. Intell. 28, 650–656 (2006)CrossRefGoogle Scholar
  17. 17.
    Yang, Q., Wang, L., Yang, R., Stewenius, H., Nister, D.: Stereo Matching with Color-weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling. IEEE Trans. Pattern Anal. Mach. Intell. 31, 492–504 (2009)CrossRefGoogle Scholar
  18. 18.
    Veksler, O.: Fast Variable Window for Stereo Correspondence using Integral Images. In: International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 556–564 (2003)Google Scholar
  19. 19.
    Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Optic. Soc. America 4, 629–642 (1987)CrossRefGoogle Scholar
  20. 20.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient Belief Propagation for Early Vision. Int. J. Comput. Vision 70 (2006)Google Scholar
  21. 21.
    Fung, J., Mann, S., Aimone, C.: OpenVIDIA: Parallel GPU Computer Vision. In: ACM Multimedia, pp. 849–852 (2005)Google Scholar
  22. 22.

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Danail Stoyanov
    • 1
  • Marco Visentini Scarzanella
    • 1
  • Philip Pratt
    • 1
  • Guang-Zhong Yang
    • 1
  1. 1.Institute of Biomedical EngineeringImperial College LondonLondonUK

Personalised recommendations