Template-Based Conformal Shape-from-Motion-and-Shading for Laparoscopy

  • Abed Malti
  • Adrien Bartoli
  • Toby Collins
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7330)


Shape-from-Shading (SfS) is one of the fundamental techniques to recover depth from a single view. Such a method has shown encouraging but limited results in laparoscopic surgery due to the complex reflectance properties of the organ tissues. On the other hand, Template-Based Deformable-Shape-from-Motion (DSfM) has been recently used to recover a coarse 3D shape in laparoscopy.

We propose to combine both geometric and photometric cues to robustly reconstruct 3D human organs. Our method is dubbed Deformable-Shape-from-Motion-and-Shading (DSfMS). It tackles the limits of classical SfS and DSfM methods: First the photometric template is reconstructed using rigid SfM (Shape-from-Motion) while the surgeon is exploring – but not deforming – the peritoneal environment. Second a rough 3D deformed shape is computed using a recent method for elastic surface from a single laparoscopic image. Third a fine 3D deformed shape is recovered using shading and specularities.

The proposed approach has been validated on both synthetic data and in-vivo laparoscopic videos of a uterus. Experimental results illustrate its effectiveness compared to SfS and DSfM.


Augmented Reality Point Correspondence Attraction Point Photometric Stereo Reprojection Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brunet, F., Hartley, R., Bartoli, A., Navab, N., Malgouyres, R.: Monocular Template-Based Reconstruction of Smooth and Inextensible Surfaces. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 52–66. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Grasa, O.G., Civera, J., Montiel, J.: EKF monocular slam with relocalization for laparoscopic sequences. In: ICRA (2011)Google Scholar
  3. 3.
    Hager, G., Vagvolgyi, B., Yuh, D.: Stereoscopic video overlay with deformable registration. Medicine Meets Virtual Reality (2007)Google Scholar
  4. 4.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press (2003)Google Scholar
  5. 5.
    Hayashibe, M., Suzuki, N., Hattori, A., Nakamura, Y.: Intraoperative Fast 3D Shape Recovery of Abdominal Organs in Laparoscopy. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2489, pp. 356–363. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Hayashibe, M., Suzuki, N., Nakamura, Y.: Laser-scan endoscope system for intraoperativegeometry acquisition and surgical robot safety management. MIA 10, 509–519 (2006)Google Scholar
  7. 7.
    Malti, A., Bartoli, A., Collins, T.: Template-based conformal shape-from-motion from registered laparoscopic images. In: MIUA (2011)Google Scholar
  8. 8.
    Moreno-Noguer, F., Salzmann, M., Lepetit, V., Fua, P.: Capturing 3D stretchable surfaces from single images in closed form. In: CVPR (2009)Google Scholar
  9. 9.
    Morris, D.D., Kanade, T.: Image-consistent surface triangulation. In: CVPR (2000)Google Scholar
  10. 10.
    Nicolau, S.A., Pennec, X., Soler, L., Ayache, N.: A Complete Augmented Reality Guidance System for Liver Punctures: First Clinical Evaluation. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3749, pp. 539–547. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Nishino, K., Zhang, Z., Ikeuchi, K.: Illumination distribution from a sparse set of images for view-dependent image synthesis. In: ICCV (2001)Google Scholar
  12. 12.
    Penne, J., Höller, K., Stürmer, M., Schrauder, T., Schneider, A., Engelbrecht, R., Feußner, H., Schmauss, B., Hornegger, J.: Time-of-Flight 3-D Endoscopy. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5761, pp. 467–474. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  13. 13.
    Perriollat, M., Hartley, R., Bartoli, A.: Monocular template-based reconstruction of inextensible surfaces. In: BMVC (2008)Google Scholar
  14. 14.
    Prados, E., Camilli, F., Faugeras, O.: A unifying and rigorous shape from shading method adapted to realistic data and applications. Journal of Mathematical Imaging and Vision 25(3), 307–328 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Salzmann, M., Urtasun, R., Fua, P.: Local deformation models for monocular 3D shape recovery. In: CVPR (2008)Google Scholar
  16. 16.
    Stoyanov, D., Darzi, A., Yang, G.Z.: Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 41–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  17. 17.
    Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery. In: Jiang, T., et al. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010)Google Scholar
  18. 18.
    Totz, J., Mountney, P., Stoyanov, D., Yang, G.-Z.: Dense Surface Reconstruction for Enhanced Navigation in MIS. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011, Part I. LNCS, vol. 6891, pp. 89–96. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Winter, M.: Image-Based Incremental Reconstruction, Rendering and Augmented Visualziation of Surfaces for Endoscopic Surgery. PhD thesis, University of Erlangen (2009)Google Scholar
  20. 20.
    Wu, C., Varanasi, K., Liu, Y., Seidel, H.-P., Theobalt, C.: Shading-based dynamic shape refinement from multi-view video under general illumination. In: ICCV (2011)Google Scholar
  21. 21.
    Yoo, D.: Three-dimensional morphing of similar shapes using a template mesh. International Journal of Precision Engineering and Manufacturing (2009)Google Scholar
  22. 22.
    Zhang, L., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In: ICCV (2003)Google Scholar
  23. 23.
    Zhang, R., Tsai, P.-S., Cryer, J.E., Shah, M.: Shape from shading: A survey. PAMI 21(8), 690–706 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Abed Malti
    • 1
  • Adrien Bartoli
    • 1
  • Toby Collins
    • 1
  1. 1.ALCoV-ISIT, UMR 6284 CNRS/Université d’AuvergneClermont-FerrandFrance

Personalised recommendations