Towards Live Monocular 3D Laparoscopy Using Shading and Specularity Information

  • Toby Collins
  • Adrien Bartoli
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7330)

Abstract

We present steps toward the first real-time system for computing and visualising 3D surfaces viewed in live monocular laparoscopy video. Our method is based on estimating 3D shape using shading and specularity information, and seeks to push current Shape from Shading (SfS) boundaries towards practical, reliable reconstruction. We present an accurate method to model any laparoscope’s light source, and a highly-parallelised SfS algorithm that outperforms the fastest current method. We give details of its GPU implementation that facilitates realtime performance of an average frame-rate of 23fps. Our system also incorporates live 3D visualisation with virtual stereoscopic synthesis. We have evaluated using real laparoscopic data with ground-truth, and we present the successful in-vivo reconstruction of the human uterus. We however draw the conclusion that the shading cue alone is insufficient to reliably handle arbitrary laparoscopic images.

Keywords

Surface Albedo Thin Plate Spline Shape From Shading Camera Response Function Laparoscopic Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ackerman, J. D., Keller, K., Fuchs, H. Surface reconstruction of abdominal organs using laparoscopic structured light for augmented reality. 3DICA (2002)Google Scholar
  2. 2.
    Barron, J.T., Malik, J.: High-frequency shape and albedo from shading. In: CVPR, pp. 2521–2528 (2011)Google Scholar
  3. 3.
    Hu, M., Penney, G., Edwards, P., Figl, M., Hawkes, D.: 3D Reconstruction of Internal Organ Surfaces for Minimal Invasive Surgery. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part I. LNCS, vol. 4791, pp. 68–77. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Lau, W.W., Ramey, N.A., Corso, J.J., Thakor, N.V., Hager, G.D.: Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 494–501. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: An Accurate O(n) Solution to the PnP Problem. IJCV 81, 155–166 (2008)CrossRefGoogle Scholar
  6. 6.
    Malti, A., Bartoli, A., Collins, T.: Template-Based Conformal Shape-from-Motion from Registered Laparoscopic Images. In: MIUA (2011)Google Scholar
  7. 7.
    Mountney, P., Yang, G.-Z.: Motion Compensated SLAM for Image Guided Surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6362, pp. 496–504. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Mueller-Richter, U.D.A., Limberger, A., Weber, P., Ruprecht, K.W., Spitzer, W., Schilling, M.: Possibilities and limitations of current stereo-endoscopy. Surgical Endoscopy 18, 942–947 (2004)CrossRefGoogle Scholar
  9. 9.
    Okatani, T., Deguchi, K.: Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection center. CVIU 66, 119–131 (1997)Google Scholar
  10. 10.
    Prados, E., Faugeras, O.: Perspective Shape from Shading and Viscosity Solutions. In: ICCV, pp. 826–831 (2003)Google Scholar
  11. 11.
    Prados, E., Faugeras, O.: Shape from Shading: a well-posed problem? In: CVPR, pp. 870–877 (2005)Google Scholar
  12. 12.
    Penne, J., Höller, K., Stürmer, M., Schrauder, T., Schneider, A., Engelbrecht, R., Feußner, H., Schmauss, B., Hornegger, J.: Time-of-Flight 3-D Endoscopy. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5761, pp. 467–474. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  13. 13.
    Quartucci, C.H., Tozzi, C.L.: Towards 3D Reconstruction of Endoscope Images using Shape from Shading. In: SIBGRAPI, pp. 90–96 (2000)Google Scholar
  14. 14.
    Stoyanov, D., Darzi, A., Yang, G.Z.: Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 41–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  15. 15.
    Tankus, A., Sochen, N., Yeshurun, Y.: Perspective SfS by fast marching. In: CVPR, pp. 43–49 (2004)Google Scholar
  16. 16.
    Tankus, A., Sochen, N., Yeshurun, Y.: Shape-from-Shading Under Perspective Projection. IJCV 63, 21–43 (2007)CrossRefGoogle Scholar
  17. 17.
    Tsai, P., Shah, M.: Shape From Shading Using Linear Approximation. Image and Vision Computing 12, 487–498 (1994)CrossRefGoogle Scholar
  18. 18.
    Wu, C., Narasimhan, S.G., Jaramaz, B.: Shape-from-Shading under Near Point Lighting and Partial views for Orthopedic Endoscopy. In: PACV (2007)Google Scholar
  19. 19.
    Yeung, S.Y., Tsui, H.T., Yim, A.: Global Shape from Shading for an Endoscope Image. In: Taylor, C., Colchester, A. (eds.) MICCAI 1999. LNCS, vol. 1679, pp. 318–327. Springer, Heidelberg (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Toby Collins
    • 1
  • Adrien Bartoli
    • 1
  1. 1.ALCoV-ISITUniversité d’AuvergneClermont-FerrandFrance

Personalised recommendations