Skip to main content

Guided Depth Enhancement via Anisotropic Diffusion

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNISA,volume 8294)

Abstract

In this paper, we propose to conduct inpainting and upsampling for defective depth maps when aligned color images are given. These tasks are referred to as the guided depth enhancement problem. We formulate the problem based on the heat diffusion framework. The pixels with known depth values are treated as the heat sources and the depth enhancement is performed via diffusing the depth from these sources to unknown regions. The diffusion conductivity is designed in terms of the guidance color image so that a linear anisotropic diffusion problem is formed. We further cast the steady state problem of this diffusion into the famous random walk model, by which the enhancement is achieved efficiently by solving a sparse linear system. The proposed algorithm is quantitatively evaluated on the Middlebury stereo dataset and is applied to inpaint Kinect data and upsample Lidar’s range data. Comparisons to the commonly used bilateral filter and Markov Random Field based methods are also presented, showing that our algorithm is competent.

Keywords

  • Depth Inpainting
  • Depth Upsampling
  • Anisotropic Diffusion

This research work was supported in parts by the National Natural Science Foundation of China via grants 61001171, 61071219, 90820306, and Chinese Universities Scientific Fund.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-03731-8_38
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-03731-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barash, D.: Bilateral Filtering and Anisotropic Diffusion: Towards a Unified Viewpoint, HP Laboratories Israel, HPL-2000-18(R.1) (2000)

    Google Scholar 

  2. Bertalmio, M., Sapiro, G.: Image Inpainting. SIGGRAPH (2000)

    Google Scholar 

  3. Chan, T., Shen, J.: Mathematical Models for Local Non-texture Inpaintings. UCLA CAM TR 00-11 (2001)

    Google Scholar 

  4. Clarenz, U., Diewald, U., Rumpf, M.: Anisotropic Geometric Diffusion in Surface Processing. In: IEEE Conference on Visualization (2000)

    Google Scholar 

  5. Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing 13(9), 1–13 (2004)

    CrossRef  Google Scholar 

  6. Diebel, J., Thrun, S.: An Application of Markov Random Fields to Range Sensing. In: NIPS (2005)

    Google Scholar 

  7. Dolson, J., Baek, J., Plagemann, C., Thrun, S.: Upsampling Range Data in Dynamic Environments. In: CVPR (2010)

    Google Scholar 

  8. Grady, L.: Random Walks for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(11), 1–17 (2006)

    CrossRef  Google Scholar 

  9. He, K., Sun, J., Tang, X.: Guided Image Filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  10. Janoch, A., Karayev, S., Jia, Y., Barron, J.T., Fritz, M., Saenko, K., Darrell, T.: A category-level 3-d object dataset: Putting the kinect to work. In: ICCV Workshops (2011)

    Google Scholar 

  11. Kim, G., Xing, E.P., Fei-Fei, L., Kanade, T.: Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion. In: ICCV (2011)

    Google Scholar 

  12. Kim, S.-Y., Cho, J.-H., Koschan, A., Abidi, M.A.: Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor. In: ICPR (2010)

    Google Scholar 

  13. Larsson, S., Thomee, V.: Partial Differential Equations with Numerical Methods. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  14. Liu, J., Gong, X., Liu, J.: Guided Inpainting and Filtering for Kinect Depth Maps. In: ICPR (2012)

    Google Scholar 

  15. Gong, X., Liu, J., Zhou, W., Liu, J.: Guided Depth Enhancement via A Fast Marching Method. Image and Vision Computing 31, 695–703 (2013)

    CrossRef  Google Scholar 

  16. Microsoft (2010), http://www.xbox.com/en-US/kinect/

  17. Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. International Journal of Computer Vision 99(3), 24–52 (2009)

    CrossRef  Google Scholar 

  18. Park, J., Kim, H., Tai, Y.-W., Brown, M.S., Kweon, I.: High Quality Depth Map Upsampling for 3DTOF Cameras. In: ICCV (2011)

    Google Scholar 

  19. Parker, R.: Kinect depth inpainting and filtering (2011), http://www.radfordparker.com/papers

  20. Perona, P., Malik, J.: Scale-Space and Edge Detection Using Anisotropic Diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(7), 629–639 (1990)

    CrossRef  Google Scholar 

  21. Qi, F., Han, J., Wang, P., Shi, G., Li, F.: Structure Guided Fusion for Depth Map Inpainting. In: Pattern Recognition Letters (2012) (in press)

    Google Scholar 

  22. Scharstein, D., Pal, C.: On Learning conditional random fields for stereo. International Journal of Computer Vision 99(3), 319–337 (2007)

    MathSciNet  Google Scholar 

  23. Telea, A.: An image inpainting technique based on the fast marching method. Journal of Graphics Tools 9(1), 25–36 (2004)

    CrossRef  Google Scholar 

  24. Tschumperle, D., Deriche, R.: Anisotropic Diffusion PDE’s for Multi-Channel Image Regularization: Framework and Applications. In: Advances in Imaging and Electron Physics, pp. 1–47 (2007)

    Google Scholar 

  25. Velodyne. Velodyne HDL-64E (2012), http://velodynelidar.com/lidar/

  26. Wasza, J., Bauer, S., Hornegger, J.: Real-time Preprocessing for Dense 3D Range Imaging on the GPU: Defect Interpolation, Bilateral Temporal Averaging and Guided Filtering. In: Workshop in ICCV (2011)

    Google Scholar 

  27. Weickert, J.: Anisotropic Diffusion in Image Processing. ECMI Series. Teubner-Verlag (1998)

    Google Scholar 

  28. Wong, U., Garney, B., Whittaker, W., Whittaker, R.: Camera and lidar fusion for mapping of actively illuminated subterranean voids. In: Howard, A., Iagnemma, K., Kelly, A. (eds.) Field and Service Robotics. STAR, vol. 62, pp. 421–430. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  29. Yang, Q., Yang, R., Davis, J., Nister, D.: Spatial-Depth Super Resolution for Range Images. In: CVPR (2007)

    Google Scholar 

  30. Zhang, J., Zheng, J., Cai, J.: A Diffusion Approach to Seeded Image Segmentation. In: CVPR (2010)

    Google Scholar 

  31. Zhu, J., Wang, L., Gao, J., Yang, R.: Spatial-Temporal Fusion for High Accuracy Depth Maps Using Dynamic MRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(5), 899–909 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this paper

Cite this paper

Liu, J., Gong, X. (2013). Guided Depth Enhancement via Anisotropic Diffusion. In: Huet, B., Ngo, CW., Tang, J., Zhou, ZH., Hauptmann, A.G., Yan, S. (eds) Advances in Multimedia Information Processing – PCM 2013. PCM 2013. Lecture Notes in Computer Science, vol 8294. Springer, Cham. https://doi.org/10.1007/978-3-319-03731-8_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-03731-8_38

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-03730-1

  • Online ISBN: 978-3-319-03731-8

  • eBook Packages: Computer ScienceComputer Science (R0)