Advertisement

Depth Image Super-Resolution: A Review and Wavelet Perspective

  • Chandra Shaker BalureEmail author
  • M. Ramesh Kini
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 460)

Abstract

We propose an algorithm which utilizes the Discrete Wavelet Transform (DWT) to super-resolve the low-resolution (LR) depth image to a high-resolution (HR) depth image. Commercially available depth cameras capture depth images at a very low-resolution as compared to that of the optical cameras. Having an high-resolution depth camera is expensive because of the manufacturing cost of the depth sensor element. In many applications like robot navigation, human-machine interaction (HMI), surveillance, 3D viewing, etc. where depth images are used, the LR images from the depth cameras will restrict these applications, thus there is a need of a method to produce HR depth images from the available LR depth images. This paper addresses this issue using DWT method. This paper also contributes to the compilation of the existing methods for depth image super-resolution with their advantages and disadvantages, along with a proposed method to super-resolve depth image using DWT. Haar basis for DWT has been used as it has an intrinsic relationship with super-resolution (SR) for retaining the edges. The proposed method has been tested on Middlebury and Tsukuba dataset and compared with the conventional interpolation methods using peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) performance metrics.

Keywords

Discrete wavelet transform Depth image Interpolation Normalization 

References

  1. 1.
    Okutomi, M., Kanade, T.: A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(4), 353–363 (1993)Google Scholar
  2. 2.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International journal of computer vision 47(1–3), 7–42 (2002)Google Scholar
  3. 3.
    Geman, S., Geman, D.: Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence (6), 721–741 (1984)Google Scholar
  4. 4.
    Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: NIPS. vol. 5, pp. 291–298 (2005)Google Scholar
  5. 5.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Sixth International Conference on Computer Vision, 1998. pp. 839–846. IEEE (1998)Google Scholar
  6. 6.
    Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. In: ACM Transactions on Graphics (TOG). vol. 26, p. 96. ACM (2007)Google Scholar
  7. 7.
    Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications-M2SFA2 2008 (2008)Google Scholar
  8. 8.
    Gevrekci, M., Pakin, K.: Depth map super resolution. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 3449–3452. IEEE (2011)Google Scholar
  9. 9.
    Yang, Y., Wang, Z.: Range image super-resolution via guided image filter. In: Proceedings of the 4th International Conference on Internet Multimedia Computing and Service. pp. 200–203. ACM (2012)Google Scholar
  10. 10.
    He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(6), 1397–1409 (2013)Google Scholar
  11. 11.
    Yang, Q., Ahuja, N., Yang, R., Tan, K.H., Davis, J., Culbertson, B., Apostolopoulos, J., Wang, G.: Fusion of median and bilateral filtering for range image upsampling. IEEE Transactions on Image Processing 22(12), 4841–4852 (2013)Google Scholar
  12. 12.
    Lu, J., Forsyth, D.: Sparse depth super resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2245–2253 (2015)Google Scholar
  13. 13.
    Ji, H., Fermuller, C.: Robust wavelet-based super-resolution reconstruction: theory and algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(4), 649–660 (2009)Google Scholar
  14. 14.
    Nguyen, N., Milanfar, P.: A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution). Circuits, Systems and Signal Processing 19(4), 321–338 (2000)Google Scholar
  15. 15.
    Robinson, M.D., Toth, C., Lo, J.Y., Farsiu, S., et al.: Efficient fourier-wavelet super-resolution. IEEE Transactions on Image Processing 19(10), 2669–2681 (2010)Google Scholar
  16. 16.
    Demirel, H., Anbarjafari, G.: Discrete wavelet transform-based satellite image resolution enhancement. IEEE Transactions on Geoscience and Remote Sensing 49(6), 1997–2004 (2011)Google Scholar
  17. 17.
    Demirel, H., Anbarjafari, G.: Image resolution enhancement by using discrete and stationary wavelet decomposition. IEEE Transactions on Image Processing 20(5), 1458–1460 (2011)Google Scholar
  18. 18.
    Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. vol. 1, pp. I–195. IEEE (2003)Google Scholar
  19. 19.
    Peris, M., Maki, A., Martull, S., Ohkawa, Y., Fukui, K.: Towards a simulation driven stereo vision system. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 1038–1042. IEEE (2012)Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2017

Authors and Affiliations

  1. 1.National Institute of Technology Karnataka (NITK)Surathkal, MangaloreIndia

Personalised recommendations