Skip to main content
SpringerLink
Log in

Depth Map Denoising via CDT-Based Joint Bilateral Filter

  • Chapter
  • First Online:
Computer Vision and Machine Learning with RGB-D Sensors

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Bi-modal image processing can be defined as a series of steps taken to enhance a target image with a guidance image. This is done by using exploitable information derived from acquiring two images of the same scene with different image modalities. However, while the potential benefit of bi-modal image processing may be significant, there is an inherent risk; if noise or defects in the guidance image are allowed to transfer to the target image, the target image could become corrupted rather than improved. In this chapter, we present a new method to enhance a noisy depth map from its color information via the joint bilateral filter (JBF) based on common distance transform (CDT). This method is composed of two main steps: CDT map generation and CDT-based JBF. In the first step, a CDT map is generated that represents the degree of pixel-modal similarity between a depth pixel and its corresponding color pixel. Then, based on the CDT map, JBF is carried out in order to enhance depth information with the aid of color information. Experimental results show that CDT-based JBF outperforms other conventional methods objectively and subjectively in terms of noise reduction, as well as inherent visual artifacts suppression.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Borgefors G (1988) Hierarchical chamfer matching: a parametric edge matching algorithm. IEEE Trans Pattern Anal Mach Intell 10(6):849–865

    Article  Google Scholar 

  2. Bose K, Ahuja N (2006) Superresolution and noise filtering using moving least squares. IEEE Trans Image Process 15(8):2239–2248

    Article  Google Scholar 

  3. Brooks S, Saunders I, Dodgson NA (2007) Image compression using sparse colour sampling combined with non-linear image processing. In: Proceedings of electronic imaging: human vision and electronic imaging, vol XII

    Google Scholar 

  4. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698

    Article  Google Scholar 

  5. Chiabrando F, Chiabrando R, Piatti D, Rinaudo F (2009) Sensors for 3D imaging: metric evaluation and calibration of a CCD/CMOS time-of-flight camera. Sensors 9(12):10080–10096

    Article  Google Scholar 

  6. Cho J, Kim S-Y, Ho YS, Lee KH (2008) Dynamic 3D human actor generation method using a time-of-flight depth camera. IEEE Trans Consum Electron 54(4):1514–1521

    Article  Google Scholar 

  7. Diebel J, Thrun S (2005) An application of Markov random fields to range sensing. In: Proceedings of advances in neural information processing systems, pp 291–298

    Google Scholar 

  8. Dorrington AA, Payne AD, Cree MJ (2010) An evaluation of time-of-flight cameras for close range methodology applications. International archives of photogrammetry, remote sensing and spatial information sciences, vol. XXXVIII, Part 5 Commission V Symposium

    Google Scholar 

  9. Elad M (2002) On the origin of the bilateral filter and ways to improve it. IEEE Trans Image Process 11(10):1141–1150

    Article  MathSciNet  Google Scholar 

  10. Fehn C (2003) A 3D-TV system based on video plus depth information. In: Proceedings of asilomar conference on signals, systems and computers, vol 2, pp 1529–1533

    Google Scholar 

  11. Fehn C, Barré R, Pastoor S (2006) Interactive 3-D TV- concepts and key technologies. Proc. IEEE 94(3):524–538

    Article  Google Scholar 

  12. Felzenszwalb PF, Huttenlocher DP (2004) Distance transforms of sampled functions. Cornell computing and information science, TR2004-1963

    Google Scholar 

  13. Feng X, Milanfar P (2002) Multiscale principal components analysis for image local orientation estimation. In: Proceedings of asilomar conference on signals, systems and computers, pp 478–482

    Google Scholar 

  14. Gangwal OP, Djapic B (2010) Real-time implementation of depth map post-processing for 3D-TV in dedicated hardware. In: Proceedings of international conference on consumer electronics

    Google Scholar 

  15. Gonzalez RC, Woods RE (2002) Digital image processing. Prentice Hall, Englewood Cliffs

    Google Scholar 

  16. Grabb R, Tracey C, Puranik A, Davis J (2008) Real-time foreground segmentation via range and color imaging. In: Proceedings of IEEE computer vision and pattern recognition workshops, pp 1–5

    Google Scholar 

  17. Iddan GJ, Yahav G (2001) 3D imaging in the studio and elsewhere. In: Proceedings of SPIE videometrics and optical methods for 3D shape measurements, pp 48–55

    Google Scholar 

  18. Jachalsky J, Schlosser M, Gandolph D (2010) Reliability aware cross multilateral filtering for robust disparity map refinement. In: Proceedings of 3DTV conference

    Google Scholar 

  19. Kauff P, Atzpadin N, Fehn C, Müller M, Schreer O, Smolic A, Tanger R (2007) Depth map creation and image-based rendering for advanced 3D-TV services providing interoperability and scalability. Signal Process. Image Commun. 22(2):217–234

    Article  Google Scholar 

  20. Kim SM, Cha J, Ryu J, Lee KH (2006) Depth video enhancement for haptic interaction using a smooth surface reconstruction. IEICE Trans Inf Syst E89-D(1):37–44

    Google Scholar 

  21. Kim S-Y, Cho J, Koschan A, Abidi M (2010) Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor. In: Proceedings of IEEE international conference on pattern recognition, pp 2358–2361

    Google Scholar 

  22. Kim S-Y, Cho W, Koschan A, Abidi M (2011) Depth map enhancement using adaptive steering kernel regression based on distance transform. In: Proceedings of international symposium and visual computing, vol 6938. Lecture notes in computer science, pp 291–300

    Google Scholar 

  23. Kim S-Y, Lee SB, Ho YS (2006) Three-dimensional natural video system based on layered representation of depth maps. IEEE Trans Consum Electron 52(3):1035–1042

    Article  Google Scholar 

  24. Kopf J, Cohen MF, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. ACM Trans Comput Graph 26(3):1–6

    Article  Google Scholar 

  25. Lee EK, Ho YS (2011) Generation of high-quality depth maps using hybrid camera system for 3-D video. J Vis Commun Image Represent 22:73–84

    Article  Google Scholar 

  26. Petschnigg G, Agrawala M, Hoppe H, Szeliski R, Cohen M, Toyama K (2004) Digital photography with flash and no-flash image pairs. ACM Trans Comput Graph 23(3):664–672

    Article  Google Scholar 

  27. Riemens AK, Gangwal OP, Barenbrug B, Berretty RM (2009) Multi-step joint bilateral depth upsampling. In Proceedings of electronic imaging: visual communications and image processing

    Google Scholar 

  28. Rosenfeld A, Pfaltz J (1968) Distance functions in digital pictures. Pattern Recognit 1:33–61

    Article  MathSciNet  Google Scholar 

  29. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vision 47(1–3):7–42

    Article  MATH  Google Scholar 

  30. Shao L, Yan R, Li X, Liu Y (2014) From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms. IEEE Trans Cybern. doi:10.1109/TCYB.2013.2278548

  31. Shotton J, Fitzgibbon A, Cook M, Blake A (2011) Real-time human pose recognition in parts from single depth images. In: Proceedings of IEEE computer vision and pattern recognition, pp 1297–1304

    Google Scholar 

  32. Takeda H, Farsiu S, Milanfar P (2007) Kernel regression for image processing and reconstruction. IEEE Trans Image Process 16(2):349–366

    Article  MathSciNet  Google Scholar 

  33. Takeda H, Farsiu S, Milanfar P (2008) Deblurring using regularized locally-adaptive kernel regression. IEEE Trans Image Process 17(4):550–563

    Article  MathSciNet  Google Scholar 

  34. Tan H, Tian F, Qiu Y, Wang S, Zhang J (2010) Multihypothesis recursive video denoising based on separation of motion state. IET Image Process 4(4):261–268

    Article  Google Scholar 

  35. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proceedings of IEEE international conference on computer vision, pp 839–846

    Google Scholar 

  36. Wand MP, Jones MC (1995) Kernel smoothing, ser. Monographs on statistics and applied probability. Chapman and Hall, London

    Book  Google Scholar 

  37. Xiao C, Nie Y, Hua W, Zheng W (2010) Fast multi-scale joint bilateral texture upsampling. Visual Comput 26(3):263–275

    Article  Google Scholar 

  38. Yan R, Shao L, Liu Y (2013) Nonlocal hierarchical dictionary learning using wavelets for image denoising. IEEE Trans Image Process 22(12):4689–4698

    Article  MathSciNet  Google Scholar 

  39. Yang Q, Tan KH, Ahuja N (2009) Real-time O(1) bilateral filtering. In: Proceedings of IEEE computer vision and pattern recognition, pp 557–564

    Google Scholar 

  40. Yang Q, Yang R, Davis J, Nistér D (2007) Spatial-depth super resolution for range images. In: Proceedings of IEEE computer vision and pattern recognition, pp 1–8

    Google Scholar 

  41. Yoon SU, Ho YS (2007) Multiple color and depth video coding using a hierarchical representation. IEEE Trans. Circ. Syst. Video Technol. 17(11):1450–1460

    Article  Google Scholar 

  42. Yu H, Zhao L, Wang H (2009) Image denoising using trivariate shrinkage filter in the wavelet domain and joint bilateral filter in the spatial domain. IEEE Trans Image Process 18(10):2364–2369

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the U.S. Air Force under Grant FA8650-10-1-5902. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Air Force Research Laboratory or the U.S. Government. We are thankful to Dr. Sung-Yeol Kim for the implementation and evaluation of various filters.

Author information

Authors and Affiliations

  1. Imaging, Robotics, and Intelligent Systems Laboratory, Department of Electrical Engineering and Computer Science, The University of Tennessee, Knoxville, TN, USA

    Andreas Koschan & Mongi Abidi

Authors
  1. Andreas Koschan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mongi Abidi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Koschan .

Editor information

Editors and Affiliations

  1. University of Sheffield, United Kingdom

    Ling Shao

  2. Civolution Technology, Eindhoven, The Netherlands

    Jungong Han

  3. Microsoft Research, Cambridge, United Kingdom

    Pushmeet Kohli

  4. Microsoft Research, Redmond, Washington, USA

    Zhengyou Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Koschan, A., Abidi, M. (2014). Depth Map Denoising via CDT-Based Joint Bilateral Filter. In: Shao, L., Han, J., Kohli, P., Zhang, Z. (eds) Computer Vision and Machine Learning with RGB-D Sensors. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-08651-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-08651-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-08650-7

  • Online ISBN: 978-3-319-08651-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation