Advertisement

Mobile Networks and Applications

, Volume 19, Issue 3, pp 414–425 | Cite as

Real-Time Refinement of Kinect Depth Maps using Multi-Resolution Anisotropic Diffusion

  • Krishna Rao Vijayanagar
  • Maziar Loghman
  • Joohee Kim
Article

Abstract

In this paper, we present a novel real-time algorithm to refine depth maps generated by low-cost commercial depth sensors like the Microsoft Kinect. The Kinect sensor falls under the category of RGB-D sensors that can generate a high resolution depth map and color image of a scene. They are relatively inexpensive and are commercially available off-the-shelf. However, owing to their low complexity, there are several artifacts that one encounters in the depth map like holes, mis-alignment between the depth map and color image and lack of sharp object boundaries in the depth map. This is a potential problem in applications that require the color image to be projected in 3-D using the depth map. Such applications depend heavily on the depth map and thus the quality of the depth map is of vital importance. In this paper, a novel multi-resolution anisotropic diffusion based algorithm is presented that accepts a Kinect generated depth map and color image and computes a dense depth map in which the holes have been filled and the edges of the objects are sharpened and aligned with the objects in the color image. The proposed algorithm also ensures that regions in the depth map where the depth is properly estimated are not filtered and ensures that the depth values in the final depth map are the same values that existed in the original depth map. Experimental results are provided to demonstrate the improvement in the quality of the depth map and also execution time results are provided to prove that the proposed method can be executed in real-time.

Keywords

RGB-D sensors Depth map refinement 3-D video Anisotropic diffusion Image pyramids 

References

  1. 1.
    Fehn C (2003) A 3D-TV system based on video plus depth information. In: Proceedings conference record of the thirty-seventh asilomar conference on signals, systems and computers, vol 2, pp 1529–1533Google Scholar
  2. 2.
    Tian C, Lai P-L, Lopez P, Gomila C (2009) View synthesis techniques for 3D video. In: Proceedings of the SPIE, vol 7443, no 2, pp 74430T–74430T-11Google Scholar
  3. 3.
    Scharstein D, Szeliski R, Zabih R (2001) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In: Proceedings of the IEEE workshop on stereo and multi-baseline vision (SMBV 2001), pp 131–140Google Scholar
  4. 4.
    Zhang Z (2012) Microsoft Kinect sensor and its effect. In: IEEE multimedia, vol 19, no 2, pp 4–10Google Scholar
  5. 5.
    Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proceedings of the sixth international conference on computer vision, pp 839–846Google Scholar
  6. 6.
    Kopf J, Cohen MF, Lichinski D, Uyttendaele M (2007) Joint bilateral upsampling. In: Proceedings of ACM transactions graph (SIGGRAPH ’07), vol 26, no 3, article 96Google Scholar
  7. 7.
    Matyunin S, Vatolin D, Berdnikov Y, Smirnov M (2011) Temporal filtering for depth maps generated by Kinect depth camera. In: Proceedings of 3DTV conference: the true vision - capture, transactions and display of 3D video (3DTV-CON ’11), pp 1–4Google Scholar
  8. 8.
    Berdnikov Y, Vatolin D (2011) Real-time depth map occlusion filling and scene background restoration for projected-pattern-based depth cameras. In: Proceedings of the 21st international conference on computer graphics and vision (GraphiCon ’11)Google Scholar
  9. 9.
    Camplani M, Salgado L (2012) Efficient spatio temporal hole filling strategy for Kinect depth maps. In: Proceedings of IS & T/SPIE international conference on 3D image process and applications, vol 8290, pp 82900E1–10Google Scholar
  10. 10.
    Miao C, Fu J, Lu Y, Li S, Chen CW (2012) Texture-assisted Kinect depth inpainting. In: Proceedings of the IEEE international symposium on circuits and systems (ISCAS ’12), pp 604–607Google Scholar
  11. 11.
    Xu K, Zhou J, Wang Z (2012) A method of hole-filling for the depth map generated by Kinect with moving objects detection. In: Proceedings of the IEEE international symposium on broadband multimedia systems and broadcasting (BMSB ’12), pp 1–5Google Scholar
  12. 12.
    Fu J, Wang S., Yan L, Shipeng Z, Zeng W (2012) Kinect-like depth denoising. In: Proceedinds of the IEEE international symposium on circuits and systems (ISCAS ’12), pp 512–515Google Scholar
  13. 13.
    Milani S, Calvagno G (2011) Joint denoising and interpolation of depth maps for MS Kinect sensors. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing (ICASSP ’12), pp 797–800Google Scholar
  14. 14.
    Richardt C, Stoll C, Dodgson NA, Seidel H-P, Theobalt C (2012) Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos. In: Proceedings of eurgraphicsGoogle Scholar
  15. 15.
    Park J, Kim H, Tai Y-W, Brown MS, Kweon I (2011) High quality depth map upsampling for 3D-TOF cameras. In: Proceedings of the internationl conference on computer vision (ICCV ’11), pp 1623–1630Google Scholar
  16. 16.
    Li Y, Sun L (2010) A novel upsampling scheme for depth map compression in 3DTV system. In: Proceedings of picture coding symposium (PCS ’10), pp 186–189Google Scholar
  17. 17.
    Ekmekcioglu E, Mrak M, Worral S, Kondoz A (2009) Utilisation of edge adaptive upsampling in compression of depth map videos for enhanced free-viewpoint rendering. In: Proceedings of international conference on image processing (ICIP ’09), pp 733–736Google Scholar
  18. 18.
    Ekmekcioglu E, Worral S, Kondoz A (2008) Bit-rate adaptive downsampling for the coding of multi-view video with depth information. In: Proceedings of the 3DTV conference: the true vision - capture, transmission and display of 3D video (3DTV-CON ’08), pp 137–140Google Scholar
  19. 19.
    Schwarz S, Olsson R, Sjostrom M, Tourancheau S (2012) Adaptive depth filtering for HEVC 3D video coding. In: Proceedings of picture coding symposium (PCS ’12), pp 49–52Google Scholar
  20. 20.
    Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. In: IEEE transactions on pattern analysis and machine intelligence, vol 12, pp 629–639Google Scholar
  21. 21.
    Canny J (1986) A computational approach to edge detection. In: IEEE transactions pattern analysis and machine intelligence, vol 6, pp 679–698Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Krishna Rao Vijayanagar
    • 1
  • Maziar Loghman
    • 1
  • Joohee Kim
    • 1
  1. 1.Illinois Institute of TechnologyChicagoUSA

Personalised recommendations