Multimedia Tools and Applications

, Volume 76, Issue 9, pp 11285–11303 | Cite as

A new depth image quality metric using a pair of color and depth images

Article

Abstract

Typical depth quality metrics require the ground truth depth image or stereoscopic color image pair, which are not always available in many practical applications. In this paper, we propose a new depth image quality metric which demands only a single pair of color and depth images. Our observations reveal that the depth distortion is strongly related to the local image characteristics, which in turn leads us to formulate a new distortion assessment method for the edge and non-edge pixels in the depth image. The local depth distortion is adaptively weighted using the Gabor filtered color image and added up to the global depth image quality metric. The experimental results show that the proposed metric closely approximates the depth quality metrics that use the ground truth depth or stereo color image pair.

Keywords

Depth image Image quality assessment Reduced reference Quality metric 

Notes

Acknowledgments

Dr. Thanh-Ha Le’s work was supported by the basic research projects in natural science in 2012 of the National Foundation for Science & Technology Development (Nafosted), Vietnam (102.01-2012.36, Coding and communication of multiview video plus depth for 3D Television Systems). Prof. Seung-Won Jung’s research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2014R1A1A2057970).

References

  1. 1.
    Chai B-B, Sethuraman S, Sawhney HS, Hatrack P (2004) Depth map compression for real-time view-based rendering. Pattern Recogn Lett 25(7):755–766CrossRefGoogle Scholar
  2. 2.
    Daugman J (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Opt Soc Amer J A: Opt Image Sci 2:1160–1169CrossRefGoogle Scholar
  3. 3.
    D’Angelo A, Barni M, Menegaz G (2007) Perceptual quality evaluation of geometric distortions in images. In: Proceedings SPIE, Human Vision and Electronic Imaging XII, pp 1–12Google Scholar
  4. 4.
    D’Angelo A, Zhaoping L, Barni M (2010) A full-reference quality metric for geometrically distorted images. IEEE Trans Image Process 19(4):867–881MathSciNetCrossRefGoogle Scholar
  5. 5.
    Fehn C (2004) Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3d-TV. In: Proceedings of SPIE, pp 93–104Google Scholar
  6. 6.
    Fehn C, De la Barré R, Pastoor S (2006) Interactive 3-DTV-concepts and key technologies. Proc IEEE 94(3):524–538Google Scholar
  7. 7.
    Gallant J, Braun J, Essen DV (1993) Selectivity for polar, hyperbolic, and cartesian gratings in macaque visual cortex. Science 259(5091):100–103CrossRefGoogle Scholar
  8. 8.
    Geiger A, Lenz P, Urtasun R (2012) Are We ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In: Conference on Computer Vision and Pattern RecognitionGoogle Scholar
  9. 9.
    Grigorescu C, Petkov N, Kruizinga P (2002) Comparison of texture features based on Gabor filters. IEEE Trans Image Process 11(10):1160–1167MathSciNetCrossRefGoogle Scholar
  10. 10.
    Grigorescu C, Petkov N, Westenberg MA (2003) Contour detection based on nonclassical receptive field inhibition. IEEE Trans Image Process 12(7):729–739CrossRefGoogle Scholar
  11. 11.
    Guney F, Geiger A (2015) Displets: Resolving Stereo Ambiguities using Object Knowledge. In: Conference on Computer Vision and Pattern RecognitionGoogle Scholar
  12. 12.
    Ha K, Bae S. -H., Kim M (2013) An objective no-reference perceptual quality assessment metric based on temporal complexity and disparity for stereoscopic video. IEIE Trans Smart Signal Comput 2(5):255–265Google Scholar
  13. 13.
    Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334CrossRefGoogle Scholar
  14. 14.
    Hartley RI (1999) Theory and practice of projective rectification. Int J Comput Vis 35(2):115–127CrossRefGoogle Scholar
  15. 15.
    Jain M, Gupta M, Jain NK (2014) The design of the IIR differintegrator and its application in edge detection. J Inf Process Syst 10(2):223–239CrossRefGoogle Scholar
  16. 16.
    Janoch A, Karayev S, Jia Y, Barron JT, Fritz M, Saenko K, Darrell T (2011) Acategory-level 3-d object dataset: putting the kinect to work. In: Proceedings of ICCV Workshop on Consumer Depth Cameras for Computer Vision, pp 1168–1174Google Scholar
  17. 17.
    Jung S-W (2014) Image contrast enhancement using color and depth histograms. IEEE Signal Process Lett 21(4):382–385CrossRefGoogle Scholar
  18. 18.
    Khongkraphan K (2014) An efficient color edge detection using the Mahalanobis distance. J Inf Process Syst 10(4):589–601CrossRefGoogle Scholar
  19. 19.
    Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using brief propagation and a self-adapting dissimilarity measure. In: Proceedings of IEEE Conference on Pattern Recongnition, pp 15–18Google Scholar
  20. 20.
    Le T-H, Lee S, Jung S -W, Won CS (2015) Reduced reference quality metric for depth images. In: Proceedings Advanced Multimedia and Ubiquitous Engineering, pp 117–122Google Scholar
  21. 21.
    Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1-3):7–42CrossRefMATHGoogle Scholar
  22. 22.
    Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from rgbd images. In: Proceedings of European Conference on Computer Vision, pp 746–760Google Scholar
  23. 23.
    Smith SM, Brady JM (1997) SUSAN-A new approach to low level image processing. Int J Comput Vis 23(1):45–78CrossRefGoogle Scholar
  24. 24.
    Smolic A, Mueller K, Stefanoski N, Ostermann J, Gotchev A, Akar GB, Triantafyllidis G, Koz A (2007) Coding algorithms for 3DTV-a survey. IEEE Trans Circ Syst Video Technol 17(11):1606–1621CrossRefGoogle Scholar
  25. 25.
    Smolic A, Mueller K, Merkle P, Kauff P, Wiegand T (2009) An overview of available and emerging 3D video formats and depth enhanced stero as efficient generic solution. In: Proceedings of 27th Conference on Picture Coding Symposium, pp 389–392Google Scholar
  26. 26.
    Strecha C, Fransens R, Gool L.V (2006) Combined depth and outlier estimation in multi-view stereo. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 2394–2401Google Scholar
  27. 27.
    Um G. -M., Kim KY, Ahn C, Lee K.H (2005) Three-dimensional scene reconstruction using multiview images and depth camera, pp 271–280Google Scholar
  28. 28.
    Vipparthi S, Nagar S (2014) Color directional local quinary patterns for content based indexing and retrieval. Human-centric Comput Inf Sci 4(6):1–13Google Scholar
  29. 29.
    Wang Z-F, Zheng Z-G (2008) A region based stereoo matching algorithm using cooperative optimization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recongnition, pp 1–8Google Scholar
  30. 30.
    Xiao J, Owens A, Torralba A (2013) SUN3d: A database of big spaces reconstructed using SfM and object labels. In: Proceedings International Conference on Computer Vision, pp 1–8Google Scholar
  31. 31.
    Yang Q, Yang R, Davis J, Nister D (2007) Spatial-depth super resolution for range images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8Google Scholar
  32. 32.
    Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A (2014) Learning deep features for scene recognition using places database. In: Proceedings of Advances in Neural Information Processing Systems, pp 1–9Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.University and Engineering and TechnologyVietnam National UniversityHanoiVietnam
  2. 2.Department of Multimedia EngineeringDongguk UniversityPildong-ro 1gil, Jung-guKorea
  3. 3.Department of Electronics and Electrical EngineeringDongguk UniversityPildong-ro 1gil, Jung-guKorea

Personalised recommendations