Skip to main content
SpringerLink
Log in

No-reference quality metric for HEVC compression distortion estimation in depth maps

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Multiview video plus depth (MVD) is the most popular 3D video format due to its efficient compression and provision for novel view generation enabling the free-viewpoint applications. In addition to color images, MVD format provides depth maps which are exploited to generate intermediate virtual views using the depth image-based rendering (DIBR) techniques. Compression affects the quality of the depth maps which in turn may introduce various structural and textural distortions in the DIBR-synthesized images. Estimation of the compression-related distortion in depth maps is very important for a high-quality 3D experience. The task becomes challenging when the corresponding reference depth maps are unavailable, e.g., when evaluating the quality on the decoder side. In this paper, we present a no-reference quality assessment algorithm to estimate the distortion in the depth maps induced by compression. The proposed algorithm exploits the depth saliency and local statistical characteristics of the depth maps to predict the compression distortion. The proposed ‘depth distortion evaluator’ (DDE) is evaluated on depth videos from standard MVD database compressed with the state-of-the-art high-efficiency video coding at various quality levels. The results demonstrate that DDE can be used to effectively estimate the compression distortion in depth videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Version HM 11.0 of the HEVC reference software with main profile.

  2. http://www.di.unito.it/~farid/3DQA/DDE.html.

References

  1. Tanimoto, M.: FTV: free-viewpoint television. Signal Process. Image Commun. 27(6), 555–570 (2012)

    Article  Google Scholar 

  2. Tehrani, M.P., et al.: Introduction of super multiview video systems for requirement discussion. Document ISO/IEC JTC1/SC29/WG11/M31052, Geneva, CH, Oct 2013

  3. Vetro, A., Tourapis, A.M., Muller, K., Chen, T.: 3D-TV content storage and transmission. IEEE Trans. Broadcast. 57(2), 384–394 (2011)

    Article  Google Scholar 

  4. Muller, K., Merkle, P., Tech, G., Wiegand, T.: 3D video formats and coding methods. In: Proceedings of International Conference on Image Processing, pp. 2389–2392 (2010)

  5. Smolic, A., Mueller, K., Merkle, P., Kauff, P., Wiegand, T.: An overview of available and emerging 3D video formats and depth enhanced stereo as efficient generic solution. In: Proceedings of Picture Coding Symposium, pp. 1–4 (2009)

  6. Ballocca, G., D’Amato, P., Grangetto, M., Lucenteforte, M.: Tile format: a novel frame compatible approach for 3d video broadcasting. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 1–4 (2011)

  7. Vetro, A.: Frame compatible formats for 3D video distribution. In: Proceedings of International Conference on Image Processing, pp. 2405–2408 (2010)

  8. Cagnazzo, M., Pesquet-Popescu, B., Dufaux, F.: 3D Video Representation and Formats, pp. 102–120. Wiley, Hoboken (2013)

    Google Scholar 

  9. Smolic, A., et al.: Multi-view video plus depth (MVD) format for advanced 3D video systems. Doc. ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q6 (2007)

  10. Call for proposals on 3D video coding technology, Mar 2011. Document ISO/IEC JTC1/SC29/WG11

  11. Fehn, C.: Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. In: Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems XI, vol. 5291, pp. 93–104 (2004)

  12. Domanski, M., et al.: High efficiency 3D video coding using new tools based on view synthesis. IEEE Trans. Image Process. 22(9), 3517–3527 (2013)

    Article  Google Scholar 

  13. Farid, M.S., Lucenteforte, M., Grangetto, M.: Panorama view with spatiotemporal occlusion compensation for 3D video coding. IEEE Trans. Image Process. 24(1), 205–219 (2015)

    Article  MathSciNet  Google Scholar 

  14. Maugey, T., Ortega, A., Frossard, P.: Graph-based representation for multiview image geometry. IEEE Trans. Image Process. 24(5), 1573–1586 (2015)

    Article  MathSciNet  Google Scholar 

  15. Ozkalayci, B.O., Alatan, A.A.: 3D planar representation of stereo depth images for 3DTV applications. IEEE Trans. Image Process. 23(12), 5222–5232 (2014)

    Article  MathSciNet  Google Scholar 

  16. Farid, M.S., Lucenteforte, M., Grangetto, M.: A panoramic 3d video coding with directional depth aided inpainting. In: Proceedings of International Conference on Image Processing, pp. 3233–3237 (2014)

  17. Purica, A.I., Mora, E.G., Pesquet-Popescu, B., Cagnazzo, M., Ionescu, B.: Multiview plus depth video coding with temporal prediction view synthesis. IEEE Trans. Circuits Syst. Video Technol. 26(2), 360–374 (2016)

    Article  Google Scholar 

  18. Merkle, P., Smolic, A., Muller, K., Wiegand, T.: Efficient prediction structures for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1461–1473 (2007)

    Article  Google Scholar 

  19. Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A.: Overview of the H.264/AVC video coding standard. IEEE Trans. Circuits Syst. Video Technol. 13(7), 560–576 (2003)

    Article  Google Scholar 

  20. Kang, J., Chung, K.: High-performance depth map coding for 3D-AVC. Signal Image Video Process. 10(6), 1017–1024 (2016)

    Article  Google Scholar 

  21. Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  22. Ohm, J., Sullivan, G.J., Schwarz, H., Tan, T.K., Wiegand, T.: Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE Trans. Circuits Syst. Video Technol. 22(12), 1669–1684 (2012)

    Article  Google Scholar 

  23. Tech, G., Chen, Y., Müller, K., Ohm, J.R., Vetro, A., Wang, Y.K.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26(1), 35–49 (2016)

    Article  Google Scholar 

  24. Sullivan, G.J., et al.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Topics Signal Process. 7(6), 1001–1016 (2013)

    Article  Google Scholar 

  25. Muller, K., et al.: 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans. Image Process. 22(9), 3366–3378 (2013)

    Article  MathSciNet  Google Scholar 

  26. Jing, R., Zhang, Q., Wang, B., Cui, P., Yan, T., Huang, J.: CART-based fast CU size decision and mode decision algorithm for 3D-HEVC. Signal Image Video Process. 13(2), 209–216 (2019)

    Article  Google Scholar 

  27. Merkle, P., Morvan, Y., Smolic, A., Farin, D., Müller, K., de With, P.H.N., Wiegand, T.: The effects of multiview depth video compression on multiview rendering. Signal Process. Image Commun. 24(1–2), 73–88 (2009)

    Article  Google Scholar 

  28. Fang, L., Cheung, N.M., Tian, D., Vetro, A., Sun, H., Au, O.C.: An analytical model for synthesis distortion estimation in 3D video. IEEE Trans. Image Process. 23(1), 185–199 (2014)

    Article  MathSciNet  Google Scholar 

  29. Farid, M.S., Lucenteforte, M., Grangetto, M.: Edge enhancement of depth based rendered images. In: Proceedings of International Conference on Image Processing, pp. 5452–5456 (2014)

  30. Leon, G., Kalva, H., Furht, B.: 3D video quality evaluation with depth quality variations. In: Proceedings of 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, pp. 301–304 (2008)

  31. Farid, M.S., Lucenteforte, M., Grangetto, M.: Edges shape enforcement for visual enhancement of depth image based rendering. In: Proceedings of International Workshop on Multimedia Signal Processing, pp. 406–411 (2013)

  32. Bosc, E., Pepion, R., Le Callet, P., Koppel, M., Ndjiki-Nya, P., Pressigout, M., Morin, L.: Towards a new quality metric for 3-D synthesized view assessment. IEEE J. Sel. Topics Signal Process. 5(7), 1332–1343 (2011)

    Article  Google Scholar 

  33. You, J., Xing, L., Perkis, A., Wang, X.: Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. In: Proceedings of International Workshop Video Processing Quality Metrics for Consumer and Electronics (2010)

  34. Farid, M.S., Lucenteforte, M., Grangetto, M.: Blind depth quality assessment using histogram shape analysis. In: Proceedings of 3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video, pp. 1–5 (2015)

  35. Zhang, H., Huang, Y., Chen, X., Deng, D.: MLSIM: a multi-level similarity index for image quality assessment. Signal Process. Image Commun. 28(10), 1464–1477 (2013)

    Article  Google Scholar 

  36. Aflaki, P., Hannuksela, M.M., Gabbouj, M.: Subjective quality assessment of asymmetric stereoscopic 3D video. Signal Image Video Process. 9(2), 331–345 (2015)

    Article  Google Scholar 

  37. Solh, M., AlRegib, G., Bauza, J.M.: 3VQM: a vision-based quality measure for DIBR-based 3D videos. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 1–6 (2011)

  38. Ekmekcioglu, E., Worrall, S., De Silva, D., Fernando, A., Kondoz, A.M.: Depth Based Perceptual Quality Assessment for Synthesised Camera Viewpoints, pp. 76–83. Springer, Berlin (2012)

    Google Scholar 

  39. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  40. Benoit, A., Le Callet, P., Campisi, P., Cousseau, R.: Quality assessment of stereoscopic images. EURASIP J. Image Video Process. 2008, 659024 (2009)

    Google Scholar 

  41. Kim, S., Pak, D., Lee, S.: SSIM-based distortion metric for film grain noise in HEVC. Signal Image Video Process. 12(3), 489–496 (2018)

    Article  Google Scholar 

  42. Jang, W.D., Chung, T.Y., Sim, J.Y., Kim, C.S.: FDQM: fast quality metric for depth maps without view synthesis. IEEE Trans. Circuits Syst. Video Technol. 25(7), 1099–1112 (2015)

    Article  Google Scholar 

  43. Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. In: Vaina, L.M. (ed.) Matters of Intelligence, Volume 188 of Synthese Library, pp. 115–141. Springer, Netherlands (1987)

  44. Wang, J., DaSilva, M.P., LeCallet, P., Ricordel, V.: Computational model of stereoscopic 3D visual saliency. IEEE Trans. Image Process. 22(6), 2151–2165 (2013)

    Article  MathSciNet  Google Scholar 

  45. Fang, Y., Wang, J., Narwaria, M., Le Callet, P., Lin, W.: Saliency detection for stereoscopic images. IEEE Trans. Image Process. 23(6), 2625–2636 (2014)

    Article  MathSciNet  Google Scholar 

  46. VQEG. RRNR-TV Group Test Plan (2007). Version 2.2

  47. Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Punjab University College of Information Technology, University of the Punjab, Lahore, Pakistan

    Muhammad Shahid Farid

  2. Dipartimento di Informatica, Università degli Studi di Torino, Turin, Italy

    Maurizio Lucenteforte & Marco Grangetto

Authors
  1. Muhammad Shahid Farid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurizio Lucenteforte
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Grangetto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Shahid Farid.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farid, M.S., Lucenteforte, M. & Grangetto, M. No-reference quality metric for HEVC compression distortion estimation in depth maps. SIViP 14, 195–203 (2020). https://doi.org/10.1007/s11760-019-01542-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01542-0

Keywords

Search

Navigation