Skip to main content
SpringerLink
Log in

An efficient human visual system based quality metric for 3D video

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Stereoscopic video technologies have been introduced to the consumer market in the past few years. A key factor in designing a 3D system is to understand how different visual cues and distortions affect the perceptual quality of stereoscopic video. The ultimate way to assess 3D video quality is through subjective tests. However, subjective evaluation is time consuming, expensive, and in some cases not possible. The other solution is developing objective quality metrics, which attempt to model the Human Visual System (HVS) in order to assess perceptual quality. Although several 2D quality metrics have been proposed for still images and videos, in the case of 3D efforts are only at the initial stages. In this paper, we propose a new full-reference quality metric for 3D content. Our method mimics HVS by fusing information of both the left and right views to construct the cyclopean view, as well as taking to account the sensitivity of HVS to contrast and the disparity of the views. In addition, a temporal pooling strategy is utilized to address the effect of temporal variations of the quality in the video. Performance evaluations showed that our 3D quality metric quantifies quality degradation caused by several representative types of distortions very accurately, with Pearson correlation coefficient of 90.8 %, a competitive performance compared to the state-of-the-art 3D quality metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. “View synthesis reference software (VSRS) 3.5,” wg11.sc29.org, March 2010

  2. Banitalebi-Dehkordi A, Pourazad MT, Nasiopoulos P (2013) 3D video quality metric for 3D video compression. 11th IEEE IVMSP Workshop: 3D Image/Video Technologies and Applications, Seoul

    Book  Google Scholar 

  3. Banitalebi-Dehkordi A, Pourazad MT, Nasiopoulos P (2013) 3D video quality metric for mobile applications. 38th International Conference on Acoustic, Speech, and Signal Processing, ICASSP, Vancouver

    Book  Google Scholar 

  4. Banitalebi-Dehkordi A, Pourazad MT, Nasiopoulos P (2013) A study on the relationship between the depth map quality and the overall 3D video quality of experience. International 3DTV Conference: Vision Beyond Depth, Scotland

    Book  Google Scholar 

  5. Banitalebi-Dehkordi A, Pourazad MT, and Nasiopoulos P (2014) “Effect of eye dominance on the perception of stereoscopic 3D video,” International Conference on Image Processing, ICIP

  6. Banitalebi-Dehkordi A, Pourazad MT, Nasiopoulos P (January 2013) “A human visual system based 3D video quality metric,” ISO/IEC JTC1/SC29/WG11, Doc. M27745, Geneva, Switzerland

  7. Barten PGJ (1999) “Contrast sensitivity of the human eye and its effects on image quality”, SPIE Press

  8. Benoit A, Le. Callet P, Campisi P, Cousseau R (2008) “Quality assessment of stereoscopic images,” EURASIP Journal on Image and Video Processing, special issue on 3D Image and Video Processing, vol. 2008, Article ID 659024

  9. Boev A, Gotchev A, Egiazarian K, Aksay A, Akar GB (June 2006) “Towards compound stereo-video quality metric: a specific encoder-based framework,” IEEE Southwest Symposium on Image Analysis and Interpretation, Denver, USA, pp. 218–222

  10. Boev A, Hollosi D, Gotchev A, Egiazarian K (Jan 2009) “Classification and simulation of stereoscopic artefacts in mobile 3DTV content,” Electronic Imaging Symposium 2009, San Jose, USA

  11. Boev A, Poikela M, Gotchev A, Aksay A (July 2010) “Modeling of the stereoscopic HVS,” Mobile 3DTV project report, available: http://sp.cs.tut.fi/mobile3dtv/results/

  12. Chen M, Cormack LK, Bovik AC (September 2013) “No-reference quality assessment of natural stereopairs,” IEEE Transactions on Image Processing, vol. 22, no. 9

  13. Chen W, Fournier J, Barkowsky M, Callet PL (2010) “New requirements of subjective video quality assessment methodologies for 3DTV,” in fifth International Workshop on Video Processing and Quality Metrics for Consumer Electronics, VPQM, USA

  14. Chen W, Fournier J, Barkowsky M, Le Callet P (2012) Quality of experience model for 3DTV. SPIE Conference on Stereoscopic Displays and Applications, San Francisco

    Book  Google Scholar 

  15. Chen MJ, Su CC, Kwon DK, Cormack LK, Bovik AC (2012) “Full-reference quality assessment of stereopairs accounting for rivalry,” 46th Annual Asimolar Conference on Signals, Systems, and Computers, United States

  16. Chen M, Su C, Kwon D, Cormack L, Bovik AC (2013) Full-reference quality assessment of stereopairs accounting for rivalry. Signal Process Image Commun 28:1143–1155

    Article  Google Scholar 

  17. Coria L, Xu D, Nasiopoulos P (January 2011) “Quality of experience of stereoscopic content on displays of different sizes: a comprehensive subjective evaluation,” IEEE International Conference on Consumer Electronics, ICCE, pp. 778–779, Las Vegas, NV, USA

  18. Coria L, Xu D, Nasiopoulos P (Oct. 2012) “Automatic stereoscopic video reframing,” 3DTV-Con: The True Vision - Capture, Transmission and Display of 3D Video

  19. 3D video database at Digital Multimedia Lab, University of British Columbia, available at: http://dml.ece.ubc.ca/data/hv3d/

  20. Egiazarian K, Astola J, Ponomarenko N, Lukin V, Battisti F, Carli M (2006) Two new full reference quality metrics based on HVS. International Workshop on Video Processing and Quality metrics, Scottsdale

    Google Scholar 

  21. Ha K, Kim M (2010) “A perceptual quality assessment metric using temporal complexity and disparity information for stereoscopic video,” 18th International Conference on Image Processing, ICIP

  22. Hamberg and H. de Ridder (1999) “Time-varying image quality: modeling the relation between instantaneous and overall quality,” Society of Motion Picture & Television Engineers (SMTPE) Journal, Vol. 108, pp. 802–811

  23. Hanhart P, De Simone F, Ebrahimi T (2012) “Quality assessment of asymmetric stereo pair formed from decoded and synthesized views,” In Quality of Multimedia Experience (QoMEX), 2012 Fourth International Workshop On, pp. 236–241

  24. HEVC at HHI Fraunhofer: https://hevc.hhi.fraunhofer.de/

  25. Hyunh-Thu Q, Callet PL, Barkowsky M (2010)“Video quality assessment: from 2D to 3D challenges and future trends,” IEEE 17th International Conference on Image Processing, (ICIP), pp. 4025–4028

  26. Irwin DE (1992) “Visual memory within and across fixations,” Eye movements and Visual Cognition: Scene Preparation and Reading, 146–165

  27. ISO/IEC JTC1/SC29/WG11 (January 2013) “3D-HEVC Test Model 3,” N13348, Geneva

  28. ISO/IEC JTC1/SC29/WG11 (MPEG) (March 2011) Document N12036, “Call for proposals on 3D video coding technology,” 96th MPEG meeting, Geneva

  29. ISO/IEC JTC1/SC29/WG11 (November 2011) “Common test conditions for HEVC- and AVC-based 3DV”, Doc. N12352, Switzerland

  30. Jin L, Boev A, Pyykko SJ, Haustola T, Gotchev A “Novel stereo video quality metric,” Mobile 3DTV project report, available: http://sp.cs.tut.fi/mobile3dtv/results/

  31. Jin L, Boev A, Gotchev A, Egiazarian K (2011) “3D-DCT based perceptual quality assessment of stereo video,” 18th International Conference on Image Processing, ICIP

  32. Jin L, Boev A, Gotchev A, Egiazarian K (Sept. 2011) “Validation of a new full reference metric for quality assessment of mobile 3DTV content,” in 19th European Signal Processing Conference (EUROSIPCO 2011)

  33. Kauff P, Atzpadin N, Fehn C, Müller M, Schreer O, Smolic A, Tanger R (2007) Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Signal Process Image Commun 22(2):217–234

    Article  Google Scholar 

  34. Lee K, Park J, Lee S, Bovik AC (2010) Temporal pooling of video quality estimates using perceptual motion models. IEEE 17th International on Image Processing, ICIP, Hong Kong

    Book  Google Scholar 

  35. Mai Z, Pourazad MT, Nasiopoulos P (Sept 2012) “Effect of displayed brightness on 3D video viewing”, IEEE International Conference on Consumer Electronics, ICCE, pp. 1–3, Berlin

  36. Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322

    Article  Google Scholar 

  37. Pourazad MT, Doutre C, Azimi M, Nasiopoulos P (2012) HEVC: The new gold standard for video compression: how does HEVC compare with H.264/AVC. IEEE Consum Electron Mag 1(3):36–46

    Article  Google Scholar 

  38. Recommendation ITU P.910 (1999) “Subjective video quality assessment methods for multimedia applications,” ITU

  39. Recommendation ITU-R BT.500-13 (2012) “Methodology for the subjective assessment of the quality of the television pictures”

  40. Recommendation ITU-TJ.149 (2004) “Method for specifiying accuracy and cross-calibration of video quality metrics (VQM)”

  41. Rimac-Drlje S, Vranjes M, Zagar D (2009) “Influence of temporal pooling method on the objective video quality evaluation,” in Broadband Multimedia Systems and Broadcasting, BMSB

  42. Ryu S, Sohn K (April 2014) “No-reference quality assessment for stereoscopic images based on binocular qulaity perception,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no. 4

  43. Seshadrinathan K, Bovik AC (2011) “Temporal hysteresis model of time varying subjective video quality,” IEEE International Conference on Acoustic, Speech, and Signal Processing, ICASSP

  44. Seshadrinathan K, Bovik AC (Febuary 2010) “Motion tuned spatio-temporal quality assessment of natural videos,” IEEE Transactions on Image Processing, vol. 19, no. 2

  45. Seuntiens P (2006) “Visual experience of 3D TV,” Doctoral thesis, Eindhoven University of Technology

  46. Shao F, Lin W, Gu SH, Jiang G, Srikanthan T (May 2013) “Perceptual full-reference quality assessment of stereoscopic images by considering binocular visual characteristics,” IEEE Transactions on Image Processing, vol. 22, no. 5

  47. Sheikh HR, Bovic AC (Feb. 2006) “Image information and visual quality”, IEEE Transactions on Image Processing, Vol. 15, NO. 2

  48. Sobel I (2014) “History and definition of the Sobel operator”

  49. Sullivan GJ, Ohm J, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circ Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  50. Tanimoto M, Fujii T, Suzuki K (2009)” Video depth estimation reference software (DERS) with image segmentation and block matching,” ISO/IEC JTC1/SC29/WG11 MPEG/M16092, Switzerland

  51. Wang Z, Bovik AC (2009) Mean squared error: love it or leave it? IEEE Signal Process Mag 26(1):98–117

    Article  Google Scholar 

  52. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Article  Google Scholar 

  53. Wang J, Silva MPD, Callet PL, Ricordel V (June 2013) “Computational model of stereoscopic 3D visual saliency,” IEEE Transactions on Image Processing, vol. 22, no. 6

  54. Xu D, Coria LE, Nasiopoulos P (2012) Guidelines for an improved quality of experience in 3D TV and 3D mobile displays. J Soc Inf Disp 20(7):397–407. doi:10.1002/jsid.99

    Article  Google Scholar 

  55. You J, Xing L, Perkis A, Wang X (2010) Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. International Workshop on Video Processing and Quality Metrics for Consumer Electronics, VPQM, Arizona

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ICICS, University of British Columbia and TELUS Communications Inc, Vancouver, BC, Canada

    Mahsa T. Pourazad

  2. Electrical Engineering Department and ICICS, University of British Columbia, Vancouver, BC, Canada

    Amin Banitalebi-Dehkordi & Panos Nasiopoulos

Authors
  1. Amin Banitalebi-Dehkordi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahsa T. Pourazad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Panos Nasiopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amin Banitalebi-Dehkordi.

Additional information

This work was partly supported by Natural Sciences and Engineering Research Council of Canada (NSERC) under Grant STPGP 447339-13 and the Institute for Computing Information and Cognitive Systems (ICICS) at UBC.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Banitalebi-Dehkordi, A., Pourazad, M.T. & Nasiopoulos, P. An efficient human visual system based quality metric for 3D video. Multimed Tools Appl 75, 4187–4215 (2016). https://doi.org/10.1007/s11042-015-2466-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2466-z

Keywords

Search

Navigation