Skip to main content
SpringerLink
Log in

Using Saliency-Weighted Disparity Statistics for Objective Visual Comfort Assessment of Stereoscopic Images

  • 3DR Express
  • Published:
3D Research

Abstract

Visual comfort assessment (VCA) for stereoscopic images is a particularly significant yet challenging task in 3D quality of experience research field. Although the subjective assessment given by human observers is known as the most reliable way to evaluate the experienced visual discomfort, it is time-consuming and non-systematic. Therefore, it is of great importance to develop objective VCA approaches that can faithfully predict the degree of visual discomfort as human beings do. In this paper, a novel two-stage objective VCA framework is proposed. The main contribution of this study is that the important visual attention mechanism of human visual system is incorporated for visual comfort-aware feature extraction. Specifically, in the first stage, we first construct an adaptive 3D visual saliency detection model to derive saliency map of a stereoscopic image, and then a set of saliency-weighted disparity statistics are computed and combined to form a single feature vector to represent a stereoscopic image in terms of visual comfort. In the second stage, a high dimensional feature vector is fused into a single visual comfort score by performing random forest algorithm. Experimental results on two benchmark databases confirm the superior performance of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Hur, N., Lee, H., Lee, G., Lee, S., Gotchev, A., & Park, S. (2011). 3DTV broadcasting and distribution systems. IEEE Transactions on Broadcasting, 57(2), 395–407.

    Article  Google Scholar 

  2. Lambooij, M., Ijsselsteijn, W., Fortuin, M., & Heynderickx, I. (2009). Visual discomfort and visual fatigue of stereoscopic displays: A review. Journal of Imaging Science and Technology, 53(3), 1–14.

    Article  Google Scholar 

  3. Tam, W., Speranza, F., Yano, S., Shimono, K., & Ono, H. (2011). Stereoscopic 3D-TV: Visual comfort. IEEE Transactions on Broadcasting, 57(2), 335–346.

    Article  Google Scholar 

  4. Urvoy, M., Barkowsky, M., & Le Callet, P. (2013). How visual fatigue and discomfort impact 3D-TV quality of experience: a comprehensive review of technological, psychophysical, and psychological factors. Annals of Telecommunications-Annales Des Télécommunications, 68(11–12), 641–655.

    Article  Google Scholar 

  5. A. Mittal, A. Moorthy, J. Ghosh, and A.C. Bovik (2011) Algorithmic assessment of 3D quality of experience for images and videos, Proceedings of IEEE Digital Signal Processing Workshop, pp. 338–343

  6. Lambooij, M., IJsselsteijn, W., & Heynderickx, I. (2011). “Visual discomfort of 3D TV: Assessment methods and modeling”. Displays, 32(4), 209–218.

    Article  Google Scholar 

  7. Ukai, K., & Howarth, P. (2008). Visual fatigue caused by viewing stereoscopic motion images: Background, theories, and observations. Displays, 29(2), 106–116.

    Article  Google Scholar 

  8. Yano, S., Ide, S., Mitsuhashi, T., & Thwaites, H. (2002). A study of visual fatigue and visual comfort for 3D HDTV/HDTV images. Displays, 23(4), 191–201.

    Article  Google Scholar 

  9. Kim, D., & Sohn, K. (2011). Visual fatigue prediction for stereoscopic image. IEEE Transactions on Circuits System and Video Technology, 21(2), 231–236.

    Article  Google Scholar 

  10. Choi, J., Kim, D., Choi, S., & Sohn, K. (2010). Visual fatigue modeling and analysis for stereoscopic video. Optical Engineering, 51(1), 017206.

    Article  Google Scholar 

  11. Jiang, Q., Shao, F., Jiang, G., Yu, M., & Peng, Z. (2015). Three dimensional visual comfort assessment via preference learning. Journal of Electronic Imaging, 24(4), 043002.

    Article  Google Scholar 

  12. Borji, A., & Itti, L. (2013). State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 185–207.

    Article  MathSciNet  Google Scholar 

  13. Sohn, H., Jung, Y., Lee, S., & Ro, Y. (2013). Predicting visual discomfort using object size and disparity information in stereoscopic images. IEEE Transactions on Broadcasting, 59(1), 28–37.

    Article  Google Scholar 

  14. Lee, S., Jung, Y., Sohn, H., Speranza, F., & Ro, Y. (2013). Effect of stimulus width on the perceived visual discomfort in viewing stereoscopic 3D-TV. IEEE Transactions on Broadcasting, 59(4), 580–590.

    Article  Google Scholar 

  15. Jung, Y., Sohn, H., Lee, S., Park, H., & Ro, Y. (2013). Predicting visual discomfort of stereoscopic images using human attention model. IEEE Transactions on Circuits System and Video Technology, 23(12), 2077–2082.

    Article  Google Scholar 

  16. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    Article  MathSciNet  MATH  Google Scholar 

  17. Gu, K., Zhai, G., Yang, X., & Zhang, W. (2015). Using free energy principle for blind image quality assessment. IEEE Transactions on Multimedia, 17(1), 50–63.

    Article  Google Scholar 

  18. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.

    Article  Google Scholar 

  19. Harel, J., Koch, C., Perona, P. (2006) Graph-based visual saliency, Proceedings of Advances in Neural Information Processing Systems.

  20. Goferman, S., Zelnik-Manor, L., & Tal, A. (2012). Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10), 1915–1926.

    Article  Google Scholar 

  21. Hou, X., Harel, J., & Koch, C. (2012). Image signature: Highlighting sparse salient regions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1), 194–201.

    Article  Google Scholar 

  22. Li, J., Levine, M., An, X., Xu, X., & He, H. (2013). Visual saliency based on scale-space analysis in the frequency domain. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4), 996–1010.

    Article  Google Scholar 

  23. Martinel, N., Micheloni, C., & Foresti, G. L. (2015). Kernelized saliency-based person re-identification through multiple metric learning. IEEE Transactions on Image Processing, 24(12), 5645–5658.

    Article  MathSciNet  Google Scholar 

  24. Lang, C., Nguyen, T., Katti, H., Yadati, K., Kankanhalli, M., Yan, S. (2012). Depth matters: Influence of depth cues on visual saliency. In Proceeding of 12th European Conference on Computer Vision (ECCV).

  25. Park, J., Lee, S., & Bovik, A. (2014). 3D visual discomfort prediction: Vergence, foveation, and the physiological optics of accommodation. IEEE Journal of Selected Topics in Signal Processing, 8(3), 415–426.

    Article  Google Scholar 

  26. Final report from the video quality experts group on the validation of objective models of video quality assessment VQEG, 2000. http://www.vqeg.org

Download references

Acknowledgments

This work was supported in part by Natural Science Foundation of China (Grant No. 61501270), in part by Zhejiang Provincial Natural Science Foundation of China (Grant No. LY14F010004), in part by Open fund of Zhejiang Provincial Key Academic Project (first level), in part by College Students Science and Technology Innovation Project (Xin Miao Talent Project) of Zhejiang Province (2014R405077), in part by Ningbo Natural Science Foundation (2016A610071), and in part by the Scientific Research Foundation of Ningbo University (XYL15025).

Author information

Authors and Affiliations

  1. College of Science and Technology, Ningbo University, Ningbo, 315211, China

    Wenlan Zhang, Ting Luo & Jing Lu

  2. Faculty of Information Science and Engineering, Ningbo University, Ningbo, 315211, China

    Ting Luo, Gangyi Jiang, Qiuping Jiang & Hongwei Ying

Authors
  1. Wenlan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ting Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gangyi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiuping Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongwei Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jing Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ting Luo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Luo, T., Jiang, G. et al. Using Saliency-Weighted Disparity Statistics for Objective Visual Comfort Assessment of Stereoscopic Images. 3D Res 7, 17 (2016). https://doi.org/10.1007/s13319-016-0079-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13319-016-0079-6

Keywords

Search

Navigation