Advertisement

Perceptual quality assessment of stereoscopic images based on local and global visual characteristics

  • Lei Chen
  • Jiying Zhao
Article

Abstract

The quality assessment of stereoscopic images has attracted considerable attention and become an important issue in 3D multimedia applications. The 3D image quality assessment (IQA) encounters many challenges and simple extension of the 2D quality metrics to the 3D case is not satisfying. In this paper, we propose a new perceptual quality assessment scheme for stereoscopic 3D images by considering the local and global visual characteristics. The design of this scheme is motivated by studies on the perception of distorted stereoscopic images. To be more specific, after the log-Gabor filter processing, the local amplitude and phase from the left and right views of the reference and distorted 3D images are utilized as features in local quality evaluation. Meanwhile, the global structure changes of the left and right views are also incorporated into the final quality pooling. The overall 3D quality score is obtained by combining the local and global quality indexes together. The effectiveness of the designed metric is verified on publicly available 3D image quality assessment databases. Experimental results show that the proposed scheme exhibits better performance than other related algorithms in terms of consistency with subjective assessment of stereoscopic 3D images.

Keywords

Quality assessment Stereoscopic images Local amplitude and phase Global structure change 

Notes

References

  1. 1.
    Benoit A, Callet PL, Campisi P, Cousseau R (2008) Using disparity for quality assessment of stereoscopic images. In: Proceedings of IEEE international conference on image processing, San Diego, pp 389–392Google Scholar
  2. 2.
    Boev A, Gotchev A, Egiazarian K, Aksay A, Akar GB (2006) Toward compound stereo-video quality metric: a specific encoder-based framework. In: Proceedings of the IEEE southwest symposium on image analysis and interpretation, Denver, pp 218–222Google Scholar
  3. 3.
    Bosse S, Maniry D, Muller KR, Wiegand T, Samek W (2018) Deep neural networks for no-reference and full-reference image quality assessment. IEEE Image Process 27(1):206–219MathSciNetCrossRefGoogle Scholar
  4. 4.
    Cao Y, Hong W, Yu L (2016) Full-reference perceptual quality assessment for stereoscopic images based on primary visual processing mechanism. In: Proceedings of the IEEE international conference on multimedia and expo, Seattle, pp 1–6Google Scholar
  5. 5.
    Chen L, Zhao J (2017) Quality assessment of stereoscopic 3D images based on local and global visual characteristics. In: Proceedings of the IEEE international conference on multimedia and expo, Hong Kong, pp 61–66Google Scholar
  6. 6.
    Chen L, Zhao J (2017) Robust contourlet-based blind watermarking for depth-image-based rendering 3D images. Signal Process: Image Commun 54:56–65Google Scholar
  7. 7.
    Chen MJ, Cormack LK, Bovik AC (2013) No-reference quality assessment of natural stereopairs. IEEE Trans Image Process 22(9):3379–3391MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chen MJ, Su CC, Kwon DK, Cormack LK, Bovik AC (2013) Full-reference quality assessment of stereopairs accounting for rivalry. Signal Process: Image Commun 28(9):1143–1155Google Scholar
  9. 9.
    Chen Z, Lin J, Liao N, Chen CW (2017) Full reference quality assessment for image retargeting based on natural scene statistics modeling and bi-directional saliency similarity. IEEE Trans Image Process 26(11):5138–5148MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182CrossRefGoogle Scholar
  11. 11.
    Fan Y, Larabi MC, Cheikh FA (2017) Full-reference stereoscopic image quality assessment accounting for binocular combination and disparity information. In: Proceedings of the IEEE international conference on image processing, Beijing, pp 760–764Google Scholar
  12. 12.
    Fang Y, Ma K, Wang Z, Lin W, Fang Z, Zhai G (2015) No-reference quality assessment of contrast-distorted images based on natural scene statistics. IEEE Signal Process Lett 22(7):838–842Google Scholar
  13. 13.
    Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. J Opt Soc Amer A 4(12):2379–2394CrossRefGoogle Scholar
  14. 14.
    Gorley P, Holliman N (2008) Stereoscopic image quality metrics and compression. In: Proceedings of SPIE, vol 6803Google Scholar
  15. 15.
    Gottschalk PG, Dunn JR (2005) The five-parameter logistic: a characterization and comparison with four-parameter logistic. Anal Biochem 343(1):54–65CrossRefGoogle Scholar
  16. 16.
    Hewage CTER, Worrall ST, Dogan S, Villette S, Knodoz AM (2009) Quality evaluation of color plus depth map-based stereoscopic video. IEEE J Sel Topics Signal Process 3(2):304–318CrossRefGoogle Scholar
  17. 17.
    Howard IP, Rogers BJ (1995) Binocular fusion and rivalry in binocular vision and stereopsis. Oxford University Press, New YorkGoogle Scholar
  18. 18.
    Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Columbus, pp 1733–1740Google Scholar
  19. 19.
    Kolmogorov V, Zabih R (2001) Computing visual correspondence with occlusions using graph cuts. In: Proceedings of the IEEE international conference on computer vision, Vancouver, pp 508–515Google Scholar
  20. 20.
    Lebreton P, Raake A, Barkowsky M, PLe Callet (2012) Evaluating depth perception of 3D stereoscopic videos. IEEE J Select Topics Signal Process 6(6):710–720CrossRefGoogle Scholar
  21. 21.
    Lee K, Lee S (2015) 3D perception based quality pooling: stereopsis, binocular rivalry, and binocular suppression. IEEE J Sel Topics Signal Process 9(3):533–545CrossRefGoogle Scholar
  22. 22.
    Lin YH, Wu JL (2014) Quality assessment of stereoscopic 3D image compression by binocular integration behaviors. IEEE Trans Image Process 23(4):1527–1542MathSciNetCrossRefGoogle Scholar
  23. 23.
    Liu Y, Yang J, Meng Q, Lv Z, Song Z, Gao Z (2016) Stereoscopic image quality assessment method based on binocular combination saliency model. Signal Process 125:237–248CrossRefGoogle Scholar
  24. 24.
    Maalouf A, Larabi MC (2011) CYCLOP: stereo color image quality assessment metric. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing, Prague, pp 1161–1164Google Scholar
  25. 25.
    Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708MathSciNetCrossRefGoogle Scholar
  26. 26.
    Moorthy AK, Su CC, Mittal A, Bovik AC (2012) Subjective evaluation of stereoscopic image quality. Signal Process: Image Commun 28(8):870–883Google Scholar
  27. 27.
    Qi F, Zhao D, Jiang T, Ma S (2012) Quality of experience assessment for stereoscopic images. In: Proceedings IEEE international symposium on circuits and systems, Seoul, pp 1712–1715Google Scholar
  28. 28.
    Rehman A, Wang Z (2012) Reduced-reference image quality assessment by structural similarity estimation. IEEE Trans Image Process 21(8):3378–3389MathSciNetCrossRefGoogle Scholar
  29. 29.
    Ryu S, Sohn K (2014) No-reference quality assessment for stereoscopic images based on binocular quality perception. IEEE Trans Circ Syst Video Technol 24(4):591–602CrossRefGoogle Scholar
  30. 30.
    Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the DCT doamin. IEEE Trans Image Process 21(8):3339–3352MathSciNetCrossRefGoogle Scholar
  31. 31.
    Sazzad ZMP, Yamanaka S, Kawayoke Y, Horita Y (2009) Stereoscopic image quality prediction. In: Proceedings of the IEEE international conference on quality of multimedia experience, San Diego, pp 180–185Google Scholar
  32. 32.
    Seuntiens P, Meesters L, Ijsselsteijn W (2006) Perceived quality of compressed stereoscopic images: effect of symmetric and asymmetric JPEG coding and camera separation. ACM Trans Appl Percept 3(2):95–109CrossRefGoogle Scholar
  33. 33.
    Shao F, Tian W, Lin W, Jiang G, Dai Q (2013) Perceptual full-reference quality assessment of stereoscopic images by considering binocular visual characteristics. IEEE Trans Image Process 22(5):1940–1953MathSciNetCrossRefGoogle Scholar
  34. 34.
    Shao F, Li K, Lin W, Jiang G, Yu M, Dai Q (2015) Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties. IEEE Trans Image Process 24(10):2971–2983MathSciNetCrossRefGoogle Scholar
  35. 35.
    Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444CrossRefGoogle Scholar
  36. 36.
    Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evalution of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11):3440–3451CrossRefGoogle Scholar
  37. 37.
    Song R, Ko H, Kuo CCJ (2015) MCL-3D: a database for stereoscopic image quality assessment using 2D-image-plus-depth source. J Inf Sci Eng 31(5):1593–1611Google Scholar
  38. 38.
    Tam WJ, Speranza F, Yano S, Shimono K, Ono H (2011) Stereoscopic 3D-TV: visual comfort. IEEE Trans Broadcast 57(2):335–346CrossRefGoogle Scholar
  39. 39.
    Thomson MGA, Foster DH, Summers RJ (2000) Human sensitivity to phase perturbations in natural images: a statistical framework. Perception 29(9):1057–1070CrossRefGoogle Scholar
  40. 40.
    Tong F, Nakayama K, Vaughan JT, Kanwisher N (1998) Binocular rivalry and visual awareness in human extrastriate cortex. Neuron 21(4):753–759CrossRefGoogle Scholar
  41. 41.
    Wang Z, Simoncelli EP, Bovik AC (2003) Multi-scale structural similarity for image quality assessment. In: Proceedings of the IEEE Asilomar conference on signals, systems, and computers, vol 1, Pacific Grove, pp 1398–1402Google Scholar
  42. 42.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612CrossRefGoogle Scholar
  43. 43.
    Wang X, Kwong S, Zhang Y (2011) Considering binocular spatial sensitivity in stereoscopic image quality assessment. In: Proceedings of the IEEE visual communications and image processing, Taiwan, pp 1–4Google Scholar
  44. 44.
    Wang S, Zheng D, Zhao J, Tam WJ, Speranaza F (2014) Adaptive watermarking and tree structure based image quality estimation. IEEE Trans Multimed 16(2):311–325CrossRefGoogle Scholar
  45. 45.
    Wang J, Rehman A, Zeng K, Wang S, Wang Z (2015) Quality prediction of asymmetrically distorted stereoscopic 3D images. IEEE Trans Image Process 24 (11):3400–3414MathSciNetCrossRefGoogle Scholar
  46. 46.
    Wu J, Lin W, Shi G, Liu A (2013) Reduced-reference image quality assessment with visual information fidelity. IEEE Trans Multimed 15(7):1700–1705CrossRefGoogle Scholar
  47. 47.
    Yang J, Hou C, Zhou Y, Zhang Z, Guo J (2009) Objective quality assessment method of stereo images. In: Proceedings of the 3D TV conference true vision?capture, transmission and display 3D video, Potsdam, pp 1–4Google Scholar
  48. 48.
    Yasakethu SLP, Hewage CTER, Fernado WAC, Kondoz AK (2008) Quality analysis for 3D video using 2D video quality models. IEEE Trans Consum Electron 54(4):1969–1976CrossRefGoogle Scholar
  49. 49.
    You J, Xing L, Perkis A, Wang X (2010) Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. In: Proceedings of IEEE international workshop on video processing and quality metrics for consumer electronics, Scottsdale, pp 61–66Google Scholar
  50. 50.
    Zhang L, Tam WJ (2005) Stereoscopic image generation based on depth images for 3D TV. IEEE Trans Broadcast 51(2):191–199CrossRefGoogle Scholar
  51. 51.
    Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Electrical Engineering and Computer ScienceUniversity of OttawaOttawaCanada

Personalised recommendations