A perceptual metric for stereoscopic image quality assessment based on the binocular energy

Article

Abstract

Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.

Keywords

Stereoscopic quality assessment Binocular energy 3D perception Simple and complex cells 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adelson E. H., Bergen J. R. (1985) Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America 2(2): 284–299CrossRefGoogle Scholar
  2. Akhter, R., Sazzad, Z. M. P., Horita, Y., & Baltes, J. (2010). No reference stereoscopic image quality assessment. In Image quality and system performance (vol. 7524, pp. 17–21). San Jose, California, USA.Google Scholar
  3. Ates H. F., Orchard M. T. (2003) A nonlinear image representation in wavelet domain using complex signals with single quadrant spectrum. Asilomar Conference on Signals, Systems, Computers 2: 1966–1970Google Scholar
  4. Avcibas I., Sankur B., Sayood K. (2002) Statistical evaluation of image quality measures. Journal of Electronic Imaging 11(2): 206–223CrossRefGoogle Scholar
  5. Barlow H. B., Blakemore C., Pettigrew J. D. (1967) The neural mechanism of binocular depth discrimination. Journal of Physiology 193: 327–342Google Scholar
  6. Bensalma, R., & Larabi, M. C. (2010). Stereo image coding based on binocular energy modeling. In International conference image processing (ICIP) (pp. 2989–2992).Google Scholar
  7. Bensalma, R., & Larabi, M. C. (2010). Towards a perceptual quality metric for color stereo images. In International conference image processing (ICIP) (pp. 4037–4040). Hong Kong.Google Scholar
  8. Blake R., Wilson H. R. (1991) Neural models of stereoscopic vision. Trends in neurosciences. International Journal of Computer Vision 14: 445–452Google Scholar
  9. Boev, A., Gotchev, A., Egiazarian, K. O., Aksay, A., & Akar, G. B. (2010). Towards compound stereo-video quality metric: A specific encoder-based framework. In IEEE SSIAI (pp. 218–222). Denver, Colorado, USA.Google Scholar
  10. Campbell F. W., Cooper G. F., Enroth-Cugell C. (1969) The spatial selectivity of the visual cells of the cat. Journal of Physiology 203: 223–235Google Scholar
  11. Campisi, P., LeCallet, P., & Marini, E. (2007). Stereoscopic images quality assessment. In European signal processing conference. Poznan, Poland.Google Scholar
  12. Candès E., ans Demanet L., Donoho D., Ying L. (2006) Fast discrete curvelet transforms. Multiscale Modelling and Simulation 5(3): 861–899MATHCrossRefGoogle Scholar
  13. Cheng, I., & Boulanger, P. (2005). A 3D perceptual metric using just-noticeable-difference. In: In Eurographics Short Presentations (pp. 97–100).Google Scholar
  14. Damera-Venkata N., Kite T. D., Evans B. L., Geisler W. S., Bovik A. C. (2000) Image quality assessment based on a degradation model. IEEE Transaction Image Processing 4(4): 636–650CrossRefGoogle Scholar
  15. DeAngelis G. C., Ohzawa I., Freeman R. D. (1991) Depth is encoded in the visual cortex by a specialized receptive field structure. Journal on Nature 352: 156–195CrossRefGoogle Scholar
  16. Do, M., & Vetterli, M. (2001). Pyramidal directional filter banks and curvelets. IEEE proceedings on international conference image processing.Google Scholar
  17. Donghyun K., Dongbo M., Juhyun O., Kwanghoon S., Seonggyu J. (2009) Depth map quality metric for three-dimensional video. Image Quality and System Performance 7237(29): 723719Google Scholar
  18. Ellinas J. N., Sangriotis M. S. (2004) Stereo image compression using wavelet coefficients morphology. Image and Vision Computing 22(4): 281–290CrossRefGoogle Scholar
  19. Field D. J., Tolhurst D. J. (1986) The structure and symmetry of simple-cell receptive field profiles in the cat’s visual cortex. Proceedings of the Royal Society of London 228: 379–400CrossRefGoogle Scholar
  20. Fleet D. J., Wagner H., Heeger D. J. (1996) Neural encoding of binocular disparity: Energy model, position shifts and phase shifts. Vision Research 36(12): 1839–1857CrossRefGoogle Scholar
  21. Foster K.H., Gaska J. P., Marcelja S., Pollen D.A. (1983) Phase relationships between adjacent simple cells in the feline visual cortex. Journal of Physiology 345: 22Google Scholar
  22. Goldmann, L., & Ebrahimi, T. (2010) 3D quality is more than just the sum of 2D and depth. In IEEE International workshop on hot topics in 3D.Google Scholar
  23. Gorley, P., & Holliman, N. (2008). Stereoscopic image quality metrics and compression. In Image quality and system performance, vol. 6803 (pp. 1–11). San Jose, California, USA.Google Scholar
  24. Hewage, C., & Martini, M.G. (2010). Reduced-reference quality metric for 3D depth map transmission. In IEEE 3DTV conference (pp. 1–4). Tampere, Finland.Google Scholar
  25. Hubel D. H., Wiesel T. N. (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology 160: 106–154Google Scholar
  26. Hubel D. H., Wiesel T. N. (1970) Stereoscopic vision in macaque monkey. cells sensitive to binocular depth in area 18 of the macaque monkey cortex. Journal of Nature 225: 41–42CrossRefGoogle Scholar
  27. Jones J. P., Palmer L. A. (1987) An evaluation of the two-dimensional gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology 58(6): 1233–1258Google Scholar
  28. Kaptein, R. G., Kuijsters, A., Lambooij, M. T. M., IJsselsteijn, W. A., & Heynderickx, I. (2008). Performance evaluation of 3D-TV systems. In image quality and system performance (pp. 1–11). San Jose, California, USA.Google Scholar
  29. Keita, T., & Takeshi, N. (2005). Unstructured light field rendering using on-the-fiy focus measurements. In IEEE international conference on multimedia and expo.Google Scholar
  30. Kingsbury N. (1997) Image processing with complex wavelets. Philosophical Transactions on Royal Society London A 357: 2543–2560CrossRefGoogle Scholar
  31. Kuffler S. W. (1953) Discharge patterns and functional organization of mammalian retina. Journal of Physiology 16: 37–68Google Scholar
  32. Lavoué G., Gelasca E. D., Dupont F., Baskurt A., Ebrahimi T. (2006) Perceptually driven 3D distance metrics with application to watermarking. Image Quality and System Performance 6312(29): 63120LGoogle Scholar
  33. Le Pennec E., Mallat S. (2005) Bandelet image approximation and compression. SIAM Multiscale Modeling and Simulation 4(3): 992–1039MathSciNetMATHCrossRefGoogle Scholar
  34. Liu A., Gaska J. P., Jacobson L. D., Pollen D. A. (1992) Interneuronal interaction between members of quadrature phase and anti-phase pairs in the cat’s visual cortex. Vision Research 32: 1193–1198CrossRefGoogle Scholar
  35. Mallat S. (1989) A theory for multiresolution signal decomposition : the wavelet represntation. IEEE, PAMI 11(7): 674–693MATHCrossRefGoogle Scholar
  36. Mallat S., Peyré G. (2006) Orthogonal bandelet bases for geometric image approximation. Communications on Pure and Applied Mathematics 61(9): 1173–1212Google Scholar
  37. Media Information and Communication Technology (MICT) Lab-oratory. (2011). Mict image quality evaluation database. http://mict.eng.u-toyama.ac.jp/mict/index2.html. Accessed 24 Sept 2011.
  38. Meesters L. M. J., IJsselsteijn W. A., Seuntiens P. J. H. (2004) A survey of perceptual evaluations and requirements of three-dimensional TV. IEEE Transactions on Circuits and Systems for Video Technology 14(3): 381–391CrossRefGoogle Scholar
  39. Miyahara M., Kotani K., Algazi V. R. (1998) Objective picture quality scale (PQS) for image coding. IEEE Transaction Communications 46(9): 1215–1225CrossRefGoogle Scholar
  40. Nath S. K., Dubois E. (2006) An improved, wavelet-based, stereoscopic image sequence codec with SNR and spatial scalability. Signal processing Image communication 21(3): 181–199CrossRefGoogle Scholar
  41. Ohzawa I., Freeman R. D. (1986) The binocular organization of simple cells in the cat’s visual cortex. Journal of Neurophysiology 56: 221–242Google Scholar
  42. Ohzawa I., Freeman R. D. (1986) The binocular organization of complex cells in the cat’s visual cortex. Journal of Neurophysiology 56: 243–259Google Scholar
  43. Olsson, R., & Sjostrom, M. A. (2007). Depth dependent quality metric for evaluation of coded integral imaging based 3D-images. In IEEE 3DTV. Kos, Greece.Google Scholar
  44. Palmer L. A., Davis T. L. (1981) Receptive-field structure in cat striate cortex. Vision Research 46: 260–276Google Scholar
  45. Peyre, G. (2005). Geometrie multi-échelles pour les images et les textures. Ph.D. thesis, Ecole Polytechnique.Google Scholar
  46. Pollen D. A., Ronner S. (1981) Phase relationships between adjacent simple cells in the visual cortex. Science 212: 1409–1411CrossRefGoogle Scholar
  47. Rittermann, M. A. (2004). A proposal for the quality assessment of 3D video objects. In International workshop on image analysis for multimedia interactiveservices. Lisboa, Portugal.Google Scholar
  48. Sarnoff Corporation (2003) JND metrix technology. evaluation (2003) Version available http://www.sarnoff.com/productsservices/videovision/jndmetrix/downloads.asp.
  49. Schanda J. (2007) Colorimetry: Understanding the CIE System. Wiley, Hoboken, NJ, USAGoogle Scholar
  50. Scharstein D., Szeliski R. (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47(1/2/3): 7–42MATHCrossRefGoogle Scholar
  51. Scharstein, D., & Szeliski, R. (2002). Middlebury stereo vision page. vision middlebury data base (2002) Version available http://vision.middlebury.edu/stereo/.
  52. Selesnick I.W., Baraniuk R.G., Kingsbury N.G. (2005) The dual-tree complex wavelet transform. IEEE Signal Processing Magazine 22(6): 123–151CrossRefGoogle Scholar
  53. Sheikh H. R., Bovik A. C., deVeciana G. (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing 14(12): 2117–2128CrossRefGoogle Scholar
  54. Sheikh H. R., Bovik A. C. (2006) Image information and visual quality. IEEE Transactions on Image Processing 15(2): 430–444CrossRefGoogle Scholar
  55. Tikanmäki, A., Gotchev, A., Smolic, A., & Müller, K. (2008). Quality assessment of 3D video in rate allocation experiments. In IEEE international symposium on consumer electronics. Algarve, Portugal.Google Scholar
  56. VQEG (2008). Final report from the video quality experts group on the validation of objective models of multimedia quality assessement. Technical Report on PHASE I 2008, VQEG.Google Scholar
  57. Wang Z., Lu L., Bovik A. C. (2004) Video quality assessment based on structural distortion measurement. Signal Processing Image Communication 19(2): 121–132CrossRefGoogle Scholar
  58. Wang, Z., Simoncelli, E.P., & Bovik, A. C. (2003). Multi-scale structural similarity for image quality assessment. In IEEE asilomar conference on signals, systems and computers (pp. 1398–1402). Pacific Grove, CA.Google Scholar
  59. Watson A. B. (1993) Dctune: A technique for visual optimization of dct quantization matrices for individual images. Society for Information Display Digest of Technical Papers 1: 946–949Google Scholar
  60. Weken D. V., Nachtegael M., Kerre E. E. (2004) Using similarity measures and homogeneity for the comparison of images. Image and Vision Computing 22(9): 695–702CrossRefGoogle Scholar
  61. Woo, W., Ortega, A., & Iwadate, Y. (1999). Stereo image coding based on binocular energy modeling. In International conference image processing (ICIP) (pp. 467–471).Google Scholar
  62. Xing, L., You, J., Ebrahimi, T., & Perkis, A. (2010). A perceptual quality metric for stereoscopic crosstalk perception. In IEEE international conference on image processing (pp. 4033–4036). Hong Kong, China.Google Scholar
  63. You, J., Xing, L., Perkis, A., & Wang, X. (2010). Assessment for stereoscopic images based on 2D image quality metrics and disparity analysis. In International workshop on video processing and quality metrics. Scottsdale, Arizona, USA.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.XLIM-SICUMR CNRS 7252, Université de PoitiersPoitiersFrance

Personalised recommendations