Real-Time 3D QoE Evaluation of Novel 3D Media

  • Chaminda T. E. R. Hewage
  • Maria G. Martini
  • Harsha D. Appuhami
  • Christos Politis


Recent wireless networks enable the transmission of high bandwidth multimedia data, including advanced 3D video applications. Such wireless multimedia systems should be designed with the purpose of maximizing the quality perceived by the users. For instance, quality parameters can be measured at the receiver-side and fed back to the transmitter for system optimization. Measuring 3D video quality is a challenge due to a number of perceptual attributes associated with 3D video viewing (e.g., image quality, depth perception, naturalness). Subjective as well as objective metrics have been developed to measure 3D video quality against different artifacts. However most of these metrics are Full-Reference (FR) quality metrics and require the original 3D video sequence to measure the quality at the receiver-end. Therefore, these are not a viable solution for system monitoring/update “on the fly.” This chapter presents a Near No-Reference (NR) quality metric for color plus depth 3D video compression and transmission using the extracted edge information of color images and depth maps. This work is motivated by the fact that the edges/contours of the depth map and of the corresponding color image can represent different depth levels and identify image objects/boundaries of the corresponding color image and hence can be used in quality evaluation. The performance of the proposed method is evaluated for different compression ratios and network conditions. The results obtained match well those achieved with its counterpart FR quality metric and with subjective tests, with only a few bytes of overhead for the original 3D image sequence as side-information.


Quantization Parameter Mean Opinion Score Binocular Rivalry Error Concealment Depth Image Base Rendering 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Le-Callet P, Moeller S, Perkis A (2012) Qualinet white paper on definitions of quality of experience, Version 1.1. European network on quality of experience in multimedia systems and services, COST Action IC 1003, June, 2012Google Scholar
  2. 2.
    Martini MG, Mazzotti M, Lamy-Bergot C, Huusko J, Amon P (2007) Content adaptive network aware joint optimization of wireless video transmission. IEEE Commun Mag 45(1):84–90CrossRefGoogle Scholar
  3. 3.
    Meesters LMJ, IJsselsteijn WA, Seuntiens PJH (2004) A survey of perceptual evaluations and requirements of three-dimensional TV. IEEE Trans Circuits Syst Video Technol 14(3):381–391CrossRefGoogle Scholar
  4. 4.
    Wang K, Barkowsky M, Brunnstrom K, Sjostrom M, Cousseau R, Le-Callet P (2012) Perceived {3D TV} transmission quality assessment: multi-laboratory results using absolute category rating on quality of experience scale. IEEE Trans Broadcast 58(4):544–557CrossRefGoogle Scholar
  5. 5.
    Cutting JE, Vishton PM (1995) Perceiving layout and knowing distances: the integration, relative potency, and contextual use of different information about depth. In: Epstein W, Rogers S (eds) Perception of space and motion. Academic, San Diego, pp 69–117CrossRefGoogle Scholar
  6. 6.
    Lambooij M, Fortuin M, Heynderickx I, IJsselsteijn W (2009) Visual discomfort and visual fatigue of stereoscopic displays: a review. J Imaging Sci Technol 53(3):30201-1–30201-14CrossRefGoogle Scholar
  7. 7.
    Tam W, Stelmach L, Corriveau P (1998) Psychovisual aspects of viewing stereoscopic video sequences. Proc SPIE 3295:226–235CrossRefGoogle Scholar
  8. 8.
    IJsselsteijn W, De Ridder H, Vliegen J (2000) Subjective evaluation of stereoscopic images: effects of camera parameters and display duration. IEEE Trans Circuits Syst Video Technol 10(2):225–233CrossRefGoogle Scholar
  9. 9.
    International Telecommunication Union/ITU Radio communication Sector (2000) Subjective Assessment of Stereoscopic Television Pictures. ITU-R BT.1438, Jan 2000Google Scholar
  10. 10.
    International Telecommunication Union/ITU Radio communication Sector (2002) Methodology for the subjective assessment of the quality of television pictures. ITU-R BT.500-11, Jan 2002Google Scholar
  11. 11.
    Seuntiens P, Meesters L, Ijsselsteijn W (2008) Perceived quality of compressed stereoscopic images: effects of symmetric and asymmetric JPEG coding and camera separation. ACM Trans Appl Percept (TAP) 3(2):95–109CrossRefGoogle Scholar
  12. 12.
    Seuntians P, Meesters L, IJsselsteijn W (2003) Perceptual evaluation of JPEG coded stereoscopic images. Proc SPIE 5006:215–226CrossRefGoogle Scholar
  13. 13.
    Starck J, Kilner J, Hilton A (2008) Objective quality assessment in free-viewpoint video production. In: 3DTV conference, TurkeyGoogle Scholar
  14. 14.
    Benoit A, Le Callet P, Campisi P, Cousseau R (2008) Quality assessment of stereoscopic images. EURASIP J Image Video Proc 2008:1–13CrossRefGoogle Scholar
  15. 15.
    Hewage CTER, Worrall ST, Dogan S, Villette S, Kondoz AM (2009) Quality evaluation of color plus depth map-based stereoscopic video. IEEE J Sel Top Sign Proces 3(2):304–318CrossRefGoogle Scholar
  16. 16.
    Hewage CTER, Worrall ST, Dogan S, Kondoz AM (2008) Prediction of stereoscopic video quality using objective quality models of 2-D video. IET Electron Lett 44(16):963–965CrossRefGoogle Scholar
  17. 17.
    Yasakethu SLP, Hewage CTER, Fernando WAC, Worrall ST, Kondoz AM (2008) Quality analysis for 3D video using 2D video quality models. IEEE Trans Consumer Electron 54(4):1969–1976CrossRefGoogle Scholar
  18. 18.
    Babu RV, Bopardikar AS, Perkis A, Hillestad OI (2004) No-reference metrics for video streaming applications. In: Proc Int Packet Video WorkshopGoogle Scholar
  19. 19.
    Wang Z, Sheikh HR, Bovik AC (2002) No-reference perceptual quality assessment of JPEG compressed images. In: Proc IEEE Int. Conf. on Image Processing. pp 477–480Google Scholar
  20. 20.
    Wang Z, Simoncelli E (2005) Reduced-reference image quality assessment using a wavelet-domain natural image statistic model. In: 17th SPIE annual symposium on electronic imaging, San Jose, Jan 2005Google Scholar
  21. 21.
    Wolf S, Pinson MH (2005) Low bandwidth reduced reference video quality monitoring system. In: First inter. workshop on video processing and quality metrics for cons electronics, Arizona, Jan 2005Google Scholar
  22. 22.
    Akhter R, Sazzad ZMP, Horita Y, Meek D (2010) No reference stereoscopic image quality assessment. In: Proc of SPIE: stereoscopic displays and applications XXI 7524, California, Jan 2010Google Scholar
  23. 23.
    Merkle P, Smolic A, Muller K and Wiegand T (2007) Multi-View Video Plus Depth Representation and Coding. In: IEEE Int Conf Image Processing, San Antonio, pp 201–204Google Scholar
  24. 24.
    Fehn C (2003) A 3D-TV approach using depth-image-based rendering (DIBR). In: Proceedings of Visualization, Imaging, and Image Processing (VIIP’03), pp 482–487Google Scholar
  25. 25.
    Lee C, Cho S, Choe J, Jeong T, Ahn W, Lee E (2006) Objective video quality assessment. Opt Eng 45(1):017004–017004CrossRefGoogle Scholar
  26. 26.
    Chen GH, Yang CL, Xie SL (2006) Gradient-based structural similarity for image quality assessment. In: Int Conf Image Processing, IEEE. pp 2929–2932Google Scholar
  27. 27.
    Hewage CTER, Martini MG (2010) Reduced-reference quality metric for 3D depth map transmission. In: 3DTV conference 2010, FinlandGoogle Scholar
  28. 28.
    Hewage CTER, Martini MG (2010) Reduced reference quality metric for compressed depth maps associated with colour plus depth 3D video. In: IEEE Int Conf Image Processing (ICIP 2010), September 26–29, Hong KongGoogle Scholar
  29. 29.
    Ekmekcioglu E, Mrak M, Worrall ST, Kondoz AM (2009) Edge adaptive up-sampling of depth map videos for enhanced free-viewpoint video quality. IET Electron Lett 45(7):353–354CrossRefGoogle Scholar
  30. 30.
    Wenger S (1999) Error patterns for Internet video experiments. ITU-T SG16 Document Q15-I-16-R1Google Scholar
  31. 31.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error measurement to structural similarity. IEEE Trans Image Process 13(4):600–613CrossRefGoogle Scholar
  32. 32.
    Chen GH, Yang CL, Po LM, Xie SL (2006) Edge-based structural similarity for image quality assessment. In: Int conf acoustics, speech and signal processing. pp 933–936Google Scholar
  33. 33.
    Marr D, Hildreth E (1980) Theory of edge detection. In: Proc of the Royal Society of London. Series BGoogle Scholar
  34. 34.
    Woods J (2006) Multidimensional signal, image and video processing and coding. Academic, ElsevierGoogle Scholar
  35. 35.
    Kazakova N, Margala M, Durdle NG (2004) Sobel edge detection processor for a real-time volume rendering system. In: Proc Int Symp Circuits Syst. pp 913–916Google Scholar
  36. 36.
    ISO/IEC JTC 1/SC 29/WG 11 (2006) Committee Draft of ISO/IEC 23002-3 Auxiliary Video Data Representations, Doc. N8038Google Scholar
  37. 37.
    Gangwal O, Berretty R (2009) Depth map post-processing for 3D-TV. Consumer electron (ICCE ‘09), pp 1–2Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Chaminda T. E. R. Hewage
    • 1
  • Maria G. Martini
    • 1
  • Harsha D. Appuhami
    • 1
  • Christos Politis
    • 1
  1. 1.Kingston University LondonKingston Upon ThamesUK

Personalised recommendations