Paired comparison-based subjective quality assessment of stereoscopic images
- 639 Downloads
As 3D image and video content has gained significant popularity, subjective 3D quality assessment has become an important issue for the creation, processing, and distribution of high quality 3D content. Reliable subjective quality assessment of 3D content is often difficult due to the subjects’ limited 3D experience, the interaction of multiple quality factors, minor quality differences between stimuli, etc. Among subjective evaluation methodologies, paired comparison has the advantage of improved simplicity and reliability, which can be useful to tackle the aforementioned difficulties. In this paper, we propose a new method to analyze the results of paired comparison-based subjective tests. We assume that ties convey information about the significance of quality score differences between two stimuli. Then, a maximum likelihood estimation is performed to obtain confidence intervals providing intuitive measures of significance of the quality differences. We describe the complete test procedure using the proposed method, from subjective experiment design to outlier detection and score analysis for 3D image quality assessment. Especially, we design the test procedure in a way that quality comparison across different contents is enabled while the number of pair-wise comparisons is minimized. Experimental results on a stereoscopic image database with varying camera distances demonstrate the usefulness of the proposed method and enhanced quality discriminability of paired comparison in comparison to the conventional single stimulus methodology.
KeywordsStereoscopic image Subjective quality Paired comparison Quality of experience (QoE)
This work was supported in part by the Ministry of Knowledge Economy, Korea, under the IT Consilience Creative Program (NIPA-2010-C1515-1001-0001), in part by Yonsei University Research Fund of 2011, and in part by the COST Action IC1003 European Network on Quality of Experience in Multimedia Systems and Services (Qualinet).
- 1.Bercovitz J (1998) Image-side perspective and stereoscopy. In: Proc. SPIE, vol 3295Google Scholar
- 4.Chen KT, Wu CC, Chang YC, Lei CL (2009) A crowdsourceable QoE evaluation framework for multimedia content. In: Proc. ACM multimedia, pp 491–500Google Scholar
- 6.Eichhorn A, Ni P, Eg R (2010) Randomized pair comparison- an economic and robust method for audiovisual quality assessment. In: Proc. int. workshop on network and operating systems support for digital audio and video. Amsterdam, The Netherlands, pp 63–68Google Scholar
- 8.Goldmann L, Simone FD, Ebrahimi T (2010) Impact of acquisition distortions on the quality of stereoscopic images. In: Proc. int. workshop on video processing and quality metrics for consumer electronics. Scottsdale, Arizona, USAGoogle Scholar
- 11.Huynh-Thu Q, Callet PL, Barkowsky M (2010) Video quality assessment: from 2D to 3D- challenges and future trends. In: Proc. int. conf. image processing. Hong Kong, China, pp 4025–4028Google Scholar
- 15.Lee JS, Simone FD, Ramzan N, Zhao Z, Kurutepe E, Sikora T, Ostermann J, Izquierdo E, Ebrahimi T (2010) Subjective evaluation of scalable video coding for content distribution. In: Proc. ACM multimedia. Firenze, Italy, pp 65–72Google Scholar
- 18.International Telecommunication Union (2002) Methodology for the subjective assessment of the quality of television pictures. Recommendation ITU-R BT.500-11Google Scholar
- 22.International Telecommunication Union (2000) Subjective assessment of stereoscopic television pictures. Recommendation ITU-R BT.1438Google Scholar
- 23.International Telecommunication Union (1999) Subjective video quality assessment methods for multimedia applications. Recommendation ITU-R P.910Google Scholar