The accuracy of PSNR in predicting video quality for different video scenes and frame rates
- 645 Downloads
Peak Signal-to-Noise Ratio (PSNR) is widely used as a video quality metric or performance indicator. Some studies have indicated that it correlates poorly with subjective quality, whilst others have used it on the basis that it provides a good correlation with subjective data. Existing literature seems to provide conflicting evidence of the accuracy of PSNR as a video quality metric. Based on experimental results, we explain a scenario where PSNR provides a reliable indication of the variation of subjective video quality and scenarios where PSNR is not a reliable video quality metric. We show that PSNR follows a monotonic relationship with subjective quality in the case of full frame rate encoding when the video content and codec are fixed. We provide evidence that PSNR becomes an unreliable and inaccurate quality metric when several videos with different content are jointly assessed. Furthermore, PSNR is inaccurate in measuring video quality of a video content encoded at different frame rates because it is not capable of assessing the perceptual trade-off between the spatial and temporal qualities. Finally, where PSNR is not a reliable video quality metric across different video contents and frame rates, we show that a perceptual video model recently approved by the International Telecommunication Union (ITU) provides quality predictions highly correlating with subjective scores even if different video scenes coded at different frame rates are considered in the test set.
KeywordsVideo Coding Subjective quality PSNR Objective perceptual model
Unable to display preview. Download preview PDF.
- 1.Girod, B. (1993). What’s wrong with mean-squared error? In A. B. Watson (Ed.), Digital images and human vision (pp. 207–220). Cambridge: MIT Press. Google Scholar
- 2.Teo, P., & Heeger, D. (1994). Perceptual image distortion. In Proceedings of SPIE human vision, visual processing and digital display V (Vol. 2179, pp. 127–141), San Jose, February 1994. Google Scholar
- 4.Wang, Z., & Bovik, A. (2002). Why is image quality assessment so difficult? In Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP) (Vol. 4, pp. 3313–3316), Orlando, May 2002. Google Scholar
- 5.Nemethova, O., Ries, M., Siffel, E., & Rupp, M. (2002). Quality assessment for H.264 coded low-rate and low-resolution video sequences. In Third IASTED international conference on communications, Internet, and information technology (CIIT 2004) (pp. 136–140), St. Thomas, US Virgin Islands, 22–24 January 2004. Google Scholar
- 6.Richardson, I. (2004). H.264 and MPEG-4 video compression: video coding for next-generation multimedia. New York: Wiley. Google Scholar
- 7.Wolf, S., Pinson, M., Webster, A., Cermak, G., & Tweedy, E. P. (1996). Objective and subjective measures of MPEG video quality. ITS/NTIA, Tech. Rep. ANSI T1A1.5/96-121, October 1996. Google Scholar
- 8.Masry, M., & Hemami, S. (2001). An analysis of subjective quality in low bit rate video. In Proceedings of IEEE international conference on image processing (Vol. 1, pp. 465–468), October 2001. Google Scholar
- 9.Oelbaum, T., Diepold, K., & Zia, W. (2007). A generic method to increase the prediction accuracy of visual quality metrics. In Proceedings of picture coding symposium, Lisbon, November 2007. Google Scholar
- 10.Video Quality Experts Group (2008). Draft final report from the Video Quality Experts Group on the validation of objective models of multimedia quality assessment Phase I, May 2008, available at http://www.vqeg.org.
- 14.International Telecommunication Union. (2004). Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference. ITU-T Rec.J.144, March 2004. Google Scholar
- 15.International Telecommunication Union. (2004). Objective perceptual video quality measurement techniques for standard definition digital broadcast television in the presence of a full reference. ITU-R Rec. BT.1683, June 2004. Google Scholar
- 16.Video Quality Experts Group. Final report from the Video Quality Experts Group on FRTV Phase 2: Test validation of objective models of video quality assessment, August 2003. Available at http://www.vqeg.org.
- 21.International Telecommunication Union. (2008). Subjective video quality assessment methods for multimedia applications. ITU-T Rec. P.910, April 2008. Google Scholar
- 22.International Telecommunication Union. (2008). Objective perceptual multimedia video quality measurement in the presence of a full reference. ITU-T Rec. J.247, August 2008. Google Scholar
- 23.Yadavalli, G., Masry, M., & Hemami, S. (2003). Frame rate preferences in low bit rate video. In Proceedings of IEEE international conference on image processing (Vol. 1, pp. 441–444), September 2003. Google Scholar