Advertisement

Telecommunication Systems

, Volume 49, Issue 1, pp 35–48 | Cite as

The accuracy of PSNR in predicting video quality for different video scenes and frame rates

  • Quan Huynh-Thu
  • Mohammed Ghanbari
Article

Abstract

Peak Signal-to-Noise Ratio (PSNR) is widely used as a video quality metric or performance indicator. Some studies have indicated that it correlates poorly with subjective quality, whilst others have used it on the basis that it provides a good correlation with subjective data. Existing literature seems to provide conflicting evidence of the accuracy of PSNR as a video quality metric. Based on experimental results, we explain a scenario where PSNR provides a reliable indication of the variation of subjective video quality and scenarios where PSNR is not a reliable video quality metric. We show that PSNR follows a monotonic relationship with subjective quality in the case of full frame rate encoding when the video content and codec are fixed. We provide evidence that PSNR becomes an unreliable and inaccurate quality metric when several videos with different content are jointly assessed. Furthermore, PSNR is inaccurate in measuring video quality of a video content encoded at different frame rates because it is not capable of assessing the perceptual trade-off between the spatial and temporal qualities. Finally, where PSNR is not a reliable video quality metric across different video contents and frame rates, we show that a perceptual video model recently approved by the International Telecommunication Union (ITU) provides quality predictions highly correlating with subjective scores even if different video scenes coded at different frame rates are considered in the test set.

Keywords

Video Coding Subjective quality PSNR Objective perceptual model 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Girod, B. (1993). What’s wrong with mean-squared error? In A. B. Watson (Ed.), Digital images and human vision (pp. 207–220). Cambridge: MIT Press. Google Scholar
  2. 2.
    Teo, P., & Heeger, D. (1994). Perceptual image distortion. In Proceedings of SPIE human vision, visual processing and digital display V (Vol. 2179, pp. 127–141), San Jose, February 1994. Google Scholar
  3. 3.
    Wang, Z., & Bovik, A. C. (2002). A universal image quality index. IEEE Signal Processing Letters, 9(3), 81–84. CrossRefGoogle Scholar
  4. 4.
    Wang, Z., & Bovik, A. (2002). Why is image quality assessment so difficult? In Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP) (Vol. 4, pp. 3313–3316), Orlando, May 2002. Google Scholar
  5. 5.
    Nemethova, O., Ries, M., Siffel, E., & Rupp, M. (2002). Quality assessment for H.264 coded low-rate and low-resolution video sequences. In Third IASTED international conference on communications, Internet, and information technology (CIIT 2004) (pp. 136–140), St. Thomas, US Virgin Islands, 22–24 January 2004. Google Scholar
  6. 6.
    Richardson, I. (2004). H.264 and MPEG-4 video compression: video coding for next-generation multimedia. New York: Wiley. Google Scholar
  7. 7.
    Wolf, S., Pinson, M., Webster, A., Cermak, G., & Tweedy, E. P. (1996). Objective and subjective measures of MPEG video quality. ITS/NTIA, Tech. Rep. ANSI T1A1.5/96-121, October 1996. Google Scholar
  8. 8.
    Masry, M., & Hemami, S. (2001). An analysis of subjective quality in low bit rate video. In Proceedings of IEEE international conference on image processing (Vol. 1, pp. 465–468), October 2001. Google Scholar
  9. 9.
    Oelbaum, T., Diepold, K., & Zia, W. (2007). A generic method to increase the prediction accuracy of visual quality metrics. In Proceedings of picture coding symposium, Lisbon, November 2007. Google Scholar
  10. 10.
    Video Quality Experts Group (2008). Draft final report from the Video Quality Experts Group on the validation of objective models of multimedia quality assessment Phase I, May 2008, available at http://www.vqeg.org.
  11. 11.
    Reed, E., & Dufaux, F. (2001). Constrained bit-rate control for very low bit-rate streaming-video applications. IEEE Transactions on Circuits and Systems for Video Technology, 11(7), 882–889. CrossRefGoogle Scholar
  12. 12.
    Song, H., & Kuo, C. J. (2001). Rate control for low-bit-rate video via variable-encoding frame rates. IEEE Transactions on Circuits and Systems for Video Technology, 11(4), 512–521. CrossRefGoogle Scholar
  13. 13.
    Feghali, R., Speranza, F., Wang, D., & Vincent, A. (2007). Video quality metric for bit rate control via joint adjustment of quantization and frame rate. IEEE Transactions on Broadcasting, 53(1), 441–446. CrossRefGoogle Scholar
  14. 14.
    International Telecommunication Union. (2004). Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference. ITU-T Rec.J.144, March 2004. Google Scholar
  15. 15.
    International Telecommunication Union. (2004). Objective perceptual video quality measurement techniques for standard definition digital broadcast television in the presence of a full reference. ITU-R Rec. BT.1683, June 2004. Google Scholar
  16. 16.
    Video Quality Experts Group. Final report from the Video Quality Experts Group on FRTV Phase 2: Test validation of objective models of video quality assessment, August 2003. Available at http://www.vqeg.org.
  17. 17.
    Wiegand, T., Schwarz, H., Joch, A., Kossentini, F., & Sullivan, G. (2003). Rate-constrained coder control and comparison of video coding standards. IEEE Transactions on Circuits and Systems for Video Technology, 13(7), 688–703. CrossRefGoogle Scholar
  18. 18.
    Ichigaya, A., Kurozumi, M., Hara, N., Nishida, Y., & Nakasu, E. (2006). A method of estimating coding PSNR using quantized DCT coefficients. IEEE Transactions on Circuits and Systems for Video Technology, 16(2), 251–259. CrossRefGoogle Scholar
  19. 19.
    Eden, A. (2007). No-reference estimation of the coding PSNR for H.264-coded sequences. IEEE Transactions on Consumer Electronics, 53(2), 667–674. CrossRefGoogle Scholar
  20. 20.
    Huynh-Thu, Q., & Ghanbari, M. (2008). Scope of validity of PSNR in image/video quality assessment. IET Electronic Letters, 44(13), 800–801. CrossRefGoogle Scholar
  21. 21.
    International Telecommunication Union. (2008). Subjective video quality assessment methods for multimedia applications. ITU-T Rec. P.910, April 2008. Google Scholar
  22. 22.
    International Telecommunication Union. (2008). Objective perceptual multimedia video quality measurement in the presence of a full reference. ITU-T Rec. J.247, August 2008. Google Scholar
  23. 23.
    Yadavalli, G., Masry, M., & Hemami, S. (2003). Frame rate preferences in low bit rate video. In Proceedings of IEEE international conference on image processing (Vol. 1, pp. 441–444), September 2003. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Psytechnics LtdIpswichUK
  2. 2.University of EssexColchesterUK

Personalised recommendations