Skip to main content

Advertisement

SpringerLink
Log in

An error-based video quality assessment method with temporal information

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Videos are amongst the most popular online media for Internet users nowadays. Thus, it is of utmost importance that the videos transmitted through the internet or other transmission media to have a minimal data loss and acceptable visual quality. Video quality assessment (VQA) is a useful tool to determine the quality of a video without human intervention. A new VQA method, termed as Error and Temporal Structural Similarity (EaTSS), is proposed in this paper. EaTSS is based on a combination of error signals, weighted Structural Similarity Index (SSIM) and difference of temporal information. The error signals are used to weight the computed SSIM map and subsequently to compute the quality score. This is a better alternative to the usual SSIM index, in which the quality score is computed as the average of the SSIM map. For the temporal information, the second-order time-differential information are used for quality score computation. From the experiments, EaTSS is found to have competitive performance and faster computational speed compared to other existing VQA algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Akramullah S (2014) Digital video concepts, methods, and metrics. New York, USA

  2. Banno H, Saiki J (2015) The use of higher-order statistics in rapid object categorization in natural scenes. J Vis 15(2):1–20. https://doi.org/10.1167/15.2.4

    Article  Google Scholar 

  3. Bong DBL, Khoo BE (2015) Objective blur assessment based on contraction errors of local contrast maps. Multimed Tools Appl 74(17):7355–7378. https://doi.org/10.1007/s11042-014-1983-5

    Article  Google Scholar 

  4. Cardoso JVM, Alencar MS, Regis CDM, Oliveira IP (2014) Temporal analysis and perceptual weighting for objective video quality measurement. IEEE southwest symposium on image analysis and interpretation (SSIAI). IEEE, 2014, pp 57–60. https://doi.org/10.1109/SSIAI.2014.6806028

  5. Choi LK, Bovik AC (2016) Flicker sensitive motion tuned video quality assessment. 2016 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI). IEEE, 2016, pp 29–32. https://doi.org/10.1109/SSIAI.2016.7459167

  6. Hall CF, Hall EL (1977) A nonlinear model for the spatial characteristics of the human visual system. IEEE Trans Syst Man Cybern 7(3):274–283. https://doi.org/10.1109/TSMC.1977.4309680

    Article  MathSciNet  Google Scholar 

  7. Ítalio PO, José VMC, Carlos DMR, Marcelo SA (2013) Spatial and temporal analysis considering relevant regions applied to video quality assessment. XXXI Brazilian Telecommunications Symposium (SBrT), 2013, pp 1–5

  8. ITU-T (1999) ITU-T recommendation P. 910: subjective video quality assessment methods for multimedia applications. ITU-T

  9. ITU-T (2016) ITU-T recommendation H.264: advanced video coding for generic audiovisual services. ITU-T

  10. Larson EC, Chandler DM (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electron Imaging 19(1):011006–011006. https://doi.org/10.1117/1.3267105

    Article  Google Scholar 

  11. Lee S, Pattichis MS, Bovik AC (2002) Foveated video quality assessment. IEEE Trans Multimedia 4(1):129–132. https://doi.org/10.1109/6046.985561

    Article  Google Scholar 

  12. Li C, Bovik AC (2009) Three-component weighted structural similarity index. Proceding SPIE 7242, image quality and system performance VI, 72420Q. SPIE, 2009, pp 1–9

  13. Li P, Xie J, Wang Q, Zuo W (2017) Is second-order information helpful for large-scale visual recognition?. IEEE international conference on computer vision (ICCV). IEEE, 2017, pp 2070–2078

  14. Limb JO (1979) Distortion criteria of the human viewer. IEEE Trans Syst Man Cybern 9(12):778–793. https://doi.org/10.1109/TSMC.1979.4310129

    Article  Google Scholar 

  15. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: recognizing complex activities from sensor data. Proceedings of the 24th international conference on artificial intelligence. IJCAI, 2015, pp 1617–1623

  16. Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (2016) Recognizing complex activities by a probabilistic interval-based model. Proceedings of the thirtieth AAAI conference on artificial intelligence. AAAI, 2016, pp 1266–1272

  17. Loh WT, Bong DBL (2015) Video quality assessment method: MD-SSIM. IEEE international conference on consumer electronics - Taiwan (ICCE-TW). IEEE, 2015, pp 290–291. https://doi.org/10.1109/ICCE-TW.2015.7216904

  18. Ooyala (2016) Ooyala global video index q3 2015. Ooyala. http://go.ooyala.com/rs/OOYALA/images/Ooyala-Global-Video-Index-Q3-2015.pdf. Accessed 14 Apr 2016

  19. Peng P, Cannons K, Li ZN (2013) Efficient video quality assessment based on spacetime texture representation. In: Proceedings of the 21st ACM international conference on multimedia. ACM, 2013, pp 641–644. https://doi.org/10.1145/2502081.2502168

  20. Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322. https://doi.org/10.1109/TBC.2004.834028

    Article  Google Scholar 

  21. Pinson MH, Choi LK, Bovik AC (2014) Temporal video quality model accounting for variable frame delay distortions. IEEE Trans Broadcast 60(4):637–649. https://doi.org/10.1109/TBC.2014.2365260

    Article  Google Scholar 

  22. Rimac-Drlje S, Vranješ M, Žagar D (2010) Foveated mean squared error - a novel video quality metric. Multimed Tools Appl 49(3):425–445. https://doi.org/10.1007/s11042-009-0442-1

    Article  Google Scholar 

  23. Seshadrinathan K, Bovik AC (2010) Motion tuned spatio-temporal quality assessment of natural videos. IEEE Trans Image Process 19(2):335–350. https://doi.org/10.1109/TIP.2009.2034992

    Article  MathSciNet  MATH  Google Scholar 

  24. Seshadrinathan K, Soundararajan R, Bovik AC, Cormack LK (2010) Study of subjective and objective quality assessment of video. IEEE Trans Image Process 19(6):1427–1441. https://doi.org/10.1109/TIP.2010.2042111

    Article  MathSciNet  MATH  Google Scholar 

  25. Seshadrinathan K, Soundararajan R, Bovik AC, Cormack LK (2010) A subjective study to evaluate video quality assessment algorithms. IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, 2010, pp 75270H-75270H-10. https://doi.org/10.1117/12.845382

  26. Sheikh HR, Bovik AC (2005) A visual information fidelity approach to video quality assessment. First international workshop on video processing and quality metrics for consumer electronics. Springer, 2005, pp 23–25

  27. Sheikh HR, Bovik AC, de Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128. https://doi.org/10.1109/TIP.2005.859389

    Article  Google Scholar 

  28. Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11):3440–3451. https://doi.org/10.1109/TIP.2006.881959

    Article  Google Scholar 

  29. Soundararajan R, Bovik AC (2013) Video quality assessment by reduced reference Spatio-temporal entropic differencing. IEEE Trans Circuits Syst Video Technol 23(4):684–694. https://doi.org/10.1109/TCSVT.2012.2214933

    Article  Google Scholar 

  30. Suchow JW, Alvarez GA (2011) Motion silences awareness of visual change. Curr Biol 21(2):140–143. https://doi.org/10.1016/j.cub.2010.12.019

    Article  Google Scholar 

  31. Sullivan GJ, Ohm JR, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668. https://doi.org/10.1109/TCSVT.2012.2221191

    Article  Google Scholar 

  32. Vo DT, Nguyen TQ (2008) Quality enhancement for motion JPEG using temporal redundancies. IEEE Trans Circuits Syst Video Technol 18(5):609–619. https://doi.org/10.1109/TCSVT.2008.918807

    Article  Google Scholar 

  33. VQEG (2003) Final report from the video quality experts group on the validation of objective models of video quality assessment. VQEG

  34. Vranješ M, Rimac-Drlje S, Grgić K (2013) Review of objective video quality metrics and performance comparison using different databases. Signal Process Image Commun 28(1):1–19. https://doi.org/10.1016/j.image.2012.10.003

    Article  Google Scholar 

  35. Vu PV, Chandler DM (2014) ViS3: an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices. J Electron Imaging 23(1):013016–013016. https://doi.org/10.1117/1.JEI.23.1.013016

    Article  Google Scholar 

  36. Vu C, Deshpande S (2012) ViMSSIM: from image to video quality assessment. Proc of the 4th workshop on mobile video. ACM, 2012, pp 1–6. https://doi.org/10.1145/2151677.2151679

  37. Wandell BA (1995) Foundations of vision. Sinauer Associates, Sunderland

    Google Scholar 

  38. Wang Z, Bovik AC (2009) Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process Mag 26(1):98–117. https://doi.org/10.1109/MSP.2008.930649

    Article  Google Scholar 

  39. Wang Z, Li Q (2011) Information content weighting for perceptual image quality assessment. IEEE Trans Image Process 20(5):1185–1198. https://doi.org/10.1109/TIP.2010.2092435

    Article  MathSciNet  MATH  Google Scholar 

  40. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 3(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  41. Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Commun 19(2):121–132. https://doi.org/10.1016/S0923-5965(03)00076-6

    Article  Google Scholar 

  42. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. Conference record of the thirty-seventh Asilomar Conf on signals, systems and computers. IEEE, 2003, pp 1398–1402. https://doi.org/10.1109/ACSSC.2003.1292216

  43. Watson AB, Hu J, McGowan JF (2001) Digital video quality metric based on human vision. J Electron Imaging 10(1):20–29. https://doi.org/10.1117/1.1329896

    Article  Google Scholar 

  44. Wolf S, Pinson M (1998) In-service performance metrics for MPEG-2 video systems. Proc made to measure 98-measurement techniques of the digital age technical seminar. IAB, 1998, pp 12–13

  45. Xue W, Mou X, Zhang L, Bovik AC, Feng X (2014) Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features. IEEE Trans Image Process 23(11):4850–4862

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported by Ministry of Higher Education Malaysia through the provision of research grant: F02/FRGS/1492/2016.

Author information

Authors and Affiliations

  1. Faculty of Engineering, Universiti Malaysia Sarawak, Kota Samarahan, Malaysia

    Woei-Tan Loh & David Boon Liang Bong

Authors
  1. Woei-Tan Loh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Boon Liang Bong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Boon Liang Bong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Loh, WT., Bong, D.B.L. An error-based video quality assessment method with temporal information. Multimed Tools Appl 77, 30791–30814 (2018). https://doi.org/10.1007/s11042-018-6107-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6107-1

Keywords

Search

Navigation