Error sensitivity model based on spatial and temporal features

Abstract

Packet loss and error propagation induced by it are significant causes of visual impairments in video applications. Most of the existing video quality assessment models are developed at frame or sequence level, which can not accurately describe the impact of packet loss on the local regions in one frame. In this paper, we propose an error sensitivity model to evaluate the impact of a single packet loss. We also make full use of the spatio-temporal correlation of the video and analyze a set of features that directly impact the perceptual quality of videos, based on the specific situation of video packet loss. With the aid of the support vector regression (SVR), these features are used to predict the error sensitivity of the local region. The proposed model is tested on six video sequences. Experimental results show that the proposed model predicts sensitivity of videos to different packet loss cases with certain reasonable accuracy, and provides good generalization ability, which turns out outperform the state-of-art image and video quality assessment methods.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. 1.

    A. Adeyemi-Ejeye, M. Alreshoodi, L. Al-Jobouri, M. Fleury, J. Woods and M. Medhi (2017) IEEE 802.11ac wireless delivery of 4kUHD video: The impact of packet loss. Proc ICCE-Berlin 258–259. Berlin, Germany

  2. 2.

    H. Bayrak, G. N. Yilmaz (2018) No-Reference Evaluation Of 3-Dimensional Video Quality Using Spatial and Frequency Domain Components. 26th Signal Processing and Communications Applications Conference (SIU) 1–4

  3. 3.

    Bondzulic BP, Pavlovic BZ, Petrovic VS, Andric MS (2016) Performance of peak signal-to-noise ratio quality assessment in video streaming with packet losses. Electron Lett 52(6):454–456

    Article  Google Scholar 

  4. 4.

    N Chen, X Jiang, C Wang and J Su (2011) Study on relationship between network video packet loss and video quality. Proc CISP 282–286. Shanghai, China

  5. 5.

    N. Chen, X. Jiang and C. Wang (2012) Impact of packet loss distribution on the perceived IPTV video quality. Proc CISP 38–42. Chongqing, China

  6. 6.

    Frnda J, Voznak M, Sevcik L (2016) Impact of packet loss and delay variation on the quality of real-time video streaming. Telecommun Syst 62(2):265–275

    Article  Google Scholar 

  7. 7.

    Gao P, Peng Q, Xiang W (2017) Analysis of packet-loss-induced distortion in view synthesis prediction-based 3D video coding. IEEE Trans Image Process 26(6):2781–2796

    MathSciNet  Article  Google Scholar 

  8. 8.

    MN Garcia and A Raake (2010) Parametric packet-layer video quality model for IPTV. Proc Inform Sci Sign Proc Their Appl 349–352. Kuala Lumpur, Malaysia

  9. 9.

    Haralick RM, Shanmugam K, Dinstein IH (1973) Textural features for image classification. IEEE Trans Syst Man Cybern -Syst Smc-3(6):610–621

    Article  Google Scholar 

  10. 10.

    CTER Hewage, MG Martini, HD Appuhami (2012) A study on the impact of compression and packet losses on rendered 3D views. Proc SPIE 8290, Three-Dimensional Image Processing (3DIP) and Applications II, pp. 82901D–82901D–9, Burlingame, CA, USA

  11. 11.

    CTER Hewage, MG Martini, M Brandas and DVSX De Silva (2013) A study on the perceived quality of 3D video subject to packet losses. Proc IEEE ICC Workshops 662–666. Budapest, Hungary

  12. 12.

    Hu Z, Zhang Q (2018) A new approach for packet loss measurement of video streaming and its application. Multimed Tools Appl 77(10):11589–11608

    Article  Google Scholar 

  13. 13.

    Hu S, Jin L, Wang H, Zhang Y, Kwong S, Kuo C-J (2017) Objective video quality assessment based on perceptually weighted mean squared error. IEEE Trans Circ Syst Video Technol 27(9):1844–1855

    Article  Google Scholar 

  14. 14.

    ITU-T Rec. P.910 (1999) Subjective video quality assessment methods for multimedia applications, ITU, Geneva, Switzerland

  15. 15.

    Joskowicz J, Sotelo R (2014) A model for video quality assessment considering packet loss for broadcast digital television coded in H.264. Int J Digit Multimed Broadcast 2014:1–11

    Article  Google Scholar 

  16. 16.

    J Joskowicz, R Sotelo and JC Lopez Arado (2012) Comparison of parametric models for video quality estimation: Towards a general model. Proc IEEE Int Symp Broadband Multimed Syst Broadcast 1–7. Seoul, Korea

  17. 17.

    Korhonen J (2018) Study of the subjective visibility of packet loss artifacts in decoded video sequences. IEEE Trans Broadcast 64(2):354–366

    Article  Google Scholar 

  18. 18.

    J Korhonen (2018) Learning-based prediction of packet loss artifact visibility in networked video. Proc. 10th Int. Workshop QoMEX 1–6. Cagliari, Italy

  19. 19.

    Li Y, Po LM, Cheung CH, Xu X, Feng L, Yuan F, Cheung KW (2016) No-reference video quality assessment with 3D shearlet transform and convolutional neural networks. IEEE Trans Circ Syst Video Technol 26(6):1044–1057

    Article  Google Scholar 

  20. 20.

    Li X, Guo Q, Lu X (2016) Spatiotemporal statistics for video quality assessment. IEEE Trans Image Process 25(7):3329–3342

    MathSciNet  Article  Google Scholar 

  21. 21.

    K. Manasa, KVSNL Manasa Priya and SS Channappayya (2014) A perceptually motivated no-reference video quality assessment algorithm for packet loss artifacts. Proc 6th Int Workshop QoMEX 67–68. Singapore, Singapore

  22. 22.

    A. Mittal, M. Saad, A. C. Bovik (2014) Assessment of Video Naturalness Using Time-Frequency Statistics. IEEE International Conference on Image Processing (ICIP) 571–574

  23. 23.

    Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20(12):3350–3364

    MathSciNet  Article  Google Scholar 

  24. 24.

    Oszust M (2017) No-reference image quality assessment using image statistics and robust feature descriptors. IEEE Sign Proc Lett 24(11):1656–1660

    Article  Google Scholar 

  25. 25.

    R. Pauliks, I. Slaidins, K. Tretjaks and A. Krauze (2015) Assessment of IP packet loss influence on perceptual quality of streaming video. Proc APMediaCast 1–6. Kuta, Indonesia

  26. 26.

    Saad MA, Bovik AC, Charrier C (2014) Blind prediction of natural video quality. IEEE Trans Image Process 23(3):1352–1365

    MathSciNet  Article  Google Scholar 

  27. 27.

    YM Saputra and Hendrawan (2016) The effect of packet loss and delay jitter on the video streaming performance using H.264/MPEG-4 Scalable Video Coding. Proc TSSA 1–6. Denpasar, Indonesia

  28. 28.

    Shahid M, Pandremmenou K, Kondi LP, Rossholm A, Lövström B (2016) Perceptual quality estimation of H.264/AVC videos using reduced-reference and no-reference models. J Electron Imaging 25(5):053012–053012-26

    Article  Google Scholar 

  29. 29.

    Song J, Yang F (2014) No-reference video quality assessment model for distortion caused by packet loss in the real-time mobile video services. Adv Multimed 2014:1–15

    Article  Google Scholar 

  30. 30.

    Tang S, Alface PR (2014) Impact of random and burst packet losses on H.264 scalable video coding. IEEE Trans Multimed 16(8):2256–2269

    Article  Google Scholar 

  31. 31.

    M Uhrina, M Vaculík (2015) The impact of bitrate and packet loss on the video quality of H.264/AVC compression standard. Proc. TSP 1–6. Prague, Czech Republic

  32. 32.

    Valenzise G, Magni S, Tagliasacchi M, Tubaro S (2012) No-reference pixel video quality monitoring of channel-induced distortion. IEEE Trans Circ Syst 22(4):605–618

    Article  Google Scholar 

  33. 33.

    S Wan, F Yang, Z Xie (2010) Evaluation of video quality degradation due to packet loss. Proc ISPACS 1–4. Chengdu, China

  34. 34.

    Wang Z, Wang W, Xia Y, Wan Z, Wang J, Li L, Cai C (2014) Visual quality assessment after network transmission incorporating NS2 and Evalvid. Scientific World J 2014:1–7

    Google Scholar 

  35. 35.

    Yang F, Wan S, Xie Q, Wu HR (2010) No-reference quality assessment for networked video via primary analysis of bit stream. IEEE Trans Circ Syst Video Technol 20(11):1544–1554

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant No. 61301112, 61828105 and 61601278, Chen Guang Project supported by Shanghai Municipal Education Commission and Shanghai Education Development Foundation under Grant No.17CG41.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ran Ma.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ma, R., Li, T., Bo, D. et al. Error sensitivity model based on spatial and temporal features. Multimed Tools Appl 79, 31913–31930 (2020). https://doi.org/10.1007/s11042-020-09407-2

Download citation

Keywords

  • Packet loss
  • Spatial and temporal features
  • Error sensitivity
  • Regression