Advertisement

Real-time video freezing detection for 4K UHD videos

  • Ratko GrbićEmail author
  • Dejan Stefanović
  • Mario Vranješ
  • Marijan Herceg
Original Research Paper
  • 45 Downloads

Abstract

Video frame freezing is a common artifact which can occur during video content delivery due to errors in the video coding process, video transmission, storage or reproduction. This artifact can significantly decrease the end-user Quality of Experience. Therefore, accurate video frame freezing detection is of great importance for different parties involved in video content delivery. In this paper, a new Real-Time no-reference Freezing Detection Algorithm, called the RTFDA, is proposed. As newly video frames are acquired, the RTFDA performs comparison of the current video frame with the corresponding previous one to detect whether video freezing is occurring. The comparison is made by calculating the corresponding subsampled video frames and their pixel-by-pixel absolute difference comparison. The benefit of such approach is twofold: the influence of noise in freezing frames on frame comparison is significantly reduced as well as computational complexity of frame comparison. The RTFDA has a high detection rate with a very low rate of both false-positive and false-negative detections, outperforming four freezing detection algorithms on four different video databases. The proposed implementation on an x86-64 platform achieves real-time performance on 4K Ultra High Definition (UHD) videos by processing 216 frames per second (fps). Apart from that, FPGA implementation is proposed, which has efficient FPGA resource utilization.

Keywords

No-reference 4K UHD Video freezing detection Real-time FPGA 

Notes

Acknowledgements

This work was supported by Josip Juraj Strossmayer University of Osijek business fund through its internal call for proposals for research and artistic projects “UNIOS ZUP-2018” and by the Ministry of Education, Science and Technological Development of the Republic of Serbia, under grant number: TR32041. We would like to thank Velibor Škobić, MSc, for his valuable help and comments regarding the FPGA implementation of the proposed freezing detection algorithm.

References

  1. 1.
    Babić, D., Vranješ, M.: FERIT-RTRK UHD VFD. http://www.rt-rk.com/other/UHDVideoDBReadme.html (2018)
  2. 2.
    Babić, D., Stefanović, D., Vranješ, M., Herceg, M.: Real-time no-reference histogram-based freezing artifact detection algorithm for UHD videos. Multimed. Tools Appl. (2019).  https://doi.org/10.1007/s11042-019-7184-5 Google Scholar
  3. 3.
    Baik, E., Pande, A., Stover, C., Mohapatra, P.: Video acuity assessment in mobile devices. In: 2015 IEEE Conference on Computer Communications (INFOCOM), IEEE, vol. 26, pp. 1–9.  https://doi.org/10.1109/INFOCOM.2015.7218361. http://ieeexplore.ieee.org/document/7218361/ (2015)
  4. 4.
    Barkowsky, M., Sedano, I., Brunnström, K., Leszczuk, M., Staelens, N.: Hybrid video quality prediction: reviewing video quality measurement for widening application scope. Multimed. Tools Appl. 74(2), 323–343 (2015).  https://doi.org/10.1007/s11042-014-1978-2 CrossRefGoogle Scholar
  5. 5.
    Black Box Testing—BBT: RT–AV4K Network attached 4K UHD Audio/Video capture device. http://bbt.rt-rk.com/hardware/rt-av4k/ (2018)
  6. 6.
    Borer, S.: A model of jerkiness for temporal impairments in video transmission. In: 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX), IEEE, pp. 218–223.  https://doi.org/10.1109/QOMEX.2010.5516155. http://ieeexplore.ieee.org/document/5516155/ (2010)
  7. 7.
    Cabello, F., León, J., Iano, Y., Arthur, R.: Implementation of a fixed-point 2D Gaussian filter for image processing based on FPGA. In: 2015 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA), pp. 28–33. (2015).  https://doi.org/10.1109/SPA.2015.7365108
  8. 8.
    Chen, M.J., Bovik, A.C.: Fast structural similarity index algorithm. J. Real Time Image Process. 6(4), 281–287 (2011).  https://doi.org/10.1007/s11554-010-0170-9 CrossRefGoogle Scholar
  9. 9.
  10. 10.
    HajiRassouliha, A., Taberner, A.J., Nash, M.P., Nielsen, P.M.: Suitability of recent hardware accelerators (DSPs, FPGAs, and GPUs) for computer vision and image processing algorithms. Signal Process. Image Commun. 68, 101–119 (2018).  https://doi.org/10.1016/j.image.2018.07.007. http://www.sciencedirect.com/science/article/pii/S0923596518303606
  11. 11.
    Hong, B.H., Joo, H.J., Lee, E.S.: Quality analysis of real-time digital broadcasting images applying the Qoe measurement technology. J. Real Time Image Process. 9(3), 579–585 (2014).  https://doi.org/10.1007/s11554-014-0398-x CrossRefGoogle Scholar
  12. 12.
    Huynh-Thu, Q., Ghanbari, M.: No-reference temporal quality metric for video impaired by frame freezing artefacts. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2221–2224 (2009).  https://doi.org/10.1109/ICIP.2009.5413894
  13. 13.
    International Telecommunication Union: Recommendation ITU-R BT.2020-2. https://www.itu.int/rec/R-REC-BT.2020/en (2015)
  14. 14.
    van Kester, S., TXiao, Kooij R.E., Brunnstrom, K., Ahmed. O.K.: Estimating the impact of single and multiple freezes on video quality, vol. 7865, pp. 7865–10 (2011).  https://doi.org/10.1117/12.873390
  15. 15.
    Mitra, G., Johnston, B., Rendell, A.P., McCreath, E., Zhou, J.: Use of SIMD vector operations to accelerate application code performance on low-powered ARM and intel platforms. In: 2013 IEEE International Symposium on Parallel AND Distributed Processing, Workshops and Phd Forum, IEEE, pp. 1107–1116 (2013).  https://doi.org/10.1109/IPDPSW.2013.207. http://ieeexplore.ieee.org/document/6650996/
  16. 16.
    Moorthy, A.K., Choi, L.K., Bovik, A.C., de Veciana, G.: Video quality assessment on mobile devices: subjective, behavioral and objective studies. IEEE J. Sele. Top. Signal Process. 6(6), 652–671 (2012).  https://doi.org/10.1109/JSTSP.2012.2212417. http://ieeexplore.ieee.org/document/6263265/
  17. 17.
    Pinson, M.H., Choi, L.K., Bovik, A.C.: Temporal video quality model accounting for variable frame delay distortions. IEEE Trans. Broadcast. 60(4), 637–649 (2014).  https://doi.org/10.1109/TBC.2014.2365260. http://ieeexplore.ieee.org/document/6954510/
  18. 18.
    Usman, M.A., Usman, M.R., Shin, Soo Young: A no reference method for detection of dropped video frames in live video streaming. In: 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN), IEEE, vol. 2016-August, pp. 839–844 (2016).  https://doi.org/10.1109/ICUFN.2016.7537155. http://ieeexplore.ieee.org/document/7537155/
  19. 19.
    Usman, M.A., Shin, S.Y., Shahid, M., Lövström, B.: A no reference video quality metric based on jerkiness estimation focusing on multiple frame freezing in video streaming. IETE Tech. Rev. 34(3), 309–320 (2017).  https://doi.org/10.1080/02564602.2016.1185975 CrossRefGoogle Scholar
  20. 20.
    Usman, M.A., Usman, M.R., Shin, S.Y.: A novel no-reference metric for estimating the impact of frame freezing artifacts on perceptual quality of streamed videos. IEEE Trans. Multimed. (2018).  https://doi.org/10.1109/TMM.2018.2801722 Google Scholar
  21. 21.
    Škobić, V., Marić, U., Tomić, S., Rešetar, I.: Real time video freezing detection implementation on FPGA. In: 2016 24th Telecommunications Forum (TELFOR), IEEE, pp. 1–3 (2016).  https://doi.org/10.1109/TELFOR.2016.7818805. http://ieeexplore.ieee.org/document/7818805/
  22. 22.
    Wang, Z., Sheikh, H.R., Bovik, A.C.: Objective video quality assessment. The Handbook of Video Databases: Design and Applications, pp. 1041–1078. CRC Press, Boca Raton (2003).  https://doi.org/10.1117/1.2160515 Google Scholar
  23. 23.
    Webster, A., Speranza, F.: VQEG report on the validation of video quality models for high definition video content. Technical report, Video Quality Experts Group (2010). ftp://vqeg.its.bldrdoc.gov/HDTV/VQEG_HDTV_Final_Report_version_2.0.pdf
  24. 24.
    Wolf, S.: A no reference (NR) and reduced reference (RR) metric for detecting dropped video frames. In: Fourth International Workshop on Video Processing and Quality Metrics for Consumer Electronics, pp. 15–16 (2009)Google Scholar
  25. 25.
    Xue, Y., Erkin, B., Wang, Y.: A novel no-reference video quality metric for evaluating temporal jerkiness due to frame freezing. IEEE Trans. Multimed. 17(1), 134–139 (2015).  https://doi.org/10.1109/TMM.2014.2368272. http://ieeexplore.ieee.org/document/6949688/
  26. 26.
    Yammine, G., Wige, E., Simmet, F., Niederkorn, D., Kaup, A.: Blind frame freeze detection in coded videos. In: 2012 Picture Coding Symposium, IEEE, pp. 341–344 (2012).  https://doi.org/10.1109/PCS.2012.6213315. http://ieeexplore.ieee.org/document/6213315/
  27. 27.
    Zlokolica, V., Pekovic, V., Teslic, N., Tekcan, T., Temerinac, M.: Video freezing detection system for end-user devices. In: 2011 IEEE International Conference on Consumer Electronics (ICCE), IEEE, pp. 81–82 (2011).  https://doi.org/10.1109/ICCE.2011.5722883. http://ieeexplore.ieee.org/document/5722883/

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Ratko Grbić
    • 1
    Email author
  • Dejan Stefanović
    • 2
  • Mario Vranješ
    • 1
  • Marijan Herceg
    • 1
  1. 1.Faculty of Electrical EngineeringComputer Science and Information Technology OsijekOsijekCroatia
  2. 2.RT-RK Institute for Computer Based SystemsNovi SadSerbia

Personalised recommendations