Skip to main content
SpringerLink
Log in

Real-time video freezing detection for 4K UHD videos

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Video frame freezing is a common artifact which can occur during video content delivery due to errors in the video coding process, video transmission, storage or reproduction. This artifact can significantly decrease the end-user Quality of Experience. Therefore, accurate video frame freezing detection is of great importance for different parties involved in video content delivery. In this paper, a new Real-Time no-reference Freezing Detection Algorithm, called the RTFDA, is proposed. As newly video frames are acquired, the RTFDA performs comparison of the current video frame with the corresponding previous one to detect whether video freezing is occurring. The comparison is made by calculating the corresponding subsampled video frames and their pixel-by-pixel absolute difference comparison. The benefit of such approach is twofold: the influence of noise in freezing frames on frame comparison is significantly reduced as well as computational complexity of frame comparison. The RTFDA has a high detection rate with a very low rate of both false-positive and false-negative detections, outperforming four freezing detection algorithms on four different video databases. The proposed implementation on an x86-64 platform achieves real-time performance on 4K Ultra High Definition (UHD) videos by processing 216 frames per second (fps). Apart from that, FPGA implementation is proposed, which has efficient FPGA resource utilization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Babić, D., Vranješ, M.: FERIT-RTRK UHD VFD. http://www.rt-rk.com/other/UHDVideoDBReadme.html (2018)

  2. Babić, D., Stefanović, D., Vranješ, M., Herceg, M.: Real-time no-reference histogram-based freezing artifact detection algorithm for UHD videos. Multimed. Tools Appl. (2019). https://doi.org/10.1007/s11042-019-7184-5

    Article  Google Scholar 

  3. Baik, E., Pande, A., Stover, C., Mohapatra, P.: Video acuity assessment in mobile devices. In: 2015 IEEE Conference on Computer Communications (INFOCOM), IEEE, vol. 26, pp. 1–9. https://doi.org/10.1109/INFOCOM.2015.7218361. http://ieeexplore.ieee.org/document/7218361/ (2015)

  4. Barkowsky, M., Sedano, I., Brunnström, K., Leszczuk, M., Staelens, N.: Hybrid video quality prediction: reviewing video quality measurement for widening application scope. Multimed. Tools Appl. 74(2), 323–343 (2015). https://doi.org/10.1007/s11042-014-1978-2

    Article  Google Scholar 

  5. Black Box Testing—BBT: RT–AV4K Network attached 4K UHD Audio/Video capture device. http://bbt.rt-rk.com/hardware/rt-av4k/ (2018)

  6. Borer, S.: A model of jerkiness for temporal impairments in video transmission. In: 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX), IEEE, pp. 218–223. https://doi.org/10.1109/QOMEX.2010.5516155. http://ieeexplore.ieee.org/document/5516155/ (2010)

  7. Cabello, F., León, J., Iano, Y., Arthur, R.: Implementation of a fixed-point 2D Gaussian filter for image processing based on FPGA. In: 2015 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA), pp. 28–33. (2015). https://doi.org/10.1109/SPA.2015.7365108

  8. Chen, M.J., Bovik, A.C.: Fast structural similarity index algorithm. J. Real Time Image Process. 6(4), 281–287 (2011). https://doi.org/10.1007/s11554-010-0170-9

    Article  Google Scholar 

  9. Cisco: The Zettabyte Era: Trends and Analysis. https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/vni-hyperconnectivity-wp.html (2017)

  10. HajiRassouliha, A., Taberner, A.J., Nash, M.P., Nielsen, P.M.: Suitability of recent hardware accelerators (DSPs, FPGAs, and GPUs) for computer vision and image processing algorithms. Signal Process. Image Commun. 68, 101–119 (2018). https://doi.org/10.1016/j.image.2018.07.007. http://www.sciencedirect.com/science/article/pii/S0923596518303606

  11. Hong, B.H., Joo, H.J., Lee, E.S.: Quality analysis of real-time digital broadcasting images applying the Qoe measurement technology. J. Real Time Image Process. 9(3), 579–585 (2014). https://doi.org/10.1007/s11554-014-0398-x

    Article  Google Scholar 

  12. Huynh-Thu, Q., Ghanbari, M.: No-reference temporal quality metric for video impaired by frame freezing artefacts. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2221–2224 (2009). https://doi.org/10.1109/ICIP.2009.5413894

  13. International Telecommunication Union: Recommendation ITU-R BT.2020-2. https://www.itu.int/rec/R-REC-BT.2020/en (2015)

  14. van Kester, S., TXiao, Kooij R.E., Brunnstrom, K., Ahmed. O.K.: Estimating the impact of single and multiple freezes on video quality, vol. 7865, pp. 7865–10 (2011). https://doi.org/10.1117/12.873390

  15. Mitra, G., Johnston, B., Rendell, A.P., McCreath, E., Zhou, J.: Use of SIMD vector operations to accelerate application code performance on low-powered ARM and intel platforms. In: 2013 IEEE International Symposium on Parallel AND Distributed Processing, Workshops and Phd Forum, IEEE, pp. 1107–1116 (2013). https://doi.org/10.1109/IPDPSW.2013.207. http://ieeexplore.ieee.org/document/6650996/

  16. Moorthy, A.K., Choi, L.K., Bovik, A.C., de Veciana, G.: Video quality assessment on mobile devices: subjective, behavioral and objective studies. IEEE J. Sele. Top. Signal Process. 6(6), 652–671 (2012). https://doi.org/10.1109/JSTSP.2012.2212417. http://ieeexplore.ieee.org/document/6263265/

  17. Pinson, M.H., Choi, L.K., Bovik, A.C.: Temporal video quality model accounting for variable frame delay distortions. IEEE Trans. Broadcast. 60(4), 637–649 (2014). https://doi.org/10.1109/TBC.2014.2365260. http://ieeexplore.ieee.org/document/6954510/

  18. Usman, M.A., Usman, M.R., Shin, Soo Young: A no reference method for detection of dropped video frames in live video streaming. In: 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN), IEEE, vol. 2016-August, pp. 839–844 (2016). https://doi.org/10.1109/ICUFN.2016.7537155. http://ieeexplore.ieee.org/document/7537155/

  19. Usman, M.A., Shin, S.Y., Shahid, M., Lövström, B.: A no reference video quality metric based on jerkiness estimation focusing on multiple frame freezing in video streaming. IETE Tech. Rev. 34(3), 309–320 (2017). https://doi.org/10.1080/02564602.2016.1185975

    Article  Google Scholar 

  20. Usman, M.A., Usman, M.R., Shin, S.Y.: A novel no-reference metric for estimating the impact of frame freezing artifacts on perceptual quality of streamed videos. IEEE Trans. Multimed. (2018). https://doi.org/10.1109/TMM.2018.2801722

    Article  Google Scholar 

  21. Škobić, V., Marić, U., Tomić, S., Rešetar, I.: Real time video freezing detection implementation on FPGA. In: 2016 24th Telecommunications Forum (TELFOR), IEEE, pp. 1–3 (2016). https://doi.org/10.1109/TELFOR.2016.7818805. http://ieeexplore.ieee.org/document/7818805/

  22. Wang, Z., Sheikh, H.R., Bovik, A.C.: Objective video quality assessment. The Handbook of Video Databases: Design and Applications, pp. 1041–1078. CRC Press, Boca Raton (2003). https://doi.org/10.1117/1.2160515

    Chapter  Google Scholar 

  23. Webster, A., Speranza, F.: VQEG report on the validation of video quality models for high definition video content. Technical report, Video Quality Experts Group (2010). ftp://vqeg.its.bldrdoc.gov/HDTV/VQEG_HDTV_Final_Report_version_2.0.pdf

  24. Wolf, S.: A no reference (NR) and reduced reference (RR) metric for detecting dropped video frames. In: Fourth International Workshop on Video Processing and Quality Metrics for Consumer Electronics, pp. 15–16 (2009)

  25. Xue, Y., Erkin, B., Wang, Y.: A novel no-reference video quality metric for evaluating temporal jerkiness due to frame freezing. IEEE Trans. Multimed. 17(1), 134–139 (2015). https://doi.org/10.1109/TMM.2014.2368272. http://ieeexplore.ieee.org/document/6949688/

  26. Yammine, G., Wige, E., Simmet, F., Niederkorn, D., Kaup, A.: Blind frame freeze detection in coded videos. In: 2012 Picture Coding Symposium, IEEE, pp. 341–344 (2012). https://doi.org/10.1109/PCS.2012.6213315. http://ieeexplore.ieee.org/document/6213315/

  27. Zlokolica, V., Pekovic, V., Teslic, N., Tekcan, T., Temerinac, M.: Video freezing detection system for end-user devices. In: 2011 IEEE International Conference on Consumer Electronics (ICCE), IEEE, pp. 81–82 (2011). https://doi.org/10.1109/ICCE.2011.5722883. http://ieeexplore.ieee.org/document/5722883/

Download references

Acknowledgements

This work was supported by Josip Juraj Strossmayer University of Osijek business fund through its internal call for proposals for research and artistic projects “UNIOS ZUP-2018” and by the Ministry of Education, Science and Technological Development of the Republic of Serbia, under grant number: TR32041. We would like to thank Velibor Škobić, MSc, for his valuable help and comments regarding the FPGA implementation of the proposed freezing detection algorithm.

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, Osijek, Croatia

    Ratko Grbić, Mario Vranješ & Marijan Herceg

  2. RT-RK Institute for Computer Based Systems, Narodnog fronta 23a, Novi Sad, Serbia

    Dejan Stefanović

Authors
  1. Ratko Grbić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dejan Stefanović
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Vranješ
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marijan Herceg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ratko Grbić.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Grbić, R., Stefanović, D., Vranješ, M. et al. Real-time video freezing detection for 4K UHD videos. J Real-Time Image Proc 17, 1211–1225 (2020). https://doi.org/10.1007/s11554-019-00873-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00873-y

Keywords

Search

Navigation