Skip to main content
SpringerLink
Log in

Recursive Bayesian Filtering-Based Error Concealment Scheme for 3D Video Communication Over Severely Lossy Wireless Channels

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

The three-dimensional video (3DV) is composed of variable-length stream sequences captured via diversified cameras surrounding an object. Thus, it is an urgent task to accomplish sufficient encoding to be compatible with incoming bandwidth demands, while achieving a recommended 3DV reception performance. In the 3DV compression framework, the lost macro-blocks (MBs) might propagate into the following frames and the adjoining views. Therefore, it is obligatory to avoid error propagation by concealing the corrupted MBs at the decoder through the utilization of appropriate post-processing error concealment (EC) techniques. The existing EC algorithms fundamentally exploit the temporal, inter-view, and spatial matching within the 3DV frames and views to reconstruct the disparity vectors (DVs) and motion vectors (MVs) of the corrupted MBs. Unluckily, in the state of high severe corruptions and heavily erroneous MBs, these concealment algorithms are predominantly unreliable and might give unreliable 3DV quality. Thence, in this work, we suggest the utilization of the outer block boundary matching algorithm to estimate the MVs and the directional interpolation EC algorithm to estimate the DVs of the erroneous MBs. After that, the Bayesian Kalman filter (BKF) is employed because of its efficiency to filter out the inherent errors in the previously predicted DVs and MVs to accomplish better 3D video performance. Experimental results on standard 3DV sequences demonstrate that the suggested BKF-based EC scheme is more powerful with heavy losses. It subjectively and objectively outperforms the traditional concealment techniques at severely random and bursty packet loss rates (PLRs).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

References

  1. O. Cagri, E. Erhan, C. Janko, K. Ahmet, Adaptive delivery of immersive 3D multi-view video over the internet. J. Multimed. Tools Appl. 75(20), 12431–12461 (2016)

    Article  Google Scholar 

  2. J. Chakareski, Adaptive multiview video streaming: challenges and opportunities. IEEE Commun. Mag. 51(5), 94–100 (2013)

    Article  Google Scholar 

  3. J. Chen, C. Cai, Motion consistence and textural coherence based error concealment algorithm for corrupted macroblock. Trans. Tech. Publ. Adv. Mater. Res. 11(383), 1605–1610 (2012)

    Google Scholar 

  4. Y. Chen, A. Vetro, Next generation 3D formats with depth map support. IEEE Multimed. 21(2), 90–94 (2014)

    Article  Google Scholar 

  5. Y. Chung, L. Chen, X. Chen, W. Yeh, C. Bae, A novel intra-fame error concealment algorithm for h. 264 avc, in Proceedings of Third International Conference on Convergence and Hybrid Information Technology (ICCIT), (2008), pp. 881–886

  6. P.H. Dhangare, S.K. Jagtap, A novel approach: multiview video compression, in Proceedings of International Conference on Recent Advances and Innovations in Engineering (ICRAIE), (2014), pp. 1–6

  7. W. El-Shafai, Joint adaptive pre-processing resilience and post-processing concealment schemes for 3D video transmission. 3D Res. 6(1), 1–13 (2015)

    Article  Google Scholar 

  8. W. El-Shafai, Pixel-level matching based multi-hypothesis error concealment modes for wireless 3D H.264/MVC communication. 3D Res. 6(3), 31 (2015)

    Article  Google Scholar 

  9. W. El-Shafai, S. El-Rabaie, M. El-Halawany, F. A. El-Samie, Enhancement of wireless 3d video communication using color-plus-depth error restoration algorithms and Bayesian Kalman filtering. Wirel. Personal Commun. 79(1), 245–268 (2017)

    Article  Google Scholar 

  10. W. El-Shafai, S. El-Rabaie, M. M. El-Halawany, F. E. A. El-Samie, Encoder-independent decoder-dependent depth-assisted error concealment algorithm for wireless 3D video communication. Multimed. Tools Appl. (2017). https://doi.org/10.1007/s11042-017-4936-y

    Article  Google Scholar 

  11. W. El-Shafai, E. El-Rabaie, F.A. El-Samie, M. El-Halawany, Proposed adaptive joint error-resilience concealment algorithms for efficient color-plus-depth 3D video transmission. IET Image Process. (2018). https://doi.org/10.1049/iet-ipr.2016.1091

    Article  Google Scholar 

  12. P. Gao, Q. Peng, Q. Wang, Error-resilient multi-view video coding based on end-to-end rate-distortion optimization. Chin. J. Electron. 25(2), 277–283 (2016)

    Article  Google Scholar 

  13. Z.W. Gao, W.N. Lie, Video error concealment by using Kalman-filtering technique, in Proceedings of the International Symposium on Circuits and Systems, (2004), pp. 69–72

  14. J. Guo, H. Bai, C. Lin, M. Zhang, Y. Zhao, Intra-/inter-view correlation based multiple description coding for multiview transmission, in Proceedings of IEEE International Data Compression Conference, (2015), pp. 446–446

  15. http://iphome.hhi.de/suehring/tml/ (2010). Accessed 28 Sep. 2015

  16. H. Huang, C.T. Yong, An efficient error concealment algorithm for intra-frames of H.264, in Proceedings of IEEE International Conference on Communication Technology (ICCT), (2010), pp. 576–579

  17. M. Hwang, S. Ko, Hybrid temporal error concealment methods for block-based compressed video transmission. IEEE Trans. Broadcasting 54(2), 198–207 (2008)

    Article  Google Scholar 

  18. ISO/IEC JTC1/SC29/WG11, Common test conditions for multiview video coding, JVT-U207, Hangzhou, China, (2006)

  19. S. Khattak, T. Maugey, R. Hamzaoui, S. Ahmad, P. Frossard, Temporal and inter-view consistent error concealment technique for multiview plus depth video. IEEE Trans. Circuits Syst. Video Technol. 26(5), 829–840 (2016)

    Article  Google Scholar 

  20. D. H. Kim, J. H. Won, K. H. Choi, Video error concealment for in-vehicle IP-based wireless networks, in Proceedings of IEEE 10th International Asian Control Conference (ASCC), (2015), pp. 1–4

  21. P.T. Kovacs, Z. Nagy, A. Barsi, V.K. Adhikarla, R. Bregovic, Overview of the applicability of h.264/mvc for real-time light-field applications, in Proceedings of The True Vision-Capture, Transmission and Display of 3D Video Conference (3DTV-CON), (2014), pp. 1–4

  22. Y.K. Kuan, G.L. Li, M.J. Chen, K.H. Tai, P.C. Huang, Error concealment algorithm using inter-view correlation for multi-view video. EURASIP J. Image Video Process. 20(1), 38 (2014)

    Article  Google Scholar 

  23. P.J. Lee, K.T. Kuo, C.Y. Chi, An adaptive error concealment method based on fuzzy reasoning for multi-view video coding. J. Display Technol. 10(7), 560–567 (2014)

    Article  Google Scholar 

  24. J.Y. Lee, H.W. Park, Efficient synthesis-based depth map coding in AVC-compatible 3D video coding. IEEE Trans. Circuits Syst. Video Technol. 26(6), 1107–1116 (2016)

    Article  Google Scholar 

  25. Z. Liu, G. Cheung, Y. Ji, Optimizing distributed source coding for interactive multiview video streaming over lossy networks. IEEE Trans. Circuits Syst. Video Technol. 23(10), 1781–1794 (2013)

    Article  Google Scholar 

  26. S. Marcelino, P. Assuncao, S. M. M. Faria, S. Soares, Efficient depth error concealment for 3D video over error-prone channels, in Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), (2013), pp. 1–5

  27. B.W. Micallef, C.J. Debono, Error concealment techniques for multi-view video, in Proceedings of IEEE Wireless Days (IFIP WD), (2010), pp. 1–5‏

  28. J. Mochn, S. Marchevsk, J. Gamec, Kalman filter based error concealment algorithm, in Proceedings of 54th Internationales Wissenchaftliches Kolloquium, (2009), pp. 1–4

  29. H. Mohib, M.R. Swash, A.H. Sadka, Multi-view video delivery over wireless networks using HTTP, in Proceedings of 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA), (2013), pp. 1–5

  30. A. Purica, E. Mora, B.P. Popescu, M. Cagnazzo, B. Ionescu, Multiview plus depth video coding with temporal prediction view synthesis. IEEE Trans. Circuits Syst. Video Technol. 26(2), 360–374 (2016)

    Article  Google Scholar 

  31. N. Ramzan, A. Amira, C. Grecos, Efficient transmission of multiview video over unreliable channels, in Proceedings of IEEE International Conference on Image Processing (ICIP), (2013), pp. 1885–1889

  32. O. Salim, X. Wei, J. Leis, An efficient unequal error protection scheme for 3-D video transmission, in Proceedings of IEEE International Conference on Wireless Communications and Networking Conference (WCNC), (2013), pp. 4077–4082

  33. C. Shihua, H. Cui, K. Tang, An effective error concealment scheme for heavily corrupted h. 264/avc videos based on kalman filtering. Signal, Image and Video Procss. 8(8), 1533–1542 (2014)

    Article  Google Scholar 

  34. A. Smolic, K. Mueller, N. Stefanoski, J. Ostermann, A. Gotchev, G.B. Akar, G. Triantafyllidis, A. Koz, Coding algorithms for 3DTV-a survey. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1606–1621 (2007)

    Article  Google Scholar 

  35. K. Song, T. Chung, Y. Oh, C. Kim, Error concealment of multi-view video sequences using inter-view and intra-view correlations. J. Vis. Commun. Image Represent. 20(4), 281–292 (2009)

    Article  Google Scholar 

  36. WD 4 Reference Software for Multiview Video Coding (MVC). (2009), http://wftp3.itu.int/av-arch/jvt-site/2009_01_Geneva/JVT-AD207.zip. Accessed 25 Aug 2016

  37. K.S. Whan, K.H. Ko, Kalman filter based dead reckoning algorithm for minimizing network traffic between mobile nodes in wireless GRID, in Proceedings of 9th Pacific Rim International Conference on Artificial Intelligence Guilin, (2006), ‏pp. 61–70

  38. W. Xiang, P. Gao, Q. Peng, Robust multiview three-dimensional video communications based on distributed video coding. IEEE Syst. J. 99, 1–11 (2015)

    Google Scholar 

  39. H. Yongkai, M. El-Hajjar, L. Hanzo, Inter layer FEC aided unequal error protection for multilayer video transmission in mobile TV. IEEE Trans. Circuits Syst. Video Technol. 23(9), 1622–1634 (2013)

    Article  Google Scholar 

  40. H. Zeng, X. Wang, C. Cai, J. Chen, Y. Zhang, Fast multiview video coding using adaptive prediction structure and hierarchical mode decision. IEEE Trans. Circuits Syst. Video Technol. 24(9), 1566–1578 (2014)

    Article  Google Scholar 

  41. L. Zhang, L. Zhang, X. Mou, D. Zhang, FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    Article  MathSciNet  Google Scholar 

  42. Y. Zhou, W. Xiang, G. Wang, Frame loss concealment for multiview video transmission over wireless multimedia sensor networks. IEEE Sens. J. 15(3), 1892–1901 (2015)

    Article  Google Scholar 

  43. C. Zhu, S. Li, J. Zheng, Y. Gao, L. Yu, Texture-aware depth prediction in 3D video coding. IEEE Trans. Broadcast. 62(2), 482–486 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    W. El-Shafai, S. El-Rabaie, M. M. El-Halawany & Fathi E. Abd El-Samie

Authors
  1. W. El-Shafai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. El-Rabaie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. M. El-Halawany
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fathi E. Abd El-Samie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to W. El-Shafai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

El-Shafai, W., El-Rabaie, S., El-Halawany, M.M. et al. Recursive Bayesian Filtering-Based Error Concealment Scheme for 3D Video Communication Over Severely Lossy Wireless Channels. Circuits Syst Signal Process 37, 4810–4841 (2018). https://doi.org/10.1007/s00034-018-0786-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-018-0786-8

Keywords

Search

Navigation