Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Enhancement of Wireless 3D Video Communication Using Color-Plus-Depth Error Restoration Algorithms and Bayesian Kalman Filtering

  • 116 Accesses

  • 3 Citations

Abstract

This paper proposes a hybrid of Outer Block Boundary Matching Algorithm and Directional Interpolation Error Concealment Algorithm (DIECA) to recover the Motion Vectors (MVs) and the Disparity Vectors (DVs) of the lost color frames of the transmitted Three-Dimensional Video (3DV). For the lost 3DV depth frames, an Encoder-Independent Decoder-Dependent Depth-Assisted Error Concealment (EIDD-DAEC) algorithm is proposed. It exploits the recovered color MVs and DVs to estimate more additional concealment depth-assisted MVs and DVs. After that, the initially-estimated concealment candidate DVs and MVs are selected from all previously-predicted ones using the DIECA and the Decoder Motion Vector Estimation Algorithm (DMVEA). Finally, the proposed Bayesian Kalman Filtering (BKF) scheme is efficiently employed to filter out the inherent errors inside the selected concealment candidate color-plus-depth MVs and DVs to achieve better 3DV quality. Extensive experimental results on different standardized 3DV sequences demonstrate that the proposed color-plus-depth schemes are more robust against heavy losses and they achieve high 3DV quality performance with an improved average Peak Signal-to-Noise Ratio (PSNR) gain. They objectively and subjectively outperform the state-of-the-art error recovery techniques, especially at severe Packet Loss Rates (PLRs).

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  1. 1.

    Xiang, W., Gao, P., & Peng, Q. (2015). Robust multiview three-dimensional video communications based on distributed video coding. IEEE Systems Journal, 99, 1–11.

  2. 2.

    Cagri, O., Erhan, E., Janko, C., & Ahmet, K. (2016). Adaptive delivery of immersive 3D multi-view video over the Internet. Journal of Multimedia Tools and Applications, 75(20), 12431–12461.

  3. 3.

    Huanqiang, Z., Xiaolan, W., Canhui, C., Jing, C., & Yan, Z. (2014). Fast multiview video coding using adaptive prediction structure and hierarchical mode decision. IEEE Transactions on Circuits and Systems for Video Technology, 24(9), 1566–1578.

  4. 4.

    Ying, C., & Vetro, A. (2014). Next generation 3D formats with depth map support. IEEE Multimedia, 21(2), 90–94.

  5. 5.

    Purica, A., Mora, E., Pesquet, P. B., Cagnazzo, M., & Ionescu, B. (2016). Multiview plus depth video coding with temporal prediction view synthesis. IEEE Transactions on Circuits and Systems for Video Technology, 26(2), 360–374.

  6. 6.

    Abreu, A. D., Frossard, P., & Pereira, F. (2015). Optimizing multiview video plus depth prediction structures for interactive multiview video streaming. IEEE Journal of Selected Topics in Signal Processing, 9(3), 487–500.

  7. 7.

    Hewage, C. T. E. R., & Martini, M. G. (2013). Quality of experience for 3D video streaming. IEEE Communications Magazine, 51(5), 101–107.

  8. 8.

    Liu, Z., Cheung, G., & Ji, Y. (2013). Optimizing distributed source coding for interactive multiview video streaming over lossy networks. IEEE Transactions on Circuits and Systems for Video Technology, 23(10), 1781–1794.

  9. 9.

    El-Shafai, W. (2015). Pixel-level matching based multi-hypothesis error concealment modes for wireless 3D H.264/MVC communication. 3D Research, 6(3), 31.

  10. 10.

    Khattak, S., Maugey, T., Hamzaoui, R., Ahmad, S., & Frossard, P. (2016). Temporal and inter-view consistent error concealment technique for multiview plus depth video. IEEE Transactions on Circuits and Systems for Video Technology, 26(5), 829–840.

  11. 11.

    Zhou, Y., Xiang, W., & Wang, G. (2015). Frame loss concealment for multiview video transmission over wireless multimedia sensor networks. IEEE Sensors Journal, 15(3), 1892–1901.

  12. 12.

    Lee, P. J., Kuo, K. T., & Chi, C. Y. (2014). An adaptive error concealment method based on fuzzy reasoning for multi-view video coding. Journal of Display Technology, 10(7), 560–567.

  13. 13.

    Xiang, X., Zhao, D., Wang, Q., Ji, X., & Gao, W. (2007). A novel error concealment method for stereoscopic video coding. In Proceedings 2007 IEEE international conference on image processing (ICIP) (pp. 101–104).

  14. 14.

    Hwang, M., & Ko, S. (2008). Hybrid temporal error concealment methods for block-based compressed video transmission. IEEE Transactions on Broadcasting, 54(2), 198–207.

  15. 15.

    Lie, W. N., Lee, C. M., Yeh, C. H., & Gao, Z. W. (2014). Motion vector recovery for video error concealment by using iterative dynamic-programming optimization. IEEE Transactions on Multimedia, 16(1), 216–227.

  16. 16.

    Gadgil, N., Li H., & Delp, E. J. (2015). Spatial subsampling-based multiple description video coding with adaptive temporal-spatial error concealment. In Proceedings 2015 IEEE picture coding symposium (PCS) (pp. 90–94).

  17. 17.

    Ebdelli, M., Le-Meur, O., & Guillemot, C. (2015). Video inpainting with short-term windows: application to object removal and error concealment. IEEE Transactions on Image Processing, 24(10), 3034–3047.

  18. 18.

    Yan, B., & Jie, Z. (2012). Efficient frame concealment for depth image-based 3-D video transmission. IEEE Transactions on Multimedia, 14(3), 936–941.

  19. 19.

    Gao, Z. W., & Lie, W. N. (2004). Video error concealment by using Kalman-filtering technique. In Proceedings of the international symposium on circuits and systems (pp. 69–72).

  20. 20.

    Mochn, J., Marchevsk, S., & Gamec, J. (2009). Kalman filter based error concealment algorithm. In Proceedings of 54th Internationales Wissenchaftliches Kolloquium (pp. 1–4).

  21. 21.

    Shihua, C., Cui, H., & Tang, K. (2014). An effective error concealment scheme for heavily corrupted H.264/AVC videos based on Kalman filtering. Signal, Image and Video Processing, 8(8), 1533–1542.

  22. 22.

    Wen, N. L., & Guan, H. L. (2013). Error concealment for 3D video transmission. In Proceedings of IEEE international symposium on circuits and systems (ISCAS) (pp. 2856–2559).

  23. 23.

    Liu, Y., Wang, J., & Zhang, H. (2010). Depth image-based temporal error concealment for 3-D video transmission. IEEE Transactions on Circuits and Systems for Video Technology, 20(4), 600–604.

  24. 24.

    Chung, T. Y., Sull, S., & Kim, C. S. (2011). Frame loss concealment for stereoscopic video plus depth sequences. IEEE Transactions on Consumer Electronics, 57(3), 1336–1344.

  25. 25.

    Hong, C. S., Wang, C. C., Tai, S. C., & Luo, Y. C. (2011). Object-based error concealment in 3D video. In Proceedings of IEEE fifth international conference on genetic and evolutionary computing (ICGEC) (pp. 5–8).

  26. 26.

    H.264/AVC codec reference software. http://iphome.hhi.de/suehring/tml/. Accessed 28 September 2014.

  27. 27.

    ISO/IEC JTC1/SC29/WG11. (2006). Common test conditions for multiview video coding. JVT-U207, Hangzhou, China.

  28. 28.

    WD 4 reference software for multiview video coding (MVC). http://wftp3.itu.int/av-arch/jvt-site/2009_01_Geneva/JVT-AD207.zip. Accessed 25 October 2015.

Download references

Author information

Correspondence to W. El-Shafai.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

El-Shafai, W., El-Rabaie, S., El-Halawany, M. et al. Enhancement of Wireless 3D Video Communication Using Color-Plus-Depth Error Restoration Algorithms and Bayesian Kalman Filtering. Wireless Pers Commun 97, 245–268 (2017). https://doi.org/10.1007/s11277-017-4503-x

Download citation

Keywords

  • 3D video
  • Error concealment
  • Bayesian Kalman filter
  • Lossy macro-blocks
  • Wireless channels