Skip to main content
SpringerLink
Log in

Improved joint algorithms for reliable wireless transmission of 3D color-plus-depth multi-view video

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Error control techniques like Error Resilience (ER) and Error Concealment (EC) are preferred techniques to ameliorate the lost Macro-Blocks (MBs) in the 3D Video (3DV) communication systems. In this paper, we present different enhanced ER-EC algorithms for intra-frame images for 3DV and Depth (3DV + D) communication through wireless networks. At the encoder, the slice structured coding, explicit flexible macro-block ordering, and context adaptive variable length coding are utilized. At the decoder, a hybrid approach comprising spatial circular scan order interpolation algorithm and temporal partitioning motion compensation algorithm is suggested to reconstruct the Disparity Vectors (DVs) and Motion Vectors (MVs) of the erroneous color images. For the corrupted depth images, a depth-assisted EC algorithm is proposed. Then, the optimum concealment MVs and DVs are chosen by employing the weighted overlapping block motion and disparity compensation algorithm. Furthermore, the Bayesian Kalman Filter (BKF) is utilized as an amelioration tool due to its efficiency to smooth the remnant inherent corruptions in the formerly optimally chosen color and depth DVs and MVs to obtain a good video quality. Simulation results on several 3DV streams show that the suggested algorithms have extremely adequate subjective and objective video quality performance compared to the traditional methods, particularly at high Packet Loss Rates (PLRs).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abreu A, Frossard P, Pereira F (2015) Optimizing multiview video plus depth prediction structures for interactive multiview video streaming. IEEE Journal of Selected Topics in Signal Processing 9(3):487–500

    Article  Google Scholar 

  2. Assunçao P, Marcelino S, Soares S, Faria S (2016) Spatial error concealment for intra-coded depth maps in multiview video-plus-depth. Multimedia Tools and Applications 1–24

  3. Chakareski J (2013) Adaptive multiview video streaming: challenges and opportunities. IEEE Commun Mag 51(5):94–100

    Article  Google Scholar 

  4. Chen M, Chen L, Weng R (1997) Error concealment of lost motion vectors with overlapped motion compensation. IEEE Trans. on Circuits and Systems for Video Technology 7(3):560–563

    Article  Google Scholar 

  5. Chung T, Sull S, Kim C (2011) Frame loss concealment for stereoscopic video plus depth sequences. IEEE Trans Consumer Electronics 57(3):1336–1344

    Article  Google Scholar 

  6. Cui S, Huijuan C, Kun T (2012) An effective error concealment scheme for heavily corrupted H.264/AVC videos based on Kalman filtering. Journal of Signal, Image and Video Processing 8(8):1533–1542

    Article  Google Scholar 

  7. El-Shafai W (2015) Pixel-level matching based multi-hypothesis error concealment modes for wireless 3D H.264/MVC communication. 3D Res 6(3):31

    Article  Google Scholar 

  8. El-Shafai W (2015) Joint adaptive pre-processing resilience and post-processing concealment schemes for 3D video transmission. 3D Res 6(1):1–13

    Article  Google Scholar 

  9. El-Shafai W, Hrušovský B, El-Khamy M, El-Sharkawy M (2011) Joint space-time-view error concealment algorithms for 3D multi-view video. In: 18th IEEE Int. Conference on Image Processing (ICIP) pp 2201–2204

  10. El-Shafai W, El-Rabaie S, El-Halawany M, El-Samie F (2017) Encoder-independent decoder-dependent depth-assisted error concealment algorithm for wireless 3D video communication. Multimedia Tools and Applications 1–28

  11. Gao Z, Lie W (2004) Video error concealment by using Kalman-filtering technique. In: IEEE Int. Symposium on Circuits and Systems pp 69–72

  12. H.264/AVC codec http://iphome.hhi.de/suehring/tml/, accessed 28 December 2017

  13. Hewage C, Martini M (2013) Quality of experience for 3D video streaming. IEEE Commun Mag 51(5):101–107

    Article  Google Scholar 

  14. Huo Y, El-Hajjar M, Hanzo L (2013) Inter-layer FEC aided unequal error protection for multilayer video transmission in mobile TV. IEEE Trans. on Circuits and Systems for Video Technology 23(9):1622–1634

    Article  Google Scholar 

  15. Hwang M, Ko S (2008) Hybrid temporal error concealment methods for block-based compressed video transmission. IEEE Trans on Broadcasting 54(2):198–207

    Article  Google Scholar 

  16. Hwang M, Kim J, Duong D, Ko S (2008) Hybrid temporal error concealment methods for block-based compressed video transmission. IEEE Trans. on Broadcasting 54(2):198–207

    Article  Google Scholar 

  17. Ibrahim A, Sadka A (2014) Error resilience and concealment for multiview video coding. In: IEEE Int. Symposium on Broadband Multimedia Systems and Broadcasting pp 1–5

  18. ISO/IEC JTC1 Common test conditions for multiview video coding, (JVT-U207) pp 1–9

  19. Khattak S, Maugey T, Hamzaoui R, Ahmad S, Frossard P (2016) Temporal and inter-view consistent error concealment technique for multiview plus depth video. IEEE Trans. on Circuits and Systems for Video Technology 26(5):829–840

    Article  Google Scholar 

  20. Lee P, Kuo K, Chi C (2014) An adaptive error concealment method based on fuzzy reasoning for multi-view video coding. J Disp Technol 10(7):560–567

    Article  Google Scholar 

  21. Lie W, Lee C, Yeh C, Gao Z (2014) Motion vector recovery for video error concealment by using iterative dynamic-programming optimization. IEEE Transactions on Multimedia 16(1):216–227

    Article  Google Scholar 

  22. Liu Y, Wang J, Zhang H (2010) Depth image-based temporal error concealment for 3-d video transmission. IEEE Trans. Circuits and Systems for Video Technology 20(4):600–604

    Article  Google Scholar 

  23. Liu J, Zhang Y, Zheng X, Song J (2012) A dynamic hybrid UXP/ARQ method for scalable video transmission. In: IEEE 23rd Int. Symposium on Personal, Indoor and Mobile Radio Communications-(PIMRC) pp 2566–2571

  24. Liu Z, Cheung G, Ji Y (2013) Optimizing distributed source coding for interactive multiview video streaming over lossy networks. IEEE Trans on Circuits and Systems for Video Technology 23(10):1781–1794

    Article  Google Scholar 

  25. Liu Y, Nie L, Han L, Zhang L, Rosenblum D (2015) Action2Activity: Recognizing Complex Activities from Sensor Data. In: IJCAI pp 1617–1623

  26. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum D (2016) Fortune Teller: Predicting Your Career Path. In: AAAI pp 201–207

  27. Liu Y, Nie L, Liu L, Rosenblum D (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  28. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum D (2016) Urban water quality prediction based on multi-task multi-view learning. Google Scholar

  29. Purica A, Mora E, Pesquet-Popescu B, Cagnazzo M, Ionescu B (2016) Multiview plus depth video coding with temporal prediction view synthesis. IEEE Trans Circuits and Systems for Video Technology 26(2):360–374

    Article  Google Scholar 

  30. Salim O, Xiang W, Leis J (2013) An efficient unequal error protection scheme for 3-D video transmission. In: IEEE Wireless Communications and Networking Conf. (WCNC) pp 4077–4082

  31. Tai S, Wang C, Hong C, Luo Y (2016) An effiicient full frame algorithm for object-based error concealment in 3D depth-based video. Multimedia tools and applications 75(16):9927–9947

    Article  Google Scholar 

  32. Wang H, Wang X (2016) Important macroblock distinction model for multi-view plus depth video transmission over error-prone network. Multimedia Tools and Applications 1–23

  33. WD 4 reference software for multiview video coding (mvc) http://wftp3.itu.int/av-arch/jvt-site/2009_01_Geneva/JVT-AD207.zip, accessed 25 September 2017

  34. Xiang X, Zhao D, Wang Q, Ji X, Gao W (2007) A novel error concealment method for stereoscopic video coding. In: IEEE Int. Conf. on Image Processing pp 101–104

  35. Xiang W, Gao P, Peng Q (2015) Robust multiview three-dimensional video communications based on distributed video coding. IEEE Syst J 11(4):2456–2466

  36. Yan B, Zhou J (2012) Efficient frame concealment for depth image-based 3-d video transmission. IEEE Trans on Multimedia 14(3):936–941

    Article  Google Scholar 

  37. Zeng H, Wang X, Cai C, Chen J, Zhang Y (2015) Fast multiview video coding using adaptive prediction structure and hierarchical mode decision. IEEE Trans. Circuits and Systems for Video Technology 24(9):1566–1578

    Article  Google Scholar 

  38. Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  Google Scholar 

  39. Zhou Y, Xiang W, Wang G (2015) Frame loss concealment for multiview video transmission over wireless multimedia sensor networks. IEEE Sensors J 15(3):1892–1901

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Walid El-Shafai, EL-Sayed M. El-Rabaie, Mohamed Elhalawany & Fathi E. Abd El-Samie

Authors
  1. Walid El-Shafai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. EL-Sayed M. El-Rabaie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Elhalawany
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fathi E. Abd El-Samie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Walid El-Shafai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

El-Shafai, W., El-Rabaie, ES.M., Elhalawany, M. et al. Improved joint algorithms for reliable wireless transmission of 3D color-plus-depth multi-view video. Multimed Tools Appl 78, 9845–9875 (2019). https://doi.org/10.1007/s11042-018-6440-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6440-4

Keywords

Search

Navigation