Skip to main content
SpringerLink
Log in

Rate-distortion analysis of multiview coding in a DIBR framework

  • Published:
annals of telecommunications - annales des télécommunications Aims and scope Submit manuscript

Abstract

Depth image-based rendering techniques for multiview applications have been recently introduced for efficient view generation at arbitrary camera positions. The rate control in an encoder has thus to consider both texture and depth data. However, due to different structures of depth and texture data and their different roles on the rendered views, the allocation of the available bit budget between them requires a careful analysis. Information loss due to texture coding affects the value of pixels in synthesized views, while errors in depth information lead to a shift in objects or to unexpected patterns at their boundaries.In this paper, we address the problem of efficient bit allocation between texture and depth data of multiview sequences.We adopt a rate-distortion framework based on a simplified model of depth and texture images, which preserves the main features of depth and texture images. Unlike most recent solutions, our method avoids rendering at encoding time for distortion estimation so that the encoding complexity stays low. In addition to this, our model is independent of the underlying inpainting method that is used at the decoder for filling holes in the synthetic views. Extensive experiments validate our theoretical results and confirm the efficiency of our rate allocation strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. The extension of our analysis to the scenes with \(C^{\alpha }\) regular surfaces are straightforward.

  2. In this paper, we consider the \(\ell _{2}\) distortion. However, extensions to other error norms are straightforward.

References

  1. Zhang Z (2012) Microsoft kinetic sensor and its effect. IEEE Multimed 19:4–10

    Article  Google Scholar 

  2. Merkle P, Smolic A, Muller K, Wiegand T (2007) Efficient prediction structures for multiview video coding. IEEE Trans Circ Syst Video Technol 17(11):1461–1473

    Article  Google Scholar 

  3. Vetro A, Wiegand T, Sullivan G (2011) Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standards. Proc IEEE 99(4):626–642

    Article  Google Scholar 

  4. Müller K, Merkle P, Wiegand T (2011) 3D video representation using depth maps. Proc IEEE 99(4):643–656

    Article  Google Scholar 

  5. Tian D, Lai P, Lopez P, Gomila C (2009) View synthesis techniques for 3D video SPIE Optical Engineering + Applications. In: International society for optics and photonics, pp 74430T–74430T

  6. Fehn C (2004) Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV. Proc SPIE Stereosc Image Process Render 5291:93–104

    Google Scholar 

  7. Shao F, Jiang GY, Yu M, Zhang Y (2011) Object-based depth image-based rendering for a three-dimensional video system by color-correction optimization. Opt Eng 50:047006−047006−10

    Article  Google Scholar 

  8. Oh K-J, Yea S, Ho Y-S (2009) Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3-D video. In: Proceedings of picture coding symposium. Chicago

  9. Cheng C-M, Lin S-J, Lai S-H, Yang J-C (2008) Improved novel view synthesis from depth image with large baseline. In: Proceedings of international conference on pattern recognition. Tampa

  10. Merkle P, Morvan Y, Smolic A, Farin D, Muller K, de With PHN, Wiegand T (2009) The effects of multiview depth video compression on multiview rendering. Signal Processing: Image Commun 24:73–88

    Article  Google Scholar 

  11. Maitre M, Do MN (2010) Depth and depth-color coding using shape-adaptive wavelets. J Vis Commun Image Repr 21:513–522

    Article  Google Scholar 

  12. Donoho DL (1999) Wedgelets: nearly minimax estimation of edges. Ann Stat 27:859–897

    Article  MathSciNet  MATH  Google Scholar 

  13. Le Pennec E, Mallat S (2005) Sparse geometrical image approximation with bandelets. IEEE Trans Image Process 14(4):423–438

    Article  MathSciNet  Google Scholar 

  14. Maleki A, Rajaei B, Pourreza HR (2012) Rate-distortion analysis of directional wavelets. IEEE Trans Image Process 21(2):588–600

    Article  MathSciNet  Google Scholar 

  15. Sanchez A, Shen G, Ortega A (2009) Edge-preserving depth-map coding using graph-based wavelets. In: Proceedings asilomar conference signals, systems computers.Los Angeles, pp 578–582

  16. Daribo I, Tillier C, Pesquet-Popescu B (2008) Adaptive wavelet coding of the depth map for stereoscopic view synthesis. In: IEEE proceedings of international workshop on multimedia signal processing.Paris, pp 413–417

  17. Milani S, Zanuttigh P, Zamarin M, Forchhammer S (2011) Efficient depth map compression exploiting segmented color data. In: Proceedings of international conference on multimedia and expo. pp 1–6

  18. ITU-T and ISO/IEC JTC 1Advanced video coding for generic audiovisual services TU-T recommendation. ITU, ITU Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4 AVC), Version 1: May 2003; Version 2: May 2004; Version 3: Mar. 2005 (including FRExt extension); Version 4: Sep. 2005; Version 5 and Version 6: Jun. 2006; Version 7: Apr. 2007; Version 8: Jul. 2007 (including SVC extension); Version 9: Jul. 2009 (including MVC extension), Switzerland

  19. ISO/IEC JTC1/SC29/WG11 (2008) Text of ISO/IEC 14496-10:200X/ FDAM 1 multiview video coding. Doc. N9978. ISO/IEC, Hannover

  20. Ekmekcioglu E, Velisavljevic V, Worrall ST (2011) Content adaptive enhancement of multi-view depth maps for free viewpoint video. IEEE J Selected Topics Sig Proc 5(2):352–361

    Article  Google Scholar 

  21. Lee JY, Wey H-C, Park D-S (2011) A fast and efficient multi-view depth image coding method based on temporal and inter-view correlations of texture images. IEEE Trans Circ Syst Video Technol 21(12):1859–1868

    Article  Google Scholar 

  22. Liu Y, Ma S, Huang Q, Zha D, Gao W, Zhang N (2009) Compression-induced rendering distortion analysis for texture/depth rate allocation in 3d video compression. In: Proceedings data compression conference.Beijing, pp 352–361

  23. Nguyen HT, Do MN (2009) Error analysis for image-based rendering with depth information. IEEE Image Proc 18(4):703–716

    Article  MathSciNet  Google Scholar 

  24. Kim W-S, Ortega A, Lai P, Tian D, Gomila C (2010) Depth map coding with distortion estimation of rendered view. Proc SPIE Visual Inf Proc Commun 7543:75430B−75430B−10

    Google Scholar 

  25. Oh BT, Lee J, Park D-S (2011) Depth map coding based on synthesized view distortion function. IEEE J Selected Topics Sig Proc 5(7):1344–1352

    Article  Google Scholar 

  26. Davidoiu V, Maugey T, Pesquet-Popescu B, Frossard P (2011) Rate distortion analysis in a disparity compensated scheme. In: Proceedings IEEE international conference acoustics, speech, and signal processing. Paris, pp 857–860

  27. Maitre M, Do MN (2008) Joint encoding of the depth image based representation using shape-adaptive wavelets. In: Proceedings IEEE international conference image processing (ICIP). Urbana, pp 1768–1771

  28. Wang Q, Ji X, Dai Q, Zhang N (2012) Free viewpoint video coding with rate-distortion analysis. IEEE Trans Circ Syst Video Technol 22(6):875–889

    Article  Google Scholar 

  29. Cheung G, Velisavljevic V, Ortega A (2011) On dependent bit allocation for multiview image coding with depth-image-based rendering. IEEE Trans Image Proc 20(11):3179–3194

    Article  MathSciNet  Google Scholar 

  30. Gelman A, Dragotti PL, Velisavljevic V (2012) Multiview image coding using depth layers and an optimized bit allocation. IEEE Trans Image Proc 21(9):4092–4105

    Article  MathSciNet  Google Scholar 

  31. Mallat S (1997) A wavelet tour of signal processing. Academic, San Diego

    Google Scholar 

  32. Prandoni P, Vetterli M (1999) Approximation and compression of piecewise smooth functions. Phil Trans Royal Soc London 357(1760):2573–2591

    Article  MathSciNet  MATH  Google Scholar 

  33. Cover TM, Thomas JA (2006) Elements of information theory (telecommunications and signal processing). Wiley, New York

    Google Scholar 

  34. Anonymous (2004) Sequence microsoft ballet and breakdancers. http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload

  35. Anonymous (2005) Middlebury stereo dataset. http://vision.middlebury.edu/stereo/data/scenes2005

  36. Anonymous (2006) Middlebury stereo dataset. http://vision.middlebury.edu/stereo/data/scenes2006/

Download references

Acknowledgments

This work has been partially supported by Iran Ministry of Science, Research and Technology and the Swiss National Science Foundation under grant 200021_126894.

Author information

Authors and Affiliations

  1. Ferdowsi University of Mashhad, Mashhad, Iran

    Boshra Rajaei & Hamid-Reza Pourreza

  2. Sadjad Institute of Higher Education, Mashhad, Iran

    Boshra Rajaei

  3. Signal Processing Laboratory (LTS4), École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

    Thomas Maugey & Pascal Frossard

Authors
  1. Boshra Rajaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Maugey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hamid-Reza Pourreza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pascal Frossard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Boshra Rajaei.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rajaei, B., Maugey, T., Pourreza, HR. et al. Rate-distortion analysis of multiview coding in a DIBR framework. Ann. Telecommun. 68, 627–640 (2013). https://doi.org/10.1007/s12243-013-0375-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12243-013-0375-6

Keywords

Search

Navigation