Advertisement

Depth Map Compression for Depth-Image-Based Rendering

  • Gene Cheung
  • Antonio Ortega
  • Woo-Shik Kim
  • Vladan Velisavljevic
  • Akira Kubota
Chapter

Abstract

In this chapter, we discuss unique characteristics of depth maps, review recent depth map coding techniques, and describe how texture and depth map compression can be jointly optimized.

Keywords

Bit allocation Characteristics of depth Depth-image-based rendering (DIBR) Depth map coding Distortion model Don’t care region Edge-adaptive wavelet Graph-based transform Geometric error Joint coding Quadratic penalty function Rate-distortion optimization Rendered view distortion Sparse representation 

References

  1. 1.
    Vetro A, Wiegand T, Sullivan GJ (2011) Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard. Proc IEEE 99(4):626–642CrossRefGoogle Scholar
  2. 2.
    Kim WS, Ortega A, Lee J, Wey H (2011) 3D video quality improvement using depth transition data. In: IEEE international workshop on hot topics in 3D. Barcelona, SpainGoogle Scholar
  3. 3.
    Farre M, Wang O, Lang M, Stefanoski N, Hornung A, Smolic A (2011) Automatic content creation for multiview autostereoscopic displays using image domain warping. In: IEEE international workshop on hot topics in 3D. Barcelona, SpainGoogle Scholar
  4. 4.
    Oh H, Ho YS (2006) H.264-based depth map sequence coding using motion information of corresponding texture video. In: The Pacific-Rim symposium on image and video technology. Hsinchu, TaiwanGoogle Scholar
  5. 5.
    Daribo I, Tillier C, Pesquet-Popescu B (2009) Motion vector sharing and bit-rate allocation for 3D video-plus-depth coding. In: EURASIP: special issue on 3DTV in Journal on Advances in Signal Processing, vol 2009Google Scholar
  6. 6.
    Merkle P, Morvan Y, Smolic A, Farin D, Muller K, de With P, Wiegand T (2009) The effects of multiview depth video compression on multiview rendering. Signal Process Image Commun 24:73–88CrossRefGoogle Scholar
  7. 7.
    Leon G, Kalva H, Furht B (2008) 3D video quality evaluation with depth quality variations. In: Proceedings of 3DTV-conference: the true vision - capture, transmission and display of 3D video, 3DTV-CON 2008. Istanbul, TurkeyGoogle Scholar
  8. 8.
    Tanimoto M, Fujii T, Suzuki K (2009) View synthesis algorithm in view synthesis reference software 2.0 (VSRS2.0). Document M16090, ISO/IEC JTC1/SC29/WG11Google Scholar
  9. 9.
    Kim WS, Ortega A, Lee J, Wey H (2010) 3-D video coding using depth transition data. In: IEEE picture coding symposium. Nagoya, JapanGoogle Scholar
  10. 10.
    Müller K, Smolic A, Dix K, Merkle P, Wiegand T (2009) Coding and intermediate view synthesis of multiview video plus depth. In: Proceedings of IEEE international conference on image processing, ICIP 2009. Cairo, EgyptGoogle Scholar
  11. 11.
    Nguyen HT, Do MN (2009) Error analysis for image-based rendering with depth information. IEEE Trans Image Process 18(4):703–716MathSciNetCrossRefGoogle Scholar
  12. 12.
    Ramanathan P, Girod B (2006) Rate-distortion analysis for light field coding and streaming. Singal Process Image Commun 21(6):462–475CrossRefGoogle Scholar
  13. 13.
    Kim WS, Ortega A, Lai P, Tian D, Gomila C (2009) Depth map distortion analysis for view rendering and depth coding. In: IEEE international conference on image processing. Cairo, EgyptGoogle Scholar
  14. 14.
    Video (2010) Report on experimental framework for 3D video coding. Document N11631, ISO/IEC JTC1/SC29/WG11Google Scholar
  15. 15.
    Kim WS, Ortega A, Lai P, Tian D, Gomila C (2010) Depth map coding with distortion estimation of rendered view. In: SPIE visual information processing and communication. San Jose, CAGoogle Scholar
  16. 16.
    Lai P, Ortega A, Dorea C, Yin P, Gomila C (2009) Improving view rendering quality and coding efficiency by suppressing compression artifacts in depth-image coding. In: Proceedings of SPIE visual communication and image processing, VCIP 2009. San Jose, CA, USAGoogle Scholar
  17. 17.
    Ortega A, Ramchandran K (1998) Rate-distortion techniques in image and video compression. IEEE Signal Process Mag 15(6):23–50CrossRefGoogle Scholar
  18. 18.
    Sullivan G, Wiegand T (1988) Rate-distortion optimization for video compression. IEEE Signal Process Mag 15(6):74–90CrossRefGoogle Scholar
  19. 19.
    Wiegand T, Girod B (2001) Lagrange multiplier selection in hybrid video coder control. In: IEEE international conference on image processing. Thessaloniki, GreeceGoogle Scholar
  20. 20.
    Wiegand T, Sullivan G, Bjontegaard G, Luthra A (2003) Overview of the H.264/AVC video coding standard. IEEE Trans Circuits Syst Video Technol 13(7):560–576CrossRefGoogle Scholar
  21. 21.
    Mark W, McMillan L, Bishop G (1997) Post-rendering 3D warping. In: Symposium on interactive 3D graphics. New York, NYGoogle Scholar
  22. 22.
    Cheung G, Kubota A, Ortega A (2010) Sparse representation of depth maps for efficient transform coding. In: IEEE picture coding symposium. Nagoya, JapanGoogle Scholar
  23. 23.
    Cheung G, Ishida J, Kubota A, Ortega A (2011) Transform domain sparsification of depth maps using iterative quadratic programming. In: IEEE international conference on image processing. Brussels, BelgiumGoogle Scholar
  24. 24.
  25. 25.
    Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted \(l_1\) minimization. J Fourier Anal Appl 14(5):877–905Google Scholar
  26. 26.
    Wipf D, Nagarajan S (2010) Iterative reweighted \(l_1\) and \(l_2\) methods for finding sparse solutions. IEEE J Sel Top Sign Process 4(2):317–329Google Scholar
  27. 27.
    Papadimitriou CH, Steiglitz K (1998) Combinatorial optimization: algorithms and complexity. Dover, NYMATHGoogle Scholar
  28. 28.
    Daubechies I, Devore R, Fornasier M, Gunturk S (2010) Iteratively re-weighted least squares minimization for sparse recovery. Commun Pure Appl Math 63(1):1–38Google Scholar
  29. 29.
    Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, CambridgeMATHGoogle Scholar
  30. 30.
    Valenzise G, Cheung G, Galvao R, Cagnazzo M, Pesquet-Popescu B, Ortega A (2012) Motion prediction of depth video for depth-image-based rendering using don’t care regions. In: Picture coding symposium. Krakow, PolandGoogle Scholar
  31. 31.
    Gilge M, Engelhardt T, Mehlan R (1989) Coding of arbitrarily shaped image segments based on a generalized orthogonal transform. Signal Process Image Commun 1:153–180CrossRefGoogle Scholar
  32. 32.
    Chang SF, Messerschmitt DG (1993) Transform coding of arbitrarily-shaped image segments. In: Proceedings of 1st ACM international conference on multimedia. Anaheim, CA, pp 83–90Google Scholar
  33. 33.
    Sikora T, Bauer S, Makai B (1995) Efficiency of shape-adaptive 2-D transforms for coding of arbitrarily shaped image segments. IEEE Trans Circuits Syst Video Technol 5(3):254–258CrossRefGoogle Scholar
  34. 34.
    Li S, Li W (2000) Shape-adaptive discrete wavelet transforms for arbitrarily shaped visual object coding. IEEE Trans Circuits Syst Video Technol 10(5):725–743CrossRefGoogle Scholar
  35. 35.
    Freeman H (1961) On the encoding of arbitrary geometric configurations. IRE Trans Electron Comput 10(2):260–268MathSciNetCrossRefGoogle Scholar
  36. 36.
    Maitre M, Shinagawa Y, Do M (2008) Wavelet-based joint estimation and encoding of depth-image-based representations for free-viewpoint rendering. IEEE Trans Image Process 17(6):946–957MathSciNetCrossRefGoogle Scholar
  37. 37.
    Shen G, Kim WS, Narang S, Ortega A, Lee J, Wey H (2010) Edge-adaptive transforms for efficient depth map coding. In: IEEE picture coding symposium. Nagoya, JapanGoogle Scholar
  38. 38.
    Philips W (1999) Comparison of techniques for intra-frame coding of arbitrarily shaped video object boundary blocks. IEEE Trans Circuits Syst Video Technol 9(7):1009–1012CrossRefGoogle Scholar
  39. 39.
    Zeng B, Fu J (2006) Directional discrete cosine transforms for image coding. In: Proceedings of IEEE international conference on multimedia and expo, ICME 2006. Toronto, Canada, pp 721–724Google Scholar
  40. 40.
    Fu J, Zeng B (2007) Directional discrete cosine transforms: a theoretical analysis. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 2007, vol I. Honolulu, HI, USA, pp 1105–1108Google Scholar
  41. 41.
    Zeng B, Fu J (2008) Directional discrete cosine transforms—a new framework for image coding. IEEE Trans Circuits Syst Video Technol 18(3):305–313MathSciNetCrossRefGoogle Scholar
  42. 42.
    Zhang C, Ugur K, Lainema J, Gabbouj M (2009) Video coding using spatially varying transform. In: Proceedings of 3rd Pacific Rim symposium on advances in image and video technology, PSIVT 2007. Tokyo, Japan, pp 796–806Google Scholar
  43. 43.
    Zhang C, Ugur K, Lainema J, Gabbouj M (2009) Video coding using variable block-size spatially varying transforms. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 2009, Taipei, Taiwan, pp 905–908Google Scholar
  44. 44.
    Wien M (2003) Variable block-size transforms for H.264/AVC. IEEE Trans Circuits Syst Video Technol 13(7):604–613CrossRefGoogle Scholar
  45. 45.
    Chang CL, Makar M, Tsai SS, Girod B (2010) Direction-adaptive partitioned block transform for color image coding. IEEE Trans Image Proc 19(7):1740–1755MathSciNetCrossRefGoogle Scholar
  46. 46.
    Ye Y, Karczewicz M (2008) Improved H.264 intra coding based on bi-directional intra prediction, directional transform, and adaptive coefficient scanning. In: Proceedings of IEEE international conference on image processing, ICIP 2008. San Diego, CA, USA, pp 2116–2119Google Scholar
  47. 47.
    Soumekh M (1988) Binary image reconstruction from four projections. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 1988. New York, NY, USA, pp 1280–1283Google Scholar
  48. 48.
    Ramesh GR, Rajgopal K (1990) Binary image compression using the radon transform. In: Proceedings of XVI annual convention and exhibition of the IEEE in India, ACE 90. Bangalore, India, pp 178–182Google Scholar
  49. 49.
    Willett R, Nowak R (2003) Platelets: a multiscale approach for recovering edges and surfaces in photon-limited medical imaging. IEEE Trans Med Imaging 22(3):332–350CrossRefGoogle Scholar
  50. 50.
    Morvan Y, de With P, Farin D (2006) Platelets-based coding of depth maps for the transmission of multiview images. In: SPIE stereoscopic displays and applications. San Jose, CAGoogle Scholar
  51. 51.
    Kim WS (2011) 3-D video coding system with enhanced rendered view quality. Ph.D. thesis, University of Southern CaliforniaGoogle Scholar
  52. 52.
    Hammond D, Vandergheynst P, Gribonval R (2010) Wavelets on graphs via spectral graph theory. Elsevier: Appl Comput Harmonic Anal 30:129–150MathSciNetCrossRefGoogle Scholar
  53. 53.
    Rutishauser H (1966) The Jacobi method for real symmetric matrices. Numer Math 9(1): MathSciNetCrossRefGoogle Scholar
  54. 54.
    Kim WS, Narang SK, Ortega A (2012) Graph based transforms for depth video coding. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 2012. Kyoto, JapanGoogle Scholar
  55. 55.
    Grewatsch S, Muller E (2004) Sharing of motion vectors in 3D video coding. In: IEEE International Conference on Image Processing, SingaporeGoogle Scholar
  56. 56.
    Daribo I, Florencio D, Cheung G (2012) Arbitrarily shaped sub-block motion prediction in texture map compression using depth information. In: Picture coding symposium. Krakow, PolandGoogle Scholar
  57. 57.
    Cheung G, Ortega A, Sakamoto T (2008) Fast H.264 mode selection using depth information for distributed game viewing. In: IS&T/SPIE visual communications and image processing (VCIP’08). San Jose, CAGoogle Scholar
  58. 58.
    Gray RM, Hashimoto T (2008) Rate-distortion functions for nonstationary Gaussian autoregressive processes. In: IEEE data compression conference, pp 53–62Google Scholar
  59. 59.
    Sagetong P, Ortega A (2002) Rate-distortion model and analytical bit allocation for wavelet-based region of interest coding. In: IEEE international conference on image processing, vol 3, pp 97–100Google Scholar
  60. 60.
    Hang HM, Chen JJ (1997) Source model for transform video coder and its application—part I: fundamental theory. IEEE Trans Circuits Syst Video Technol 7:287–298CrossRefGoogle Scholar
  61. 61.
    Lin LJ, Ortega A (1998) Bit-rate control using piecewise approximated rate-distortion characteristics. IEEE Trans Circuits Syst Video Technol 8:446–459CrossRefGoogle Scholar
  62. 62.
    Na ST, Oh KJ, Ho YS (2008) Joint coding of multi-view video and corresponding depth map. In: IEEE international conference on image processing, pp 2468–2471Google Scholar
  63. 63.
    Ince S, Martinian E, Yea S, Vetor A (2007) Depth estimation for view synthesis in multiview video coding. In: IEEE 3DTV conferenceGoogle Scholar
  64. 64.
    Liu Y, Huang Q, Ma S, Zhao D, Gao W (2009) Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Elsevier, Signal Process Image Commun 24(8):666–681CrossRefGoogle Scholar
  65. 65.
    Yuan H, Chang Y, Huo J, Yang F, Lu Z (2011) Model-based joint bit allocation between texture videos and depth maps for 3-D video coding. IEEE Trans Circuits Syst Video Technol 21(4):485–497CrossRefGoogle Scholar
  66. 66.
    Wang H, Kwong S (2008) Rate-distortion optimization of rate control for H.264 with adaptive initial quantization parameter determination. IEEE Trans Circuits Syst Video Technol 18(1):140–144CrossRefGoogle Scholar
  67. 67.
    Ma S, Gao W, Lu Y (2005) Rate-distortion analysis for H.264/AVC video coding and its application to rate control. IEEE Trans Circuits Syst Video Technol 15(12):1533–1544CrossRefGoogle Scholar
  68. 68.
    Liu S, Lai P, Tian D, Chen CW (2011) New depth coding techniques with utilization of corresponding video. IEEE Trans Broadcast 57(2):551–561 part 2CrossRefGoogle Scholar
  69. 69.
    Merkle P, Smolic A, Muller K, Wiegand T (2007) Efficient prediction structures for multiview video coding. IEEE Trans Circuits Syst Video Technol 17(11):1461–1473CrossRefGoogle Scholar
  70. 70.
    Shen LQ, Liu Z, Liu SX, Zhang ZY, An P (2009) Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding. IEEE Trans Broadcast 55(4):761–766CrossRefGoogle Scholar
  71. 71.
    Liu Y, Huang Q, Ma S, Zhao D, Gao W, Ci S, Tang H (2011) A novel rate control technique for multiview video plus depth based 3D video coding. IEEE Trans Broadcast 57(2):562–571 (part 2)CrossRefGoogle Scholar
  72. 72.
    Davidoiu V, Maugey T, Pesquet-Popescu B, Frossard P (2011) Rate distortion analysis in a disparity compensated scheme. In: IEEE international conference on acoustics, speech and signal processing. Prague, Czech RepublicGoogle Scholar
  73. 73.
    Fraysse A, Pesquet-Popescu B, Pesquet JC (2009) On the uniform quantization of a class of sparse source. IEEE Trans Inf Theory 55(7):3243–3263MathSciNetCrossRefGoogle Scholar
  74. 74.
    Gelman A, Dragotti PL, Velisavljevic V (2012) Multiview image coding using depth layers and an optimized bit allocation. In: IEEE Transactions on Image Processing (to appear in 2012)Google Scholar
  75. 75.
    Velisavljevic V, Cheung G, Chakareski J (2011) Bit allocation for multiview image compression using cubic synthesized view distortion model. In: IEEE international workshop on hot topics in 3D (in conjunction with ICME 2011). Barcelona, SpainGoogle Scholar
  76. 76.
    Cheung G, Velisavljevic V, Ortega A (2011) On dependent bit allocation for multiview image coding with depth-image-based rendering. IEEE Trans Image Process 20(11):3179–3194MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Gene Cheung
    • 1
  • Antonio Ortega
    • 2
  • Woo-Shik Kim
    • 3
  • Vladan Velisavljevic
    • 4
  • Akira Kubota
    • 5
  1. 1.National Institute of InformaticsChiyoda-kuJapan
  2. 2.University of Southern CaliforniaLos AngelesUSA
  3. 3.Texas Instruments Inc.DallasUSA
  4. 4.University of BedfordshireBedfordshireUK
  5. 5.Chuo UniversityHachio-jiJapan

Personalised recommendations