Skip to main content
SpringerLink
Log in

Depth Map Compression for Depth-Image-Based Rendering

  • Chapter
  • First Online:
3D-TV System with Depth-Image-Based Rendering

Abstract

In this chapter, we discuss unique characteristics of depth maps, review recent depth map coding techniques, and describe how texture and depth map compression can be jointly optimized.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    As an example, the MPEG committee defined an MVC extension to an existing standard [1] where no new coding tools were introduced.

  2. 2.

    \(D_{l}(m,n)\) is more commonly called the disparity value, which is technically the inverse of the depth value. For simplicity of presentation, we assume this is understood from context and will refer to \(D_{l}(m,n)\) as depth value.

  3. 3.

    Note that while “edge” can refer to a link or connection between nodes in graph theory, we only use the term “edge” to refer an image edge to avoid confusion.

References

  1. Vetro A, Wiegand T, Sullivan GJ (2011) Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard. Proc IEEE 99(4):626–642

    Article  Google Scholar 

  2. Kim WS, Ortega A, Lee J, Wey H (2011) 3D video quality improvement using depth transition data. In: IEEE international workshop on hot topics in 3D. Barcelona, Spain

    Google Scholar 

  3. Farre M, Wang O, Lang M, Stefanoski N, Hornung A, Smolic A (2011) Automatic content creation for multiview autostereoscopic displays using image domain warping. In: IEEE international workshop on hot topics in 3D. Barcelona, Spain

    Google Scholar 

  4. Oh H, Ho YS (2006) H.264-based depth map sequence coding using motion information of corresponding texture video. In: The Pacific-Rim symposium on image and video technology. Hsinchu, Taiwan

    Google Scholar 

  5. Daribo I, Tillier C, Pesquet-Popescu B (2009) Motion vector sharing and bit-rate allocation for 3D video-plus-depth coding. In: EURASIP: special issue on 3DTV in Journal on Advances in Signal Processing, vol 2009

    Google Scholar 

  6. Merkle P, Morvan Y, Smolic A, Farin D, Muller K, de With P, Wiegand T (2009) The effects of multiview depth video compression on multiview rendering. Signal Process Image Commun 24:73–88

    Article  Google Scholar 

  7. Leon G, Kalva H, Furht B (2008) 3D video quality evaluation with depth quality variations. In: Proceedings of 3DTV-conference: the true vision - capture, transmission and display of 3D video, 3DTV-CON 2008. Istanbul, Turkey

    Google Scholar 

  8. Tanimoto M, Fujii T, Suzuki K (2009) View synthesis algorithm in view synthesis reference software 2.0 (VSRS2.0). Document M16090, ISO/IEC JTC1/SC29/WG11

    Google Scholar 

  9. Kim WS, Ortega A, Lee J, Wey H (2010) 3-D video coding using depth transition data. In: IEEE picture coding symposium. Nagoya, Japan

    Google Scholar 

  10. Müller K, Smolic A, Dix K, Merkle P, Wiegand T (2009) Coding and intermediate view synthesis of multiview video plus depth. In: Proceedings of IEEE international conference on image processing, ICIP 2009. Cairo, Egypt

    Google Scholar 

  11. Nguyen HT, Do MN (2009) Error analysis for image-based rendering with depth information. IEEE Trans Image Process 18(4):703–716

    Article  MathSciNet  Google Scholar 

  12. Ramanathan P, Girod B (2006) Rate-distortion analysis for light field coding and streaming. Singal Process Image Commun 21(6):462–475

    Article  Google Scholar 

  13. Kim WS, Ortega A, Lai P, Tian D, Gomila C (2009) Depth map distortion analysis for view rendering and depth coding. In: IEEE international conference on image processing. Cairo, Egypt

    Google Scholar 

  14. Video (2010) Report on experimental framework for 3D video coding. Document N11631, ISO/IEC JTC1/SC29/WG11

    Google Scholar 

  15. Kim WS, Ortega A, Lai P, Tian D, Gomila C (2010) Depth map coding with distortion estimation of rendered view. In: SPIE visual information processing and communication. San Jose, CA

    Google Scholar 

  16. Lai P, Ortega A, Dorea C, Yin P, Gomila C (2009) Improving view rendering quality and coding efficiency by suppressing compression artifacts in depth-image coding. In: Proceedings of SPIE visual communication and image processing, VCIP 2009. San Jose, CA, USA

    Google Scholar 

  17. Ortega A, Ramchandran K (1998) Rate-distortion techniques in image and video compression. IEEE Signal Process Mag 15(6):23–50

    Article  Google Scholar 

  18. Sullivan G, Wiegand T (1988) Rate-distortion optimization for video compression. IEEE Signal Process Mag 15(6):74–90

    Article  Google Scholar 

  19. Wiegand T, Girod B (2001) Lagrange multiplier selection in hybrid video coder control. In: IEEE international conference on image processing. Thessaloniki, Greece

    Google Scholar 

  20. Wiegand T, Sullivan G, Bjontegaard G, Luthra A (2003) Overview of the H.264/AVC video coding standard. IEEE Trans Circuits Syst Video Technol 13(7):560–576

    Article  Google Scholar 

  21. Mark W, McMillan L, Bishop G (1997) Post-rendering 3D warping. In: Symposium on interactive 3D graphics. New York, NY

    Google Scholar 

  22. Cheung G, Kubota A, Ortega A (2010) Sparse representation of depth maps for efficient transform coding. In: IEEE picture coding symposium. Nagoya, Japan

    Google Scholar 

  23. Cheung G, Ishida J, Kubota A, Ortega A (2011) Transform domain sparsification of depth maps using iterative quadratic programming. In: IEEE international conference on image processing. Brussels, Belgium

    Google Scholar 

  24. (2006) stereo datasets. http://vision.middlebury.edu/stereo/data/scenes2006/

  25. Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted \(l_1\) minimization. J Fourier Anal Appl 14(5):877–905

    Google Scholar 

  26. Wipf D, Nagarajan S (2010) Iterative reweighted \(l_1\) and \(l_2\) methods for finding sparse solutions. IEEE J Sel Top Sign Process 4(2):317–329

    Google Scholar 

  27. Papadimitriou CH, Steiglitz K (1998) Combinatorial optimization: algorithms and complexity. Dover, NY

    MATH  Google Scholar 

  28. Daubechies I, Devore R, Fornasier M, Gunturk S (2010) Iteratively re-weighted least squares minimization for sparse recovery. Commun Pure Appl Math 63(1):1–38

    Google Scholar 

  29. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  30. Valenzise G, Cheung G, Galvao R, Cagnazzo M, Pesquet-Popescu B, Ortega A (2012) Motion prediction of depth video for depth-image-based rendering using don’t care regions. In: Picture coding symposium. Krakow, Poland

    Google Scholar 

  31. Gilge M, Engelhardt T, Mehlan R (1989) Coding of arbitrarily shaped image segments based on a generalized orthogonal transform. Signal Process Image Commun 1:153–180

    Article  Google Scholar 

  32. Chang SF, Messerschmitt DG (1993) Transform coding of arbitrarily-shaped image segments. In: Proceedings of 1st ACM international conference on multimedia. Anaheim, CA, pp 83–90

    Google Scholar 

  33. Sikora T, Bauer S, Makai B (1995) Efficiency of shape-adaptive 2-D transforms for coding of arbitrarily shaped image segments. IEEE Trans Circuits Syst Video Technol 5(3):254–258

    Article  Google Scholar 

  34. Li S, Li W (2000) Shape-adaptive discrete wavelet transforms for arbitrarily shaped visual object coding. IEEE Trans Circuits Syst Video Technol 10(5):725–743

    Article  Google Scholar 

  35. Freeman H (1961) On the encoding of arbitrary geometric configurations. IRE Trans Electron Comput 10(2):260–268

    Article  MathSciNet  Google Scholar 

  36. Maitre M, Shinagawa Y, Do M (2008) Wavelet-based joint estimation and encoding of depth-image-based representations for free-viewpoint rendering. IEEE Trans Image Process 17(6):946–957

    Article  MathSciNet  Google Scholar 

  37. Shen G, Kim WS, Narang S, Ortega A, Lee J, Wey H (2010) Edge-adaptive transforms for efficient depth map coding. In: IEEE picture coding symposium. Nagoya, Japan

    Google Scholar 

  38. Philips W (1999) Comparison of techniques for intra-frame coding of arbitrarily shaped video object boundary blocks. IEEE Trans Circuits Syst Video Technol 9(7):1009–1012

    Article  Google Scholar 

  39. Zeng B, Fu J (2006) Directional discrete cosine transforms for image coding. In: Proceedings of IEEE international conference on multimedia and expo, ICME 2006. Toronto, Canada, pp 721–724

    Google Scholar 

  40. Fu J, Zeng B (2007) Directional discrete cosine transforms: a theoretical analysis. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 2007, vol I. Honolulu, HI, USA, pp 1105–1108

    Google Scholar 

  41. Zeng B, Fu J (2008) Directional discrete cosine transforms—a new framework for image coding. IEEE Trans Circuits Syst Video Technol 18(3):305–313

    Article  MathSciNet  Google Scholar 

  42. Zhang C, Ugur K, Lainema J, Gabbouj M (2009) Video coding using spatially varying transform. In: Proceedings of 3rd Pacific Rim symposium on advances in image and video technology, PSIVT 2007. Tokyo, Japan, pp 796–806

    Google Scholar 

  43. Zhang C, Ugur K, Lainema J, Gabbouj M (2009) Video coding using variable block-size spatially varying transforms. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 2009, Taipei, Taiwan, pp 905–908

    Google Scholar 

  44. Wien M (2003) Variable block-size transforms for H.264/AVC. IEEE Trans Circuits Syst Video Technol 13(7):604–613

    Article  Google Scholar 

  45. Chang CL, Makar M, Tsai SS, Girod B (2010) Direction-adaptive partitioned block transform for color image coding. IEEE Trans Image Proc 19(7):1740–1755

    Article  MathSciNet  Google Scholar 

  46. Ye Y, Karczewicz M (2008) Improved H.264 intra coding based on bi-directional intra prediction, directional transform, and adaptive coefficient scanning. In: Proceedings of IEEE international conference on image processing, ICIP 2008. San Diego, CA, USA, pp 2116–2119

    Google Scholar 

  47. Soumekh M (1988) Binary image reconstruction from four projections. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 1988. New York, NY, USA, pp 1280–1283

    Google Scholar 

  48. Ramesh GR, Rajgopal K (1990) Binary image compression using the radon transform. In: Proceedings of XVI annual convention and exhibition of the IEEE in India, ACE 90. Bangalore, India, pp 178–182

    Google Scholar 

  49. Willett R, Nowak R (2003) Platelets: a multiscale approach for recovering edges and surfaces in photon-limited medical imaging. IEEE Trans Med Imaging 22(3):332–350

    Article  Google Scholar 

  50. Morvan Y, de With P, Farin D (2006) Platelets-based coding of depth maps for the transmission of multiview images. In: SPIE stereoscopic displays and applications. San Jose, CA

    Google Scholar 

  51. Kim WS (2011) 3-D video coding system with enhanced rendered view quality. Ph.D. thesis, University of Southern California

    Google Scholar 

  52. Hammond D, Vandergheynst P, Gribonval R (2010) Wavelets on graphs via spectral graph theory. Elsevier: Appl Comput Harmonic Anal 30:129–150

    Article  MathSciNet  Google Scholar 

  53. Rutishauser H (1966) The Jacobi method for real symmetric matrices. Numer Math 9(1): 

    Article  MathSciNet  Google Scholar 

  54. Kim WS, Narang SK, Ortega A (2012) Graph based transforms for depth video coding. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, ICASSP 2012. Kyoto, Japan

    Google Scholar 

  55. Grewatsch S, Muller E (2004) Sharing of motion vectors in 3D video coding. In: IEEE International Conference on Image Processing, Singapore

    Google Scholar 

  56. Daribo I, Florencio D, Cheung G (2012) Arbitrarily shaped sub-block motion prediction in texture map compression using depth information. In: Picture coding symposium. Krakow, Poland

    Google Scholar 

  57. Cheung G, Ortega A, Sakamoto T (2008) Fast H.264 mode selection using depth information for distributed game viewing. In: IS&T/SPIE visual communications and image processing (VCIP’08). San Jose, CA

    Google Scholar 

  58. Gray RM, Hashimoto T (2008) Rate-distortion functions for nonstationary Gaussian autoregressive processes. In: IEEE data compression conference, pp 53–62

    Google Scholar 

  59. Sagetong P, Ortega A (2002) Rate-distortion model and analytical bit allocation for wavelet-based region of interest coding. In: IEEE international conference on image processing, vol 3, pp 97–100

    Google Scholar 

  60. Hang HM, Chen JJ (1997) Source model for transform video coder and its application—part I: fundamental theory. IEEE Trans Circuits Syst Video Technol 7:287–298

    Article  Google Scholar 

  61. Lin LJ, Ortega A (1998) Bit-rate control using piecewise approximated rate-distortion characteristics. IEEE Trans Circuits Syst Video Technol 8:446–459

    Article  Google Scholar 

  62. Na ST, Oh KJ, Ho YS (2008) Joint coding of multi-view video and corresponding depth map. In: IEEE international conference on image processing, pp 2468–2471

    Google Scholar 

  63. Ince S, Martinian E, Yea S, Vetor A (2007) Depth estimation for view synthesis in multiview video coding. In: IEEE 3DTV conference

    Google Scholar 

  64. Liu Y, Huang Q, Ma S, Zhao D, Gao W (2009) Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Elsevier, Signal Process Image Commun 24(8):666–681

    Article  Google Scholar 

  65. Yuan H, Chang Y, Huo J, Yang F, Lu Z (2011) Model-based joint bit allocation between texture videos and depth maps for 3-D video coding. IEEE Trans Circuits Syst Video Technol 21(4):485–497

    Article  Google Scholar 

  66. Wang H, Kwong S (2008) Rate-distortion optimization of rate control for H.264 with adaptive initial quantization parameter determination. IEEE Trans Circuits Syst Video Technol 18(1):140–144

    Article  Google Scholar 

  67. Ma S, Gao W, Lu Y (2005) Rate-distortion analysis for H.264/AVC video coding and its application to rate control. IEEE Trans Circuits Syst Video Technol 15(12):1533–1544

    Article  Google Scholar 

  68. Liu S, Lai P, Tian D, Chen CW (2011) New depth coding techniques with utilization of corresponding video. IEEE Trans Broadcast 57(2):551–561 part 2

    Article  Google Scholar 

  69. Merkle P, Smolic A, Muller K, Wiegand T (2007) Efficient prediction structures for multiview video coding. IEEE Trans Circuits Syst Video Technol 17(11):1461–1473

    Article  Google Scholar 

  70. Shen LQ, Liu Z, Liu SX, Zhang ZY, An P (2009) Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding. IEEE Trans Broadcast 55(4):761–766

    Article  Google Scholar 

  71. Liu Y, Huang Q, Ma S, Zhao D, Gao W, Ci S, Tang H (2011) A novel rate control technique for multiview video plus depth based 3D video coding. IEEE Trans Broadcast 57(2):562–571 (part 2)

    Article  Google Scholar 

  72. Davidoiu V, Maugey T, Pesquet-Popescu B, Frossard P (2011) Rate distortion analysis in a disparity compensated scheme. In: IEEE international conference on acoustics, speech and signal processing. Prague, Czech Republic

    Google Scholar 

  73. Fraysse A, Pesquet-Popescu B, Pesquet JC (2009) On the uniform quantization of a class of sparse source. IEEE Trans Inf Theory 55(7):3243–3263

    Article  MathSciNet  Google Scholar 

  74. Gelman A, Dragotti PL, Velisavljevic V (2012) Multiview image coding using depth layers and an optimized bit allocation. In: IEEE Transactions on Image Processing (to appear in 2012)

    Google Scholar 

  75. Velisavljevic V, Cheung G, Chakareski J (2011) Bit allocation for multiview image compression using cubic synthesized view distortion model. In: IEEE international workshop on hot topics in 3D (in conjunction with ICME 2011). Barcelona, Spain

    Google Scholar 

  76. Cheung G, Velisavljevic V, Ortega A (2011) On dependent bit allocation for multiview image coding with depth-image-based rendering. IEEE Trans Image Process 20(11):3179–3194

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, 101-8430, Japan

    Gene Cheung

  2. University of Southern California, Los Angeles, CA, USA

    Antonio Ortega

  3. Texas Instruments Inc., Dallas, TX, USA

    Woo-Shik Kim

  4. University of Bedfordshire, Bedfordshire, UK

    Vladan Velisavljevic

  5. Chuo University, Hachio-ji, Tokyo, Japan

    Akira Kubota

Authors
  1. Gene Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Ortega
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Woo-Shik Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vladan Velisavljevic
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Akira Kubota
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gene Cheung .

Editor information

Editors and Affiliations

  1. , School of Electrical & Electronic, Nanyang Technological University, Nanyang Avenue 50, Singapore, 639798, Singapore

    Ce Zhu

  2. Electronic Engineering, Department of Information Science &, Zheda Road 38, Hangzhou, 310027, China, People's Republic

    Yin Zhao

  3. , Department of Information Science &, Zhejiang University, Zheda Road 38, Hangzhou, 310027, China, People's Republic

    Lu Yu

  4. Graduate School of Engineering, Department of Electrical Engineering and, Nagoya University, Nagoya, 464-8603, Japan

    Masayuki Tanimoto

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Cheung, G., Ortega, A., Kim, WS., Velisavljevic, V., Kubota, A. (2013). Depth Map Compression for Depth-Image-Based Rendering. In: Zhu, C., Zhao, Y., Yu, L., Tanimoto, M. (eds) 3D-TV System with Depth-Image-Based Rendering. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9964-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-9964-1_9

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-9963-4

  • Online ISBN: 978-1-4419-9964-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation