3D Research

, 4:6 | Cite as

Depth-based coding of MVD data for 3D video extension of H.264/AVC

  • Dmytro Rusanovskyy
  • Miska M. Hannuksela
  • Wenyi Su
3DR Express


This paper describes a novel approach of using depth information for advanced coding of associated video data in Multiview Video plus Depth (MVD)-based 3D video systems. As a possible implementation of this conception, we describe two coding tools that have been developed for H.264/AVC based 3D Video Codec as response to Moving Picture Experts Group (MPEG) Call for Proposals (CfP). These tools are Depth-based Motion Vector Prediction (DMVP) and Backward View Synthesis Prediction (BVSP). Simulation results conducted under JCT-3V/MPEG 3DV Common Test Conditions show, that proposed in this paper tools reduce bit rate of coded video data by 15% of average delta bit rate reduction, which results in 13% of bit rate savings on total for the MVD data over the state-of-the-art MVC+D coding.

Moreover, presented in this paper conception of depth-based coding of video has been further developed by MPEG 3DV and JCT-3V and this work resulted in even higher compression efficiency, bringing about 20% of delta bit rate reduction on total for coded MVD data over the reference MVC+D coding. Considering significant gains, proposed in this paper coding approach can be beneficial for development of new 3D video coding standards.


H.264/AVC three-dimensional video video coding 3D-AVC 


  1. 1.
    T. Shibata, J. Kim, D. M. Hoffman, and M. S. Banks (2011) The zone of comfort: predicting visual discomfort with stereo displays, Journal of Vision 11(8):11, 1–29.CrossRefGoogle Scholar
  2. 2.
    A. Smolic, K. Müller, P. Merkle, N. Atzpadin, C. Fehn, M. Müller, O. Schreer, R. Tanger, P. Kauff, T. Wiegand, T. Balogh, Z. Megyesi, and A. Barsi (2007) Multi-view video plus depth (MVD) format for advanced 3D video systems, Joint Video Team, document JVT-W100.Google Scholar
  3. 3.
    ITU-T Recommendation H.264 (2012) Advanced video coding for generic audiovisual services.Google Scholar
  4. 4.
    Call for proposals on 3D video coding technology (2011) MPEG document N12036. Available online: http://mpeg.chiariglione.org/working_documents/explorations/3dav/3dv-cfp.zip Google Scholar
  5. 5.
    Applications and Requirements on 3D Video Coding, MPEG document, Online version: http://mpeg.chiariglione.org/working_documents/explorations/3dav/applications&requirements.zip
  6. 6.
    B. Bross, W.-J. Han, G. J. Sullivan, J.-R. Ohm, and T. Wiegand (ed.) (2012) High Efficiency Video Coding (HEVC) text specification draft 8, JCTVC document J1003.Google Scholar
  7. 7.
    Report of Subjective Test Results from the Call for Proposals on 3D Video Coding, Online: http://mpeg.chiariglione.org/working_documents/explorations/3dav/3d-test-report.zip
  8. 8.
    D. Rusanovskyy and M. M. Hannuksela (2011) Description of 3D video coding technology proposal by Nokia, MPEG document M22552.Google Scholar
  9. 9.
    H. Schwarz, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, D. Marpe, P. Merkle, K. Müller, H. Rhee, G. Tech, M. Winken, T, Wiegand (2011) Description of 3D Video Coding Technology Proposal by Fraunhofer HHI (HEVC compatible, configuration A), MPEG document m22571.Google Scholar
  10. 10.
  11. 11.
    Y. Chen, M. M. Hannuksela, T. Suzuki, and S. Hattori, Overview of the MVC+D 3D video coding standard, Elsevier Journal of Visual Communication and Image Representation. (In press)Google Scholar
  12. 12.
    MVC+D and 3D-AVC reference software: 3DV-ATM version 5.1r2, available online: http://mpeg3dv.research.nokia.com/svn/mpeg3dv/tags/3DVATMv5.1r2/
  13. 13.
    M. M. Hannuksela, Y. Chen, and T. Suzuki (ed.) (2013) 3DAVC draft text 5, JCT-3V document JCT3V-C1002.Google Scholar
  14. 14.
    G. Tech, K. Wegner, Y. Chen, and S. Yea (ed.) (2012) 3DHEVC test model 1, JCT-3V document A1005.Google Scholar
  15. 15.
    G. Tech, K. Wegner, Y. Chen, and M. M. Hannuksela (ed.), (2012) MV-HEVC working draft 1, JCT-3V document A1004.Google Scholar
  16. 16.
    C. Fehn (2004) Depth-image-based rendering (DIBR) compression and transmission for a new approach on 3DTV, Proc. SPIE Conf. Stereoscopic Displays and Virtual Reality Systems XI, 5291: 93–104.CrossRefGoogle Scholar
  17. 17.
    J. Zhang, M. M. Hannuksela, and H, Li (2010) Joint multiview video plus depth coding, Proc. IEEE ICIP, 2865–2868.Google Scholar
  18. 18.
    S. Yea and A. Vetro (2009) View synthesis prediction for multiview video coding, Signal Processing: Image Communication, 24(1–2): 89–100.CrossRefGoogle Scholar
  19. 19.
    D. Tian, P.-L. Lai, P. Lopez, and C. Gomila (2009) View synthesis techniques for 3D video, Proc. SPIE 7443, Applications of Digital Image Processing XXXII.Google Scholar
  20. 20.
    W. Su, D. Rusanovskyy, L. Chen, M. Hannuksela (2011) CE1 — Low complexity block-based View Synthesis Prediction, MPEG document m24915, Geneva.Google Scholar
  21. 21.
    W. Su, D. Rusanovskyy, M. M. Hannuksela (2012) 3DVCE1. a: Block-based View Synthesis Prediction for 3DVATM, JCT-3V document A0107, Stockholm, Sweden.Google Scholar
  22. 22.
    J. Y. Lee, J. Lee, D.-S. Park (2012) CE5.a results on interview motion vector derivation using max disparity in skip and direct modes, JCT-3V document B0149, Shanghai, China.Google Scholar
  23. 23.
    C.-L. Wu, Y.-L. Chang, Y.-P. Tsai, S. Lei (2012) 3D-CE1.a: interview skip/direct mode with sub-partition scheme, JCT-3V document B0094, Shanghai, China.Google Scholar
  24. 24.
    J.-L. Lin, Y.-W. Chen, X. Guo, Y.-L. Chang, Y.-P. Tsai, Y.-W. Huang, S. Lei (2012) 3D-CE5.a related motion vector competition-based Skip/Direct mode with explicit signaling, MPEG document m24847, Geneva, Switzerland.Google Scholar
  25. 25.
    D. Rusanovskyy, M. M. Hannuksella (2013) CE1.a-related: Simplification of BVSP in 3DV-ATM, JCT-3V document C0169, Geneva, Switzerland.Google Scholar
  26. 26.
    J-L. Lin, Y-W. Chen, Y.-W Huang, S. Lei (2012) 3D-CE5.a related: Simplification on the disparity vector derivation for AVC-based 3D video coding, JCT-3V document A0045, Stockholm, Sweden.Google Scholar
  27. 27.
    W. Su, D. Rusanovskyy, M. M. Hannuksela, and H. Li (2012) Depth-based motion vector prediction in 3D video coding, Proc. of Picture Coding Symposium.Google Scholar
  28. 28.
    D. Rusanovskyy, K. Müller, A. Vetro (2012) Common Test Conditions of 3DV Core Experiments, JCT-3V document A1100, Stockholm.Google Scholar
  29. 29.
    G. Bjøntegaard (2001) Calculation of average PSNR differences between RD-Curves, ITU-T SG16 Q.6, document VCEG-M33.Google Scholar
  30. 30.
    J. Y. Lee, T. Uchiumi, J. Lee, Y. Yamamoto, D.-S. Park (2012) 3D-CE5.a results on joint proposal for an improved depth-based motion vector prediction method by Samsung and Sharp, MPEG document M24824, Geneva, Switzerland.Google Scholar

Copyright information

© 3D Display Research Center and Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Dmytro Rusanovskyy
    • 1
  • Miska M. Hannuksela
    • 1
  • Wenyi Su
    • 2
  1. 1.Nokia Resarch CenterHelsinkiFinland
  2. 2.University of Science and Technoogy of ChinaHefeiChina

Personalised recommendations