Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 15, pp 19051–19069 | Cite as

SSIM-based joint-bit allocation for 3D video coding

  • Harshalatha Y
  • Prabir Kumar Biswas
Article

Abstract

The quality of a 3D video display depends on virtual view synthesis process which is affected by the bit allocation criterion. The performance of a bit allocation algorithm is dependent on various encoding parameters like quantization parameter, motion vector, mode selection, and so on. Rate-distortion optimization (RDO) is used to efficiently allocate bits with minimum distortion. In 3D video, rate-distortion (RD) property of synthesized view is used to assign bits between texture video and depth map. Existing literature on bit allocation methods use mean square error (MSE) as distortion metric which is not suitable for measuring perceptual quality. In this paper, we propose structural similarity (SSIM)-based joint bit allocation scheme to enhance visual quality of 3D video. Perceptual quality of a synthesized view depends on texture and depth map quality. Thus, SSIM-based RDO is performed on both texture and depth map where SSIM is used as distortion metric in mode decision and motion estimation. SSIM-based distortion model for synthesized view is determined experimentally. As SSIM cannot be related to quantization step, SSIM-MSE relation is used to convert distortion model in terms of MSE. The Lagrange multiplier method is used to solve the bit allocation problem. The proposed algorithm is implemented using 3DV-ATM as well as HEVC. RD curves show reduction in bitrate with an improvement in SSIM of synthesized view.

Keywords

3D video Virtual view synthesis Bit allocation Perceptual quality SSIM 

References

  1. 1.
    3DV-ATM Reference Software 3DV-ATMv5.lr2. Available at: http://mpeg3dv.nokiaresearch.com/svn/mpeg3dv/tags/3DV-ATMv5.1r2/, [Online; accessed on 06-January-2017]
  2. 2.
    Bjontegaard G Calculation of average PSNR differences between RD - curves. ITU-TQ.6/SG16 VCEG 13th Meeting, Available at: http://wftp3.itu.int/av-arch/video-site/0104_Aus/
  3. 3.
    Chen HH, Huang YH, Su PY, Ou TS (2010) Improving video coding quality by perceptual rate-distortion optimization. In: IEEE international conference on multimedia and expo (ICME), 2010. IEEE, pp 1287–1292Google Scholar
  4. 4.
    Chen Z, Lin W, Ngan KN (2010) Perceptual video coding: challenges and approaches. In: Proceedings of IEEE international conference on multimedia and expo (ICME), pp 784–789Google Scholar
  5. 5.
    Cui Z, Gan Z, Zhu X (2011) Structural similarity optimal MB layer rate control for H. 264. In: Proceedings of IEEE international conference on wireless communications and signal processing (WCSP), pp 1–5Google Scholar
  6. 6.
    Fehn C (2003) A 3D-TV approach using depth-Image-Based Rendering (DIBR). In: Proceedings of VIIP, vol 3Google Scholar
  7. 7.
    Fehn C (2004) Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. In: Electronic imaging 2004. International Society for Optics and Photonics, pp 93– 104Google Scholar
  8. 8.
    Fujii Laborotory, Nagoya University. Available at: http://www.fujii.nuee.nagoya-u.ac.jp/multiview-data/, [Online; accessed on 06-January-2017]
  9. 9.
    Harshalatha Y, Biswas PK (2016) Rate distortion optimization using SSIM for 3D video coding. In: International conference on pattern recognition (ICPR)Google Scholar
  10. 10.
  11. 11.
    Huang YH, Ou TS, Chen HH (2010) Perceptual-based coding mode decision. In: Proceedings of IEEE international symposium on circuits and systems (ISCAS), pp 393–396Google Scholar
  12. 12.
    Huang YH, Ou TS, Su PY, Chen HH (2010) Perceptual rate-distortion optimization using structural similarity index as quality metric. IEEE Trans Circuits Syst Video Technol 20(11):1614– 1624CrossRefGoogle Scholar
  13. 13.
    Liu Y, Huang Q, Ma S, Zhao D, Gao W (2009) Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Signal Process Image Commun 24(8):666–681CrossRefGoogle Scholar
  14. 14.
    Mai ZY, Yang CL, Po LM, Xie SL (2005) A new rate-distortion optimization using structural information in H. 264 I-Frame Encoder. In: Advanced concepts for intelligent vision systems. Springer, pp 435– 441Google Scholar
  15. 15.
    Morvan Y, Farin D (2007) Joint depth/texture bit-allocation for multi-view video compression. In: Picture coding symposium (PCS)Google Scholar
  16. 16.
    Muller K, Merkle P, Wiegand T (2011) 3-D video representation using depth maps. Proc IEEE 99(4):643–656CrossRefGoogle Scholar
  17. 17.
    Muller K, Schwarz H, Marpe D, Bartnik C, Bosse S, Brust H, Hinz T, Lakshman H, Merkle P, Rhee H et al (2013) 3D high efficiency video coding for multi-view video and depth dataGoogle Scholar
  18. 18.
    Oh BT, Lee J, Park DS (2013) Fast joint bit-allocation between texture and depth maps for 3D video coding. In: IEEE international conference on consumer electronics (ICCE). IEEE, pp 193– 194Google Scholar
  19. 19.
    Qi J, Li X, Su F, Tu Q, Men A (2013) Efficient rate-distortion optimization for HEVC using SSIM and motion homogeneity. In: Picture coding symposium (PCS), 2013. IEEE, pp 217–220Google Scholar
  20. 20.
    Shao F, Jiang GY, Yu M, Li FC (2011) View synthesis distortion model optimization for bit allocation in three-dimensional video coding. Opt Eng 50 (12):120502–120502CrossRefGoogle Scholar
  21. 21.
    Shao F, Jiang G, Lin W, Yu M, Dai Q (2013) Joint bit allocation and rate control for coding multi-view video plus depth based 3D video. IEEE Trans Multimed 15(8):1843–1854CrossRefGoogle Scholar
  22. 22.
    Sullivan GJ, Ohm J, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22 (12):1649–1668CrossRefGoogle Scholar
  23. 23.
    Tian D, Lai PL, Lopez P, Gomila C (2009) View synthesis techniques for 3D video. In: Proceedings of the SPIE applications of digital image processing XXXII, vol 7443, pp 74,430T–74,430TGoogle Scholar
  24. 24.
    Urey H, Chellappan KV, Erden E, Surman P (2011) State of the art in stereoscopic and autostereoscopic displays. Proc IEEE 99(4):540–555CrossRefGoogle Scholar
  25. 25.
    View Synthesis Reference Software VSRS3.5. Available at: ftp://ftp.merl.com/pub/avetro/3dv-cfp/software/, [Online; accessed on 06-January-2017]
  26. 26.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612CrossRefGoogle Scholar
  27. 27.
    Wang Y, Jiang T, Ma S, Gao W (2012) Spatio-temporal ssim index for video quality assessment. In: Visual communications and image processing (VCIP), 2012 IEEE. IEEE, pp 1–6Google Scholar
  28. 28.
    Yang C, An P, Shen L (2016) Adaptive bit allocation for 3D video coding. In: Circuits, systems, and signal processing, pp 1–23Google Scholar
  29. 29.
    Yeo C, Tan HL, Tan YH (2013) On rate distortion optimization using SSIM. IEEE Trans Circuits Syst Video Technol 23(7):1170–1181CrossRefGoogle Scholar
  30. 30.
    Yuan H, Chang Y, Li M, Yang F (2010) Model based bit allocation between texture images and depth maps. In: International conference on computer and communication technologies in agriculture engineering (CCTAE), vol 3. IEEE, pp 380–383Google Scholar
  31. 31.
    Yuan H, Chang Y, Huo J, Yang F, Lu Z (2011) Model-based joint bit allocation between texture videos and depth maps for 3-D video coding. IEEE Trans Circuits Syst Video Technol 21(4):485–497CrossRefGoogle Scholar
  32. 32.
    Zhu G, Jiang G, Yu M, Li F, Shao F, Peng Z (2012) Joint video/depth bit allocation for 3D video coding based on distortion of synthesized view. In: IEEE international symposium on broadband multimedia systems and broadcasting (BMSB). IEEE, pp 1–6Google Scholar
  33. 33.
    Zitnick CL, Kang SB, Uyttendaele M, Winder S, Szeliski Rs (2004) High-quality video view interpolation using a layered representation. In: ACM transactions on graphics (TOG), vol 23. ACM, pp 600–608Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of Electronics and Electrical Communication EngineeringIndian Institute of Technology KharagpurKharagpurIndia

Personalised recommendations