Advertisement

High Dynamic Range Video Coding

  • Konstantinos Konstantinides
  • Guan-Ming Su
  • Neeraj Gadgil
Chapter

Abstract

Methods for the efficient coding of high-dynamic range (HDR) still-images and video sequences are reviewed. In dual-layer techniques, a base layer of standard-dynamic range data is enhanced by additional image data in an enhancement layer. The enhancement layer may be additive or multiplicative. If there is no requirement for backward compatibility, adaptive HDR-to-standard dynamic range (SDR) mapping schemes in the encoder allow for improved coding efficiency versus the backward-compatible schemes. In single-layer techniques, a base layer is complemented by metadata, such as supplementary enhancement information (SEI) data or color remapping information (CRI) data, which allow a decoder to apply special “reshaping” or inverse-mapping functions to the base layer to reconstruct an approximation of the original HDR signal. New standards for exchanging HDR signals, such as SMPTE 2084 and BT. 2100, define new mapping functions for translating linear scene light captured by a camera to video and are replacing the traditional “gamma” mapping. The effect of those transforms to existing coding standards, such as high efficiency video coding (HEVC) and beyond, are reviewed, and novel quantization and coding schemes that take these new mapping functions into consideration are also presented.

References

  1. 1.
    G. Ward and M. Simmons, "JPEG-HDR: A Backwards-Compatible, High Dynamic Range Extension to JPEG," ACM SIGGRAPH 2006.Google Scholar
  2. 2.
    A. Artusi et al. "JPEG XT: A compression standard for HDR and WCG images," IEEE Signal Processing Magazine, pp. 118-124, March 2016.Google Scholar
  3. 3.
    T. Richter, T. Bruylants, P. Schelkens, and T. Ebrahimi, “The JPEG XT Suite of standards: Status and Future Plans,” SPIE Optical Engineering+ Applications, International Society for Optics and Photonics, Sept. 2015.Google Scholar
  4. 4.
    Report ITU-R BT. 2390-0, "High dynamic range television for production and international programme exchange," ITU, 2016.Google Scholar
  5. 5.
    W. Gish and S. Miller, "Unambiguous video pipeline description motivated by HDR." In Proc. IEEE Intern. Conf. on Image Processing (ICIP 2016), pp. 909-912. IEEE, 2016.Google Scholar
  6. 6.
    P.G.J. Barten, “Contrast sensitivity of the human eye and its effects on image quality,” SPIE Optical Engineering Press: Bellingham, WA, 1999.CrossRefGoogle Scholar
  7. 7.
    S. Miller et al., "Perceptual Signal Coding for More Efficient Usage of Bit Codes," SMPTE Motion Imaging Journal, vol. 122:(4), pp. 52-59, May-June 2013.Google Scholar
  8. 8.
    Rec. ITU-R BT. 2100, "Image parameter values for high dynamic range television for use in production and international programme exchange," ITU, July 2016.Google Scholar
  9. 9.
    Rec. ITU-R BT. 1866, "Reference electro-optical transfer function for flat panel displays used in HDTV studio production," ITU, 03/2011.Google Scholar
  10. 10.
    R. Mantiuk, A. Efremov, K. Myszkowski, and H.-P. Seidel, "Backward Compatible High Dynamic Range MPEG Video Compression," ACM Trans. on Graphics 25(3):713-723, July 2006.Google Scholar
  11. 11.
    Z. Mai, H. Mansour, R. Mantiuk, P. Nasiopoulos, R. K. Ward and W. Heidrich, “Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image/Video Compression,” IEEE Trans. on Image Processing, Vol. 20, No. 6, pp. 1558 – 1571, June 2011.Google Scholar
  12. 12.
    G-M. Su, R. Atkins, and Q. Chen, "Backward-Compatible Coding for Ultra High Definition Video Signals with Enhanced Dynamic Range," US 9,549,207, January 17, 2017.Google Scholar
  13. 13.
    Q. Chen, G-M. Su, and P. Yin, “Near Constant-Time Optimal Piecewise LDR to HDR Inverse Tone Mapping,” IS&T/SPIE Electronic Imaging, 2015.Google Scholar
  14. 14.
    G-M. Su, S. Qu, H. Koepfer, Y. Yuan, and S. Hulyalkar, “Multiple Color Channel Multiple Regression Predictor," US 8,811,490 B2, 2014.Google Scholar
  15. 15.
    P. Bordes, P. Andrivon, X. Li, Y. Ye, and Y. He, "Overview of Color Gamut Scalability," IEEE Trans. on Circuits and Systems for Video Technology, March 2016.Google Scholar
  16. 16.
    G-M. Su, S. Qu, W. Gish, H. Koepfer, Y. Yuan, and S. Hulyalkar, “Image Prediction based on Primary Color Grading Model," US 8,731,287 B2, 2014.Google Scholar
  17. 17.
    "ASC Color Decision List (ASC CDL) Transfer Functions and Interchange Syntax," ASC Technology Committee Digital Intermediate Subcommittee, 2008Google Scholar
  18. 18.
    ITU Rec. H.265, “High efficiency video coding," Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services – Coding of Moving Video, ITU, Dec 2016.Google Scholar
  19. 19.
    S. Lasserre, E. François, F. Le Léannec, and D. Touzé, “Single-layer HDR video coding with SDR backward compatibility,” SPIE Optical Engineering+ Applications (pp. 997108-997108), September, 2016.Google Scholar
  20. 20.
    R. Goris, R. Brondijk, R. van der Vleuten, “Philips response to CfE for HDR and WCG,” m36266, ISO/IEC JTC1/SC29/WG11, Warsaw, Poland, July 2015.Google Scholar
  21. 21.
    G-M. Su, S. Qu, S. Hulyalkar, T. Chen, W. Gish, and H. Koepfer, “Layered Decomposition in Hierarchical VDR Coding," US 9,497,456 B2, November 15, 2016.Google Scholar
  22. 22.
    D. Flynn, D. Marpe, M. Naccari, T. Nguyen, C. Rosewarne, K. Sharman, J., and J. Xu "Overview of the range extensions for the HEVC standard: Tools, profiles, and performance," IEEE Trans. on Circuits and Systems for Video Technology, vol. 26, no. 1, pp 4-19, January, 2016.Google Scholar
  23. 23.
    F. Dufaux, P. Le Callet, R. Mantiuk, and M. Mrak, eds. “High Dynamic Range Video: From Acquisition, to Display and Applications, “Academic Press, 2016.Google Scholar
  24. 24.
    E. Francois, C. Gisquet, G. Laroche, P. Onno, “AHG18: On 16-bits Support for Range Extensions, Document,” JCTVC-N0142, 14th JCT-VC Meeting Vienna, Austria, Jul-Aug. 2013.Google Scholar
  25. 25.
    W. S. Kim, W. Pu, J. Chen, Y. K. Wang, J. Sole, M. Karczewicz, “AHG 5 and 18: High Bit-Depth Coding Using Auxiliary Picture, Document,” JCTVC-O0090, 15th JCT-VC Meeting, Geneva, Switzerland, Oct.-Nov. 2013.Google Scholar
  26. 26.
    A. Aminlou, K. Ugar, “On 16 Bit coding,” Document JCTVC-P0162, 16th JCT-VC Meeting, San Jose, CA, Jan. 2014.Google Scholar
  27. 27.
    C. Auyeung, J. Xu, “AHG 5 and 18, Coding of High Bit-Depth Source with Lower Bit-Depth Encoders and a Continuity Mapping,” Document JCTVC-P0173, 16th JCT-VC Meeting, San Jose, CA, Jan. 2014.Google Scholar
  28. 28.
    S. Lasserre, F. Le Leannec, P. Lopez, Y. Olivier, D. Touze, E. Francois, “High Dynamic Range Video Coding,” JCTVC-P0159 (m32076), 16th JCT-VC Meeting, San Jose, CA, Jan. 2014.Google Scholar
  29. 29.
    F. Le Leannec, S. Lasserre, E. Francois, D. Touze, P. Andrivon, P. Bordes, Y. Olivier, “Modulation Channel Information SEI Message,” Document JCTVC-R0139 (m33776), 18th JCT-VC Meeting, Sapporo, Japan, Jun.-Jul. 2014.Google Scholar
  30. 30.
    K. Sharman, N. Saunders, and J. Gamei, “AHG5 and 18:Internal Precision for High Bit Depths,” document JCTVC-N0188, 14th Meeting, JCT-VC, Vienna, Austria, Jul. 2013.Google Scholar
  31. 31.
    M. Karczewicz and R. Joshi, “AHG18: Limiting the Worst-Case Length for Coeff_Abs_Level_Remaining Syntax Element to 32 Bits,” document JCTVC-Q0131, 17th Meeting, JCT-VC, Valencia, Spain, Apr. 2014.Google Scholar
  32. 32.
    K. Sharman, N. Saunders, and J. Gamei, “AHG5 and AHG18: Entropy Coding Throughput for High Bit Depths,” document JCTVC-O0046, 15th Meeting, JCT-VC, Geneva, Switzerland, Oct. 2013.Google Scholar
  33. 33.
    A. Luthra, E. Francois, W. Husak, “Call for Evidence (CfE) for HDR and WCG Video Coding”, MPEG2014/N15083, 110th MPEG Meeting, Geneva, 2015.Google Scholar
  34. 34.
    K. Minoo, T. Lu, P. Yin, L. Kerofsky, D. Rusanovskyy, E. Francois, “Description of the Exploratory Test Model (ETM) for HDR/WCG extension of HEVC”, JCT-VC Doc. W0092, San Diego, CA, Feb. 2016.Google Scholar
  35. 35.
    L. Kerofsky, Y. Ye, and Y. He. "Recent developments from MPEG in HDR video compression," IEEE Intern. Conf. on Image Processing (ICIP), pp. 879-883. IEEE, 2016.Google Scholar
  36. 36.
    T. Lu, F. Pu, P. Yin, Y. He, L. Kerofsky, Y. Ye, Z. Gu, D. Baylon, “Compression Efficiency Improvement over HEVC Main 10 Profile for HDR and WCG Content,” Proc. of the IEEE Data Compression Conference (DCC), Snowbird, March 2016.Google Scholar
  37. 37.
    C. Wong, G-M. Su, M. Wu, “Joint Baseband Signal Quantization and Transform Coding for High Dynamic Range Video,” IEEE Signal Processing Letters, 2016.Google Scholar
  38. 38.
    T. Lu, F. Pu, P. Yin, J. Pytlarz, T. Chen, and W. Husak. "Adaptive reshaper for high dynamic range and wide color gamut video compression," SPIE Optical Engineering+ Applications, pp. 99710B-99710B, International Society for Optics and Photonics, 2016.Google Scholar
  39. 39.
    T. Lu, F. Pu, P. Yin, T. Chen, W. Husak, J. Pytlarz, R. Atkins, J. Fröhlich, G-M. Su, “ITP Colour Space and its Compression Performance for High Dynamic Range and Wide Colour Gamut Video Distribution,” ZTE Communications, Feb. 2016.Google Scholar
  40. 40.
    J. Ström, J. Samuelsson, and K. Dovstam, “Luma Adjustment for High Dynamic Range Video,” Proc. of the IEEE Data Compression Conference (DCC), Snowbird, March 2016.Google Scholar
  41. 41.
    T. Lu, P. Yin, T. Chen, and G-M. Su, "Rate Control Adaptation for High-Dynamic Range Images," U.S. Patent Application Publication US 2016/0134870, 2016.Google Scholar
  42. 42.
    J. Samuelsson et al., “Conversion and coding practices for HDR/WCG YCbCr 4:2:0 video with PQ transfer characteristics,” Draft new Supplement 15 to the H-Series of Recommendations, JCTVC-Z1017, 26-th meeting, Geneva, CH, Jan. 2017.Google Scholar
  43. 43.
    J. Ström, K. Andersson, M. Pettersson, P. Hermansson, J. Samuelsson, A. Segall, J. Zhao, S-H. Kim, K. Misra, A. M. Tourapis, Y. Su, and D. Singer, “High Quality HDR Video Compression using HEVC Main 10 Profile,” in Proc. of the IEEE Picture Coding Symposium (PCS), Nuremberg, 2016.Google Scholar
  44. 44.
    A. Norkin, “Fast algorithm for HDR video pre-processing,” in Proc. of the IEEE Picture Coding Symposium (PCS), Nuremberg, 2016.Google Scholar
  45. 45.
    R. Mantiuk, K. J. Kim, A. G. Rempel, and W. Heidrich. "HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions," ACM Trans. on Graphics (TOG), vol. 30, no. 4, p. 40. ACM, 2011.Google Scholar
  46. 46.
    M. Narwaria, M. P. Da Silva, and P. Le Callet. "HDR-VQM: An objective quality measure for high dynamic range video," Signal Processing: Image Communication, Vol. 35, pp. 46-60, 2015.Google Scholar
  47. 47.
    J. Froehlich, G-M. Su, S. Daly, A. Schilling, and B. Eberhardt. "Content aware quantization: Requantization of high dynamic range baseband signals based on visual masking by noise and texture," IEEE International Conf. on Image Processing (ICIP), pp. 884-888. IEEE, 2016.Google Scholar
  48. 48.
    S. Daly, "A visual model for optimizing the design of image processing algorithms," Proc. Intern. Conf. on Image Processing, (ICIP-94), vol. 2, pp. 16-20, 1994.Google Scholar
  49. 49.
    A. Lukin, "Improved visible differences predictor using a complex cortex transform," International Conf. on Computer Graphics and Vision, 2009.Google Scholar
  50. 50.
    Blu-Ray Disc Read-only Format, “Audio Visual Application Format Specifications for BD-ROM Version 3.1,” White Paper, August 2016, Blu-Ray Disc Association.Google Scholar
  51. 51.
    Rec. ITU-R BT. 2020-1, “Parameter values for ultra-high definition television systems for production and international programme exchange,” ITU, June 2014.Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  • Konstantinos Konstantinides
    • 1
  • Guan-Ming Su
    • 1
  • Neeraj Gadgil
    • 1
  1. 1.Dolby LaboratoriesSan FranciscoUSA

Personalised recommendations