Advertisement

Deep Image Compression Using Decoder Side Information

Conference paper
  • 419 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12362)

Abstract

We present a Deep Image Compression neural network that relies on side information, which is only available to the decoder. We base our algorithm on the assumption that the image available to the encoder and the image available to the decoder are correlated, and we let the network learn these correlations in the training phase.

Then, at run time, the encoder side encodes the input image without knowing anything about the decoder side image and sends it to the decoder. The decoder then uses the encoded input image and the side information image to reconstruct the original image.

This problem is known as Distributed Source Coding (DSC) in Information Theory, and we discuss several use cases for this technology. We compare our algorithm to several image compression algorithms and show that adding decoder-only side information does indeed improve results. Our code is publicly available Our code is available at: https://github.com/ayziksha/DSIN.

Keywords

Deep Distributed Source Coding Deep Neural Networks Deep Learning Image reconstruction 

Notes

Acknowledgments

This work was partly funded by ISF grant number 1549/19.

Supplementary material

504472_1_En_41_MOESM1_ESM.pdf (23.9 mb)
Supplementary material 1 (pdf 24461 KB)

References

  1. 1.
    Aaron, A., Rane, S., Zhang, R., Girod, B.: Wyner-Ziv coding for video: applications to compression and error resilience. In: Data Compression Conference, pp. 93–102 (2003)Google Scholar
  2. 2.
    Aaron, A., Zhang, R., Girod, B.: Wyner-Ziv coding of motion video. In: Asilomar Conference on Signals, Systems and Computers, vol. 1, pp. 240–244 (2002)Google Scholar
  3. 3.
    Agustsson, E., Tschannen, M., Mentzer, F., Luc Van Gool, R.T.: Generative adversarial networks for extreme learned image compression. In: International Conference on Computer Vision (2019)Google Scholar
  4. 4.
    Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: International Conference on Learning Representations (2017)Google Scholar
  5. 5.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 24 (2009)CrossRefGoogle Scholar
  6. 6.
    Bellard, F.: BPG image format (2014). https://bellard.org/bpg/
  7. 7.
    Chambon, S., Crouzil, A.: Colour correlation-based matching. Int. J. Robot. Autom. 20, 78–85 (2005)Google Scholar
  8. 8.
    Chen, D., Varodayan, D., Flierl, M., Girod, B.: Wyner-Ziv coding of multiview images with unsupervised learning of disparity and gray code. In: IEEE International Conference on Image Processing, pp. 1112–1115 (2008)Google Scholar
  9. 9.
    Chen, Q., Xu, J., Koltun, V.: Fast image processing with fully-convolutional networks. In: International Conference on Computer Vision, pp. 2516–2525 (2017)Google Scholar
  10. 10.
    Cover, T.M.: A proof of the data compression theorem of Slepian and Wolf for ergodic sources. IEEE Trans. Inf. Theory 21(2), 226–228 (1975)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The Kitti vision benchmark suite. In: Computer Vision and Pattern Recognition (2012)Google Scholar
  12. 12.
    Girod, B., Aaron, A.M., Rane, S., Rebollo-Moneddero, D.: Distributed video coding. Proc. IEEE 93(1), 71–83 (2005)CrossRefGoogle Scholar
  13. 13.
  14. 14.
    Goyal, M., Lather, Y., Lather, V.: Analytical relation & comparison of PSNR and SSIM on babbon image and human eye perception using matlab. Int. J. Adv. Res. Eng. Appl. Sci. 4(5), 108–119 (2015)Google Scholar
  15. 15.
    Johnston, N., et al.: Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks. In: Computer Vision and Pattern Recognition (2018)Google Scholar
  16. 16.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv abs/1412.6980 (2014)Google Scholar
  17. 17.
    Le Gall, D.: MPEG: a video compression standard for multimedia applications. Commun. ACM 34(4), 46–58 (1991)CrossRefGoogle Scholar
  18. 18.
    Lu, G., Ouyang, W., Xu, D., Zhang, X., Gao, Z., Sun, M.: Deep Kalman filtering network for video compression artifact reduction. In: European Conference on Computer Vision, pp. 591–608 (2018)Google Scholar
  19. 19.
    Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., Gool, L.V.: Conditional probability models for deep image compression. In: Computer Vision and Pattern Recognition, pp. 4394–4402 (2018)Google Scholar
  20. 20.
    Menze, M., Heipke, C., Geiger, A.: Joint 3D estimation of vehicles and scene flow. ISPRS-Image Seq. Anal. (2015)Google Scholar
  21. 21.
    Menze, M., Heipke, C., Geiger, A.: Object scene flow. ISPRS - Photogram. Remote Sens. 140, 60–76 (2018)CrossRefGoogle Scholar
  22. 22.
    Ndajah, P., Kikuchi, H., Yukawa, M., Watanabe, H., Muramatsu, S.: SSIM image quality metric for denoised images. In: International Conference on Visualization, Imaging and Simulation, pp. 53–58 (2010)Google Scholar
  23. 23.
    Pradhan, S.S., Ramchandran, K.: Distributed source coding using syndromes (DISCUS): design and construction. IEEE Trans. Inf. Theory 49(3), 626–643 (2003)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Rippel, O., Bourdev, L.: Real-time adaptive image compression. In: International Conference on Machine Learning, vol. 70, pp. 2922–2930 (2017)Google Scholar
  25. 25.
    Slepian, D., Wolf, J.K.: Noiseless coding of correlated information sources. IEEE Trans. Inf. Theory 19(4), 471–480 (1973)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Taubman, D.S., Marcellin, M.W.: JPEG 2000: Image Compression Fundamentals, Standards and Practice. Kluwer Academic Publishers, Dordrecht (2001)Google Scholar
  27. 27.
    Theis, L., Shi, W., Cunningham, A., Huszár, F.: Lossy image compression with compressive autoencoders. In: International Conference on Learning Representations (2017)Google Scholar
  28. 28.
    Toderici, G., et al.: Variable rate image compression with recurrent neural networks. In: International Conference on Learning Representations (2016)Google Scholar
  29. 29.
    Toderici, G., et al.: Full resolution image compression with recurrent neural networks. In: Computer Vision and Pattern Recognition (2017)Google Scholar
  30. 30.
    Tsai, Y., Liu, M., Sun, D., Yang, M., Kautz, J.: Learning binary residual representations for domain-specific video streaming. In: Conference on Artificial Intelligence, pp. 7363–7370 (2018)Google Scholar
  31. 31.
    Varodayan, D., Lin, Y.C., Mavlankar, A., Flierl, M., Girod, B.: Wyner-Ziv coding of stereo images with unsupervised learning of disparity. In: Proceedings of Picture Coding Symposium, pp. 1–4 (2007)Google Scholar
  32. 32.
    Wallace, G.K.: The JPEG still picture compression standard. Commun. ACM 38, 30–44 (1991)CrossRefGoogle Scholar
  33. 33.
    Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: Asilomar Conference on Signals, Systems Computers, vol. 2, pp. 1398–1402 (2003)Google Scholar
  34. 34.
    Wu, C., Singhal, N., Krähenbühl, P.: Video compression through image interpolation. In: European Conference on Computer Vision, pp. 425–440 (2018)Google Scholar
  35. 35.
    Wyner, A.D., Ziv, J.: The rate-distortion function for source coding with side information at the decoder. IEEE Trans. Inf. Theory 22(1), 1–10 (1976)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Xiong, Z., Liveri, A.D., Cheng, S.: Distributed source coding for sensor networks. IEEE Signal Process. Mag. 21(5), 80–94 (2004)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Electrical EngineeringTel Aviv UniversityTel Aviv-YafoIsrael

Personalised recommendations