Skip to main content

Implicit Neural Representations for Image Compression

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Abstract

Implicit Neural Representations (INRs) gained attention as a novel and effective representation for various data types. Recently, prior work applied INRs to image compressing. Such compression algorithms are promising candidates as a general purpose approach for any coordinate-based data modality. However, in order to live up to this promise current INR-based compression algorithms need to improve their rate-distortion performance by a large margin. This work progresses on this problem. First, we propose meta-learned initializations for INR-based compression which improves rate-distortion performance. As a side effect it also leads to leads to faster convergence speed. Secondly, we introduce a simple yet highly effective change to the network architecture compared to prior work on INR-based compression. Namely, we combine SIREN networks with positional encodings which improves rate distortion performance. Our contributions to source compression with INRs vastly outperform prior work. We show that our INR-based compression algorithm, meta-learning combined with SIREN and positional encodings, outperforms JPEG2000 and Rate-Distortion Autoencoders on Kodak with 2x reduced dimensionality for the first time and closes the gap on full resolution images. To underline the generality of INR-based source compression, we further perform experiments on 3D shape compression where our method greatly outperforms Draco - a traditional compression algorithm.

Y. Strümpler and J. Postels—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/vsitzmann/siren.

  2. 2.

    https://quic.github.io/.

  3. 3.

    https://github.com/google/draco.

References

  1. Kodak lossless true color image suite. www.r0k.us/graphics/kodak/

  2. Agustsson, E., et al.: Soft-to-hard vector quantization for end-to-end learning compressible representations. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS), pp. 1141–1151 (2017)

    Google Scholar 

  3. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: dataset and study. In: Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017

    Google Scholar 

  4. Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., Gool, L.V.: Generative adversarial networks for extreme learned image compression. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 221–231 (2019)

    Google Scholar 

  5. Alemi, A., Poole, B., Fischer, I., Dillon, J., Saurous, R.A., Murphy, K.: Fixing a broken ELBO. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 159–168. PMLR. 10–15 July 2018. www.proceedings.mlr.press/v80/alemi18a.html

  6. Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: International Conference on Learning Representations (ICLR) (2017)

    Google Scholar 

  7. Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. In: International Conference on Learning Representations (ICLR) (2018)

    Google Scholar 

  8. Bengio, Y.: Estimating or propagating gradients through stochastic neurons (2013)

    Google Scholar 

  9. Bird, T., Ballé, J., Singh, S., Chou, P.A.: 3d scene compression through entropy penalized neural representation functions. In: 2021 Picture Coding Symposium (PCS), pp. 1–5. IEEE (2021)

    Google Scholar 

  10. Chai, S.M.: Quantization-guided training for compact TinyML models. arXiv preprint arXiv:2103.06231 (2021)

  11. Chen, H., He, B., Wang, H., Ren, Y., Lim, S.N., Shrivastava, A.: NeRV: neural representations for videos. In: Thirty-Fifth Conference on Neural Information Processing Systems (2021)

    Google Scholar 

  12. Chen, Y., Liu, S., Wang, X.: Learning continuous image representation with local implicit image function. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  13. Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  14. Cheng, Z., Sun, H., Takeuchi, M., Katto, J.: Learned image compression with discretized gaussian mixture likelihoods and attention modules. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  15. Choi, Y., El-Khamy, M., Lee, J.: Variable rate deep image compression with a conditional autoencoder. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3146–3154 (2019)

    Google Scholar 

  16. Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (1996)

    Google Scholar 

  17. Davies, T., Nowrouzezahrai, D., Jacobson, A.: On the effectiveness of weight-encoded neural implicit 3D shapes (2021)

    Google Scholar 

  18. Dong, Z., Yao, Z., Gholami, A., Mahoney, M.W., Keutzer, K.: HAWQ: hessian aware quantization of neural networks with mixed-precision. In: International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  19. Dupont, E., Golinski, A., Alizadeh, M., Teh, Y.W., Doucet, A.: COIN: compression with implicit neural representations. In: Neural Compression, From Information Theory to Applications - Workshop (ICLR) (2021)

    Google Scholar 

  20. Dupont, E., Loya, H., Alizadeh, M., Goliński, A., Teh, Y.W., Doucet, A.: Coin++: data agnostic neural compression. arXiv preprint arXiv:2201.12904 (2022)

  21. Fan*, A., et al.: Training with quantization noise for extreme model compression (2020)

    Google Scholar 

  22. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning (ICLR) (2017)

    Google Scholar 

  23. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR (2017)

    Google Scholar 

  24. Habi, H.V., Jennings, R.H., Netzer, A.: HMQ: hardware friendly mixed precision quantization block for CNNs. In: European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

  25. Habibian, A., Rozendaal, T.V., Tomczak, J.M., Cohen, T.S.: Video compression with rate-distortion autoencoders. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7033–7042 (2019)

    Google Scholar 

  26. Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural network with pruning, trained quantization and huffman coding. In: International Conference on Learning Representations, (ICLR) (2016)

    Google Scholar 

  27. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504-507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  28. Hinton, G.E., Zemel, R.S.: Autoencoders, minimum description length, and helmholtz free energy. Advances in neural information processing systems 6, 3–10 (1994)

    Google Scholar 

  29. Hu, Y., Yang, W., Liu, J.: Coarse-to-fine hyper-prior modeling for learned image compression. In: Conference on Artificial Intelligence (AAAI) (2020)

    Google Scholar 

  30. Johnston, N., et al.: Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4385–4393 (2018)

    Google Scholar 

  31. Kim, T., Yoo, Y., Yang, J.: FrostNet: towards quantization-aware network architecture search (2020)

    Google Scholar 

  32. Kingma, D.P., Ba, J.L.: Adam: a method for stochastic gradient descent. In: Proceedings of the International Conference on Learning Representations (ICLR), pp. 1–15 (2015)

    Google Scholar 

  33. Lee, J., Cho, S., Beack, S.K.: Context-adaptive entropy model for end-to-end optimized image compression. In: Proceedings of the International Conference on Learning Representations (ICLR) (2019)

    Google Scholar 

  34. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: International Conference on Computer Vision (ICCV), December 2015

    Google Scholar 

  35. Louizos, C., Reisser, M., Blankevoort, T., Gavves, E., Welling, M.: Relaxed quantization for discretized neural networks. In: International Conference on Learning Representations (ICLR) (2019)

    Google Scholar 

  36. Menghani, G.: Efficient deep learning: a survey on making deep learning models smaller, faster, and better (2021)

    Google Scholar 

  37. Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., Van Gool, L.: Conditional probability models for deep image compression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4394–4402 (2018)

    Google Scholar 

  38. Mentzer, F., Toderici, G.D., Tschannen, M., Agustsson, E.: High-fidelity generative image compression. In: Advances in Neural Information Processing Systems (NeuIPS), vol. 33 (2020)

    Google Scholar 

  39. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3d reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4460–4470 (2019)

    Google Scholar 

  40. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: representing scenes as neural radiance fields for view synthesis. In: European Conference on Computer Vision (ECCV) (2020)

    Google Scholar 

  41. Minnen, D., Ballé, J., Toderici, G.: Joint autoregressive and hierarchical priors for learned image compression. In: Advances in Neural Information Processing Systems (NeurIPS) (2018)

    Google Scholar 

  42. Nagel, M., Amjad, R.A., van Baalen, M., Louizos, C., Blankevoort, T.: Up or down? adaptive rounding for post-training quantization (2020)

    Google Scholar 

  43. Nagel, M., van Baalen, M., Blankevoort, T., Welling, M.: Data-free quantization through weight equalization and bias correction. In: International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  44. Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165–174 (2019)

    Google Scholar 

  45. Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  46. van Rozendaal, T., Huijben, I.A., Cohen, T.: Overfitting for fun and profit: instance-adaptive data compression. In: International Conference on Learning Representations (ICLR) (2021)

    Google Scholar 

  47. Shannon, C.A.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  48. Sitzmann, V., Chan, E.R., Tucker, R., Snavely, N., Wetzstein, G.: MetaSDF: meta-learning signed distance functions. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    Google Scholar 

  49. Sitzmann, V., Martel, J.N., Bergman, A.W., Lindell, D.B., Wetzstein, G.: Implicit neural representations with periodic activation functions. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    Google Scholar 

  50. Skorokhodov, I., Ignatyev, S., Elhoseiny, M.: Adversarial generation of continuous images. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  51. Takikawa, T., et al.: Neural geometric level of detail: real-time rendering with implicit 3D shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  52. Tancik, M., et al.: Learned initializations for optimizing coordinate-based neural representations. In: CVPR (2021)

    Google Scholar 

  53. Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    Google Scholar 

  54. Toderici, G., et al.: Variable rate image compression with recurrent neural networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2016)

    Google Scholar 

  55. Toderici, G., et al.: Full resolution image compression with recurrent neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5306–5314 (2017)

    Google Scholar 

  56. Uhlich, S., et al.: Mixed precision DNNs: all you need is a good parametrization. In: International Conference on Learning Representations (ICLR) (2019)

    Google Scholar 

  57. Wang, K., Liu, Z., Lin, Y., Lin, J., Han, S.: HAQ: hardware-aware automated quantization with mixed precision. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  58. Xie, Y., Cheng, K.L., Chen, Q.: Enhanced invertible encoding for learned image compression. In: ACM International Conference on Multimedia (2021)

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by Google.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yannick Strümpler .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 16802 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Strümpler, Y., Postels, J., Yang, R., Gool, L.V., Tombari, F. (2022). Implicit Neural Representations for Image Compression. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13686. Springer, Cham. https://doi.org/10.1007/978-3-031-19809-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19809-0_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19808-3

  • Online ISBN: 978-3-031-19809-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics