Skip to main content

Optimizing Image Compression via Joint Learning with Denoising

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13679))

Included in the following conference series:

Abstract

High levels of noise usually exist in today’s captured images due to the relatively small sensors equipped in the smartphone cameras, where the noise brings extra challenges to lossy image compression algorithms. Without the capacity to tell the difference between image details and noise, general image compression methods allocate additional bits to explicitly store the undesired image noise during compression and restore the unpleasant noisy image during decompression. Based on the observations, we optimize the image compression algorithm to be noise-aware as joint denoising and compression to resolve the bits misallocation problem. The key is to transform the original noisy images to noise-free bits by eliminating the undesired noise during compression, where the bits are later decompressed as clean images. Specifically, we propose a novel two-branch, weight-sharing architecture with plug-in feature denoisers to allow a simple and effective realization of the goal with little computational cost. Experimental results show that our method gains a significant improvement over the existing baseline methods on both the synthetic and real-world datasets. Our source code is available at: https://github.com/felixcheng97/DenoiseCompression.

K. L. Cheng and Y. Xie—Joint first authors

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Gain \(\propto 1 = (10^{-2.1}, 10^{-2.6})\), Gain \(\propto 2 = (10^{-1.8}, 10^{-2.3})\), Gain \(\propto 4 = (10^{-1.4}, 10^{-1.9})\), Gain \(\propto 8 = (10^{-1.1}, 10^{-1.5})\).

References

  1. Abdelhamed, A., Lin, S., Brown, M.S.: A high-quality denoising dataset for smartphone cameras. In: Proceedings of CVPR (2018)

    Google Scholar 

  2. Al-Shaykh, O.K., Mersereau, R.M.: Lossy compression of noisy images. IEEE Trans. Image Prpcess.7(12), 1641–1652 (1998)

    Google Scholar 

  3. Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimization of nonlinear transform codes for perceptual quality. In: Proceedings of PSC (2016)

    Google Scholar 

  4. Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: Proceedings of ICLR (2017)

    Google Scholar 

  5. Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. In: Proceedings of ICLR (2018)

    Google Scholar 

  6. Bégaint, J., Racapé, F., Feltman, S., Pushparaja, A.: CompressAI,: a pytorch : a PyTorch library and evaluation platform for end-to-end compression research. arXiv:2011.03029 (2020)

  7. Bellard, F.: Bpg imagae format (2015). https://bellard.org/bpg/

  8. Bishop, C.M.: Latent variable models. In: Jordan, M.I. (eds.) Learning in Graphical Models. NATO ASI Series, vol. 89, pp. 371–403. Springer, Netherlands (1998). https://doi.org/10.1007/978-94-011-5014-9_13

  9. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1), 89–97 (2004)

    MathSciNet  MATH  Google Scholar 

  10. Chang, M., Li, Q., Feng, H., Xu, Z.: Spatial-Adaptive network for single image denoising. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12375, pp. 171–187. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58577-8_11

    Chapter  Google Scholar 

  11. Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: Proceedings of CVPR (2018)

    Google Scholar 

  12. Cheng, S., Wang, Y., Huang, H., Liu, D., Fan, H., Liu, S.: Nbnet: Noise basis learning for image denoising with subspace projection. In: Proceedings of CVPR (2021)

    Google Scholar 

  13. Cheng, Z., Sun, H., Takeuchi, M., Katto, J.: Learned image compression with discretized Gaussian mixture likelihoods and attention modules. In: Proceedings of CVPR, pp. 7939–7948 (2020)

    Google Scholar 

  14. Company, E.K.: Kodak lossless true color image suite (1999). https://www.r0k.us/graphics/Kodak

  15. Condat, L., Mosaddegh, S.: Joint demosaicking and denoising by total variation minimization. In: Proceedings of ICIP (2012)

    Google Scholar 

  16. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Color image denoising via sparse 3D collaborative filtering with grouping constraint in luminance-chrominance space. In: Proceedings of ICIP (2007)

    Google Scholar 

  17. Duda, J.: Asymmetric numeral systems. arXiv:0902.0271 (2009)

  18. Ehret, T., Davy, A., Arias, P., Facciolo, G.: Joint demosaicking and denoising by fine-tuning of bursts of raw images. In: Proceedings of ICCV, pp. 8868–8877 (2019)

    Google Scholar 

  19. Farsiu, S., Elad, M., Milanfar, P.: Multiframe demosaicing and super-resolution from undersampled color images. In: Computational Imaging II (2004)

    Google Scholar 

  20. Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.O.: Practical poissonian-gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Process. 17(10), 1737–1754 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  21. Gharbi, M., Chaurasia, G., Paris, S., Durand, F.: Deep joint demosaicking and denoising. ACM TOG 35(6), 191:1–191:12 (2016)

    Google Scholar 

  22. González, M., Preciozzi, J., Musé, P., Almansa, A.: Joint denoising and decompression using cnn regularization. In: Proceedings of CVPR Workshops (2018)

    Google Scholar 

  23. Google: Web picture format (2010). http://chromium.googlesource.com/webm/libwebp

  24. Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In: Proceedings of CVPR (2014)

    Google Scholar 

  25. Guan, H., Liu, L., Moran, S., Song, F., Slabaugh, G.G.: NODE: extreme low light raw image denoising using a noise decomposition network. arXiv:1909.05249 (2019)

  26. Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of CVPR. pp. 1712–1722 (2019)

    Google Scholar 

  27. Guo, Z., Wu, Y., Feng, R., Zhang, Z., Chen, Z.: 3-D context entropy model for improved practical image compression. In: Proceedings of CVPR Workshops, pp. 116–117 (2020)

    Google Scholar 

  28. Healey, G., Kondepudy, R.: Radiometric CCD camera calibration and noise estimation. IEEE Trans. Pattern Anal. Mach. Intell. 16(3), 267–276 (1994)

    Article  Google Scholar 

  29. Hu, Y., ,ang, W., Liu, J.: Coarse-to-fine hyper-prior modeling for learned image compression. In: Proceedings of AAAI. pp. 11013–11020 (2020)

    Google Scholar 

  30. Johnston, N., et al.: Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks. In: Proceedings of CVPR (2018)

    Google Scholar 

  31. (JVET), J.V.E.T.: VVC official test model VTM (2021). http://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware/_VTM/-/tree/master

  32. Khashabi, D., Nowozin, S., Jancsary, J., Fitzgibbon, A.W.: Joint demosaicing and denoising via learned nonparametric random fields. IEEE Trans. Image Process. 23(12), 4968–4981 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  33. Kim, Y., Soh, J.W., Park, G.Y., Cho, N.I.: Transfer learning from synthetic to real-noise denoising with adaptive instance normalization. In: Proceedings of CVPR, pp. 3482–3492 (2020)

    Google Scholar 

  34. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of ICLR (2015)

    Google Scholar 

  35. Klatzer, T., Hammernik, K., Knobelreiter, P., Pock, T.: Learning joint demosaicing and denoising based on sequential energy minimization. In: Proceedings of ICCP (2016)

    Google Scholar 

  36. Lebrun, M., Colom, M., Morel, J.: The noise clinic: a blind image denoising algorithm. Image Process. Online 5, 1–54 (2015)

    Article  MATH  Google Scholar 

  37. Lee, J., Cho, S., Beack, S.: Context-adaptive entropy model for end-to-end optimized image compression. In: Proceedings of ICLR (2019)

    Google Scholar 

  38. Lin, C., Yao, J., Chen, F., Wang, L.: A spatial RNN codec for end-to-end image compression. In: Proceedings of CVPR (2020)

    Google Scholar 

  39. Liu, J., Lu, G., Hu, Z., Xu, D.: A unified end-to-end framework for efficient deep image compression. arXiv:2002.03370 (2020)

  40. Liu, L., Jia, X., Liu, J., Tian, Q.: Joint demosaicing and denoising with self guidance. In: Proceedings of CVPR, pp. 2237–2246 (2020)

    Google Scholar 

  41. Liu, Y., et al.: Invertible denoising network: a light solution for real noise removal. In: Proceedings of CVPR, pp. 13365–13374 (2021)

    Google Scholar 

  42. Mentzer, F., Toderici, G., Tschannen, M., Agustsson, E.: High-fidelity generative image compression. In: Advances in NeurIPS (2020)

    Google Scholar 

  43. Mildenhall, B., Barron, J.T., Chen, J., Sharlet, D., Ng, R., Carroll, R.: Burst denoising with kernel prediction networks. In: Proceedings of CVPR (2018)

    Google Scholar 

  44. Minnen, D., Ballé, J., Toderici, G.: Joint autoregressive and hierarchical priors for learned image compression. In: Advances in NeurIPS, pp. 10794–10803 (2018)

    Google Scholar 

  45. Minnen, D., Singh, S.: Channel-wise autoregressive entropy models for learned image compression. In: Proceedings of ICIP (2020)

    Google Scholar 

  46. Norkin, A., Birkbeck, N.: Film grain synthesis for AV1 video codec. In: Proceedings of DCC, pp. 3–12 (2018)

    Google Scholar 

  47. Plotz, T., Roth, S.: Benchmarking denoising algorithms with real photographs. In: Proceedings of CVPR (2017)

    Google Scholar 

  48. Ponomarenko, N.N., Krivenko, S.S., Lukin, V.V., Egiazarian, K.O., Astola, J.: Lossy compression of noisy images based on visual quality: A comprehensive study. EURASIP 2010 (2010)

    Google Scholar 

  49. Preciozzi, J., González, M., Almansa, A., Musé, P.: Joint denoising and decompression: a patch-based bayesian approach. In: Proceedings of ICIP (2017)

    Google Scholar 

  50. Rabbani, M.: Jpeg 2000: Image compression fundamentals, standards and practice. J. Electron. Imag. 11(2), 286 (2002)

    Article  Google Scholar 

  51. Ren, C., He, X., Wang, C., Zhao, Z.: Adaptive consistency prior based deep network for image denoising. In: Proceedings of CVPR, pp. 8596–8606 (2021)

    Google Scholar 

  52. Rissanen, J., Langdon, G.G.: Universal modeling and coding. IEEE Trans. Inf. Theory 27(1), 12–23 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  53. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  54. Testolina, M., Upenik, E., Ebrahimi, T.: Towards image denoising in the latent space of learning-based compression. In: Applications of Digital Image Processing XLIV. vol. 11842, pp. 412–422 (2021)

    Google Scholar 

  55. Theis, L., Shi, W., Cunningham, A., Huszár, F.: Lossy image compression with compressive autoencoders. In: Proceedings of ICLR (2017)

    Google Scholar 

  56. Toderici, G., et al.: Variable rate image compression with recurrent neural networks. In: Proceedings of ICLR (2016)

    Google Scholar 

  57. Toderici, G., et al.: Workshop and challenge on learned image compression (2021). http://compression.cc/

  58. Toderici, G., et al.: Full resolution image compression with recurrent neural networks. In: Proceedings of CVPR (2017)

    Google Scholar 

  59. Vandewalle, P., Krichane, K., Alleysson, D., Süsstrunk, S.: Joint demosaicing and super-resolution imaging from a set of unregistered aliased images. In: Digital Photography III (2007)

    Google Scholar 

  60. Wallace, G.K.: The jpeg still picture compression standard. IEEE TCE :38(1), xviii–xxxiv (1992)

    Google Scholar 

  61. Wang, W., Chen, X., Yang, C., Li, X., Hu, X., Yue, T.: Enhancing low light videos by exploring high sensitivity camera noise. In: Proceedings of ICCV (2019)

    Google Scholar 

  62. Wang, Y., Huang, H., Xu, Q., Liu, J., Liu, Y., Wang, J.: Practical deep raw image denoising on mobile devices. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12351, pp. 1–16. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58539-6_1

    Chapter  Google Scholar 

  63. Wang, Z., Simoncelli1, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: Proceedings of ACSSC (2003)

    Google Scholar 

  64. Wei, K., Fu, Y., Yang, J., Huang, H.: A physics-based noise formation model for extreme low-light raw denoising. In: Proceedings of CVPR (2020)

    Google Scholar 

  65. Xie, Y., Cheng, K.L., Chen, Q.: Enhanced invertible encoding for learned image compression. In: Proceedings of ACM MM, pp. 162–170 (2021)

    Google Scholar 

  66. Xing, W., Egiazarian, K.O.: End-to-end learning for joint image demosaicing, denoising and super-resolution. In: Proceedings of CVPR, pp. 3507–3516 (2021)

    Google Scholar 

  67. Xu, X., Ye, Y., Li, X.: Joint demosaicing and super-resolution (JDSR): Network design and perceptual optimization. IEEE Trans. Image Prpcess. 6, 968–980 (2020)

    Google Scholar 

  68. Yue, Z., Zhao, Q., Zhang, L., Meng, D.: Dual adversarial network: toward real-world noise removal and noise generation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12355, pp. 41–58. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58607-2_3

    Chapter  Google Scholar 

  69. Zhang, K., Zuo, W., Zhang, L.: FfdNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Prpcess. 27(9), 4608–4622 (2018)

    Article  MathSciNet  Google Scholar 

  70. Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of CVPR (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qifeng Chen .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cheng, K.L., Xie, Y., Chen, Q. (2022). Optimizing Image Compression via Joint Learning with Denoising. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13679. Springer, Cham. https://doi.org/10.1007/978-3-031-19800-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19800-7_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19799-4

  • Online ISBN: 978-3-031-19800-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics