Advertisement

Quantization Guided JPEG Artifact Correction

Conference paper
  • 588 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12353)

Abstract

The JPEG image compression algorithm is the most popular method of image compression because of it’s ability for large compression ratios. However, to achieve such high compression, information is lost. For aggressive quantization settings, this leads to a noticeable reduction in image quality. Artifact correction has been studied in the context of deep neural networks for some time, but the current methods delivering state-of-the-art results require a different model to be trained for each quality setting, greatly limiting their practical application. We solve this problem by creating a novel architecture which is parameterized by the JPEG file’s quantization matrix. This allows our single model to achieve state-of-the-art performance over models trained for specific quality settings. ...

Keywords

JPEG Discrete Cosine Transform Artifact correction Quantization 

Notes

Acknowledgement

This project was partially supported by Facebook AI and Defense Advanced Research Projects Agency (DARPA) MediFor program (FA87501620191). There is no collaboration between Facebook and DARPA.

Supplementary material

504445_1_En_18_MOESM1_ESM.pdf (82.8 mb)
Supplementary material 1 (pdf 84794 KB)

References

  1. 1.
    Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: Dataset and study. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 126–135 (2017)Google Scholar
  2. 2.
    Ahmed, N., Natarajan, T., Rao, K.R.: Discrete cosine transform. IEEE Trans. Comput. 100(1), 90–93 (1974)Google Scholar
  3. 3.
    Arbelaez, P., et al.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2010)CrossRefGoogle Scholar
  4. 4.
    Bell, S., et al.: Material recognition in the wild with the materials in context database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3479–3487 (2015)Google Scholar
  5. 5.
    Cavigelli, L., Hager, P., Benini, L.: CAS-CNN: a deep convolutional neural network for image compression artifact suppression. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 752–759. IEEE (2017)Google Scholar
  6. 6.
    Chen, H., et al.: DPW-SDNet: dual pixel-wavelet domain deep CNNs for soft decoding of JPEG-compressed images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 711–720 (2018)Google Scholar
  7. 7.
    Deguerre, B., Chatelain, C., Gasso, G.: Fast object detection in compressed JPEG Images. In: arXiv preprint arXiv:1904.08408 (2019)
  8. 8.
    Dong, C., et al.: Compression artifacts reduction by a deep convolutional network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 576–584 (2015)Google Scholar
  9. 9.
    Ehrlich, M., Davis, L.S.: Deep residual learning in the JPEG transform domain. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3484–3493 (2019)Google Scholar
  10. 10.
    Foi, A., Katkovnik, V., Egiazarian, K: Pointwise shape-adaptive DCT for high-quality deblocking of compressed color images. In: 14th European Signal Processing Conference, pp. 1–5. IEEE (2006)Google Scholar
  11. 11.
    Fu, X., et al.: JPEG artifacts reduction via deep convolutional sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2501–2510 (2019)Google Scholar
  12. 12.
    Galteri, L., et al.: Deep generative adversarial compression artifact removal. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4826–4835 (2017)Google Scholar
  13. 13.
    Galteri, L., et al.: Deep universal generative adversarial compression artifact removal. IEEE Trans. Multimedia (2019)Google Scholar
  14. 14.
    Ghosh, A., Chellappa, R.: Deep feature extraction in the DCT domain. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 3536–3541. IEEE (2016)Google Scholar
  15. 15.
    Independant JPEG Group. libjpeg. http://libjpeg.sourceforge.net
  16. 16.
    Gueguen, L., et al.: Faster neural networks straight from JPEG. In: Advances in Neural Information Processing Systems, pp. 3933–3944 (2018)Google Scholar
  17. 17.
    Guo, J., Chao, H.: Building dual-domain representations for compression artifacts reduction. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 628–644. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_38CrossRefGoogle Scholar
  18. 18.
    He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  19. 19.
    He, K., et al.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)Google Scholar
  20. 20.
    Jin, Z., et al.: Dual-stream multi-path recursive residual network for JPEG image compression artifacts reduction. IEEE Trans. Circ. Syst. Video Technol. (2020)Google Scholar
  21. 21.
    Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. In: arXiv preprint arXiv:1807.00734 (2018)
  22. 22.
    Kang, D., Dhar, D., Chan, A.B.: Crowd counting by adapting convolutional neural networks with side information. In: arXiv preprint arXiv:1611.06748 (2016)
  23. 23.
    Kim, Y., Soh, J.W., Cho, N.I.K.: AGARNet: adaptively gated JPEG compression artifacts removal network for a wide range quality factor. IEEE Access 8, 20160–20170 (2020)Google Scholar
  24. 24.
    Kim, Y., et al.: A pseudo-blind convolutional neural network for the reduction of compression artifacts. IEEE Trans. Circuits Syst. Video Technol. 30(4), 1121–1135 (2019)CrossRefGoogle Scholar
  25. 25.
    Kingma, D.P. Ba, J.: Adam: A method for stochastic optimization. In: arXiv preprint arXiv:1412.6980 (2014)
  26. 26.
    LeCun, Y., et al.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, pp. 396–404 (1990)Google Scholar
  27. 27.
    Liu, P., et al.: Multi-level wavelet-CNN for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 773–782 (2018)Google Scholar
  28. 28.
    Liu, X., et al.: Data-driven sparsity-based restoration of JPEG- compressed images in dual transform-pixel domain. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5171–5178 (2015)Google Scholar
  29. 29.
    Lo, S.-Y., Hang, H.-M.: Exploring semantic segmentation on the DCT representation. In: Proceedings of the ACM Multimedia Asia on ZZZ, pp. 1–6 (2019)Google Scholar
  30. 30.
    Loshchilov, I., Hutter, F.: SGDR: Stochastic gradient descent with warm restarts. In: arXiv preprint arXiv:1608.03983 (2016)
  31. 31.
    Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, no. 1, p. 3 (2013)Google Scholar
  32. 32.
    Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Wallach, H., et al. (eds.) Advances in Neural Information Processing Systems 32, pp. 8024–8035. Curran Associates Inc (2019). http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
  33. 33.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: arXiv preprint arXiv:1511.06434 (2015)
  34. 34.
    Rawzor. Image Compression Benchmark. http://imagecompression.info/
  35. 35.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  36. 36.
    Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Processing 15(11), 3440–3451 (2006)Google Scholar
  37. 37.
    Sheikh, H.R., et al.: LIVE image quality assessment database. http://live.ece.utexas.edu/research/quality
  38. 38.
    Smith, B.: Fast software processing of motion JPEG video. In: Proceedings of the Second ACM International Conference on Multimedia, pp. 77–88. ACM. (1994)Google Scholar
  39. 39.
    Sutskever, I., Hinton, G.E., Krizhevsky, A.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  40. 40.
    Svoboda, P., et al.: Compression artifacts removal using convolutional neural networks. In: arXiv preprint arXiv:1605.00366 (2016)
  41. 41.
    Tadala, T., Narayana, S.E.V.: A Novel PSNR-B Approach for Evaluating the Quality of De-blocked Images (2012)Google Scholar
  42. 42.
    Tai, Y., et al.: Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4539–4547 (2017)Google Scholar
  43. 43.
    Wallace, G.K.: The JPEG still picture compression standard. IEEE Trans. Consumer Electron. 38(1), xviii–xxxiv (1992)Google Scholar
  44. 44.
    Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Yu., Loy, C.C.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 63–79. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-11021-5_5CrossRefGoogle Scholar
  45. 45.
    Wang, Z., et al.: D3: deep dual-domain based fast restoration of JPEG-compressed images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2764–2772 (2016)Google Scholar
  46. 46.
    Wang, Z., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  47. 47.
    Yang, S., et al.: Blocking artifact free inverse discrete cosine transform. In: Proceedings 2000 International Conference on Image Processing (Cat. No. 00CH37101), vol. 3, pp. 869–872. IEEE (2000)Google Scholar
  48. 48.
    Zhang, K., et al.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Zhang, X., et al.: DMCNN: dual-domain multi-scale convolutional neural network for compression artifacts removal. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 390–394. IEEE (2018)Google Scholar
  50. 50.
    Zhang, Y., et al.: Residual dense network for image restoration. IEEE Trans. Pattern Anal. Mach. Intell. (2020)Google Scholar
  51. 51.
    Zheng, B., et al.: Implicit dual-domain convolutional network for robust color image compression artifact reduction. IEEE Trans. Circ. Syst. Video Technol. (2019)Google Scholar
  52. 52.
    Zheng, B., et al.: S-Net: a scalable convolutional neural network for JPEG compression artifact reduction. J. Electron. Imaging 27(4), 043037 (2018)Google Scholar
  53. 53.
    Zini, S., Bianco, S., Schettini, R.: Deep Residual Autoencoder for quality independent JPEG restoration. In: arXiv preprint arXiv:1903.06117 (2019)

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of MarylandCollege ParkUSA
  2. 2.Facebook AINew YorkUSA

Personalised recommendations