Skip to main content
SpringerLink
Log in

Widely-activated network merging perceptual loss via discrete wavelet transform for image super-resolution

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

There are some problems with image super-resolution techniques, such as insufficient utilization of image features in different layers and lack high frequency information resulting in blurred texture edges in reconstructed images. A new method for the image super-resolution reconstruction of Widely Activated Network Fused Perceptual Loss (P-WAN) is proposed, which can improve the usage of image features from different layers. The method uses a pre-training model to extract features before activation to obtain the perceptual loss, meanwhile, draws on the adversarial loss in the adversarial generation network, and combines the pixel loss of the image to form a new loss function. Finally, we optimize the loss function by adjusting the weights of the three loss terms. Based on this research, a Widely Activated Network Based on Discrete Wavelet Transform and Fused Perceptual Loss (DP-WAN) is further proposed, which can reconstruct better high frequency information and higher quality texture edges. The method mainly adds the discrete wavelet transform on the basis of P-WAN, trains the different components obtained by the transform separately, and finally reconstructs the super-resolution image through the inverse discrete wavelet transform. To validate the feasibility and effectiveness, 4 representative methods are selected to test on 5 datasets. Experimental results have shown that the proposed method achieves the best performance in objective evaluation, and can obtain a good visual experience in subjective visual evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Data availability statement

We declare that the data in this article are available.

Notes

  1. https://data.vision.ee.ethz.ch/cvl/DIV2K/.

  2. https://github.com/jbhuang0604/SelfExSR/tree/master/data/Set5.

  3. https://github.com/jbhuang0604/SelfExSR/tree/master/data/Set14.

  4. https://github.com/jbhuang0604/SelfExSR/tree/master/data/BSD100.

  5. https://github.com/jbhuang0604/SelfExSR/tree/master/data/Urban100.

References

  1. Szegedy C, Vanhoucke V, Ioffe S, et al (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 2818–2826

  2. Lei S, Shi Z, Zou Z (2019) Coupled adversarial training for remote sensing image super-resolution. IEEE Trans Geosci Remote Sens 58(5):3633–3643

    Article  Google Scholar 

  3. Zhu J, Zeng H, Huang J et al (2019) Vehicle re-identification using quadruple directional deep learning features. IEEE Trans Intell Transp Syst 21(1):410–420

    Article  Google Scholar 

  4. Chen J, Chen J, Wang Z et al (2020) Identity-aware face super-resolution for low-resolution face recognition. IEEE Signal Process Lett 27:645–649

    Article  Google Scholar 

  5. Goyal B, Dogra A, Agrawal S et al (2020) Image denoising review: From classical to state-of-the-art approaches. Inform fusion 55:220–244

    Article  Google Scholar 

  6. Rippel O, Bourdev L (2017) Real-time adaptive image compression. In: International Conference on machine learning. PMLR, pp 2922–2930

  7. Dong W, Wang P, Yin W et al (2018) Denoising prior driven deep neural network for image restoration. IEEE Trans Pattern Anal Mach Intell 41(10):2305–2318

    Article  Google Scholar 

  8. Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  9. Timofte R, Gu S, Wu J, et al (2018) Ntire 2018 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE Conference on computer vision and pattern recognition workshops, pp 852–863

  10. Kawulok M, Benecki P, Piechaczek S et al (2019) Deep learning for multiple-image super-resolution. IEEE Geosci Remote Sens Lett 17(6):1062–1066

    Article  Google Scholar 

  11. Dou J, Tu Z, Peng X (2020) Single image super-resolution reconstruction with wavelet based deep residual learning. In: 2020 Chinese Control and Decision Conference (CCDC). IEEE, pp 4270–4275

  12. Hao S, Dong X (2020) Interpolation-based plane stress anisotropic yield models. Int J Mech Sci 178:105612

    Article  Google Scholar 

  13. Zhang Y, Li K, Li K, et al (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on computer vision (ECCV), pp 286–301

  14. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Communications of the ACM 60(6):84–90

    Google Scholar 

  15. Kimlyk M, Umnyashkin S (2018) Image denoising using discrete wavelet transform and edge information. In: 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus). IEEE, pp 1823–1825

  16. Dong C, Loy C C, He K, et al (2014) Learning a deep convolutional network for image super-resolution. In: European Conference on computer vision. Springer, Cham, pp 184–199

  17. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1646–1654

  18. Lim B, Son S, Kim H, et al (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on computer vision and pattern recognition workshops, pp 136–144

  19. Ting L (2019) Research on wavelet analysis and its application in image processing. In: 2019 International Conference on advanced manufacturing, computation and optimization. The Academy of Engineering and Education 1: 71–74

  20. Ping Z, Jieqing T, Lei H (2007) Image inpainting method based on discrete wavelet transformation. Appl Res Comput 24(9):287–289

    Google Scholar 

  21. Mehta R, Rajpal N, Vishwakarma VP (2018) Robust image watermarking scheme in lifting wavelet domain using GA-LSVR hybridization. Int J Mach Learn Cybern 9(1):145–161

    Article  Google Scholar 

  22. Khan H, Sharif M, Bibi N et al (2020) Localization of radiance transformation for image dehazing in wavelet domain. Neurocomputing 381:141–151

    Article  Google Scholar 

  23. Waibel A, Hanazawa T, Hinton G et al (1989) Phoneme recognition using time-delay neural networks. IEEE Trans Acoust Speech Signal Process 37(3):328–339

    Article  Google Scholar 

  24. LeCun Y, Bottou L, Bengio Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  25. Hu T, Lin X, Wang X et al (2022) Supervised learning algorithm based on spike optimization mechanism for multilayer spiking neural networks. Int J Mach Learn Cybern 13(7):1981–1995

    Article  Google Scholar 

  26. Fan Y, Shao M, Zuo W et al (2020) Unsupervised image-to-image translation using intra-domain reconstruction loss. Int J Mach Learn Cybern 11(9):2077–2088

    Article  Google Scholar 

  27. Zhang Q, Zhang M, Chen T et al (2019) Recent advances in convolutional neural network acceleration. Neurocomputing 323:37–51

    Article  Google Scholar 

  28. Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4681–4690.

  29. Yu J, Fan Y, Yang J, et al (2018) Wide activation for efficient and accurate image super-resolution. arXiv preprint arXiv:1808.08718

  30. Dong C, Loy C, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European Conference on computer vision. Springer, Cham, pp 391–407

  31. Mateo C, Talavera JA (2018) Short-time Fourier transform with the window size fixed in the frequency domain. Digital Signal Process 77:13–21

    Article  MathSciNet  Google Scholar 

  32. Liu Y, Guan L, Hou C et al (2019) Wind power short-term prediction based on LSTM and discrete wavelet transform. Appl Sci 9(6):1108

    Article  Google Scholar 

  33. Wang X, Yu K, Wu S, et al (2018) Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on computer vision (ECCV) workshops, pp 1–16

  34. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  35. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European Conference on computer vision. Springer, Cham, pp 694–711

  36. Liu J, Zhang W, Tang Y, et al (2020) Residual feature aggregation network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 2359–2368

  37. Chatterjee S, Zielinski P (2022) On the generalization mystery in deep learning. arXiv preprint arXiv:2203.10036

Download references

Acknowledgements

This work is supported by the National Natural Science Foundations of China (no. 61976216, no. 62276265, and no. 61672522).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Lili Guo, Yanru Wang, Fanchao Wang & Shifei Ding

  2. Mine Digitization Engineering Research Center of Ministry of Education of the People’s Republic of China, Xuzhou, 221116, China

    Lili Guo & Shifei Ding

  3. College of Intelligence and Computing, Tianjin University, Tianjin, 300350, China

    Ling Ding

Authors
  1. Lili Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanru Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fanchao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ling Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shifei Ding.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, L., Wang, Y., Wang, F. et al. Widely-activated network merging perceptual loss via discrete wavelet transform for image super-resolution. Int. J. Mach. Learn. & Cyber. 14, 2793–2813 (2023). https://doi.org/10.1007/s13042-023-01799-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01799-5

Keywords

Search

Navigation