Skip to main content
SpringerLink
Log in

SR-AFU: super-resolution network using adaptive frequency component upsampling and multi-resolution features

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Image super-resolution (SR) is one of the classic computer vision tasks. This paper proposes a super-resolution network based on adaptive frequency component upsampling, named SR-AFU. The network is composed of multiple cascaded dilated convolution residual blocks (CDCRB) to extract multi-resolution features representing image semantics, and multiple multi-size convolutional upsampling blocks (MCUB) to adaptively upsample different frequency components using CDCRB features. The paper also defines a new loss function based on the discrete wavelet transform, making the reconstructed SR images closer to human perception. Experiments on the benchmark datasets show that SR-AFU has higher peak signal to noise ratio (PSNR), significantly faster training speed and more realistic visual effects compared with the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Freeman W T, Pasztor E C, Carmichael O T. Learning low-level vision. International Journal of Computer Vision, 2000, 40(1): 25–47

    Article  Google Scholar 

  2. Shi W, Caballero J, Ledig C, Zhuang X, Bai W, Bhatia K, de Marvao A M S M, Dawes T, O’Regan D, Rueckert D. Cardiac image superresolution with global correspondence using multi-atlas patchmatch. In: Proceedings of the 16th International Conference on Medical Image Computing and Computer-Assisted Intervention. 2013, 9–16

  3. Zhang S, Liang G, Pan S, Zheng L. A fast medical image super resolution method based on deep learning network. IEEE Access, 2018, 7: 12319–12327

    Article  Google Scholar 

  4. Oh J, Lee C, Seo D C. Automated HRSI georegistration using orthoimage and SRTM: focusing KOMPSAT-2 imagery. Computers & Geosciences, 2013, 52: 77–84

    Article  Google Scholar 

  5. Nogueira K, Penatti O A B, Dos Santos J A. Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recognition, 2017, 61: 539–556

    Article  Google Scholar 

  6. Hu X, Ma P, Mai Z, Peng S, Yang Z, Wang L. Face hallucination from low quality images using definition-scalable inference. Pattern Recognition, 2019, 94: 110–121

    Article  Google Scholar 

  7. Chen L C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 833–851

  8. Li P, Wang Q, Zuo W, Zhang L. Log-Euclidean kernels for sparse representation and dictionary learning. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 1601–1608

  9. Lee Y, Choe Y. Neighbor embedding based single image superresolution using hybrid feature and adaptive weight decay regularization. In: Proceedings of the 4th IEEE International Conference on Consumer Electronics Berlin. 2014, 185–187

  10. Dong C, Loy C C, He K, Tang X. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307

    Article  Google Scholar 

  11. Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1646–1654

  12. Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1637–1645

  13. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y. Residual dense network for image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 2472–2481

  14. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y. Image super-resolution using very deep residual channel attention networks. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 294–310

  15. Rad M S, Bozorgtabar B, Marti U V, Basler M, Ekenel H K, Thiran J P. SROBB: targeted perceptual loss for single image super-resolution. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 2710–2719

  16. Ng M K, Shen H, Lam E Y, Zhang L. A total variation regularization based super-resolution reconstruction algorithm for digital video. EURASIP Journal on Advances in Signal Processing, 2007, 2007: 074585

    Article  Google Scholar 

  17. Jiang K, Wang Z, Yi P, Jiang J. Hierarchical dense recursive network for image super-resolution. Pattern Recognition, 2020, 107: 107475

    Article  Google Scholar 

  18. Dong C, Loy C C, Tang X. Accelerating the super-resolution convolutional neural network. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 391–407

  19. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770–778

  20. Lai W S, Huang J B, Ahuja N, Yang M H. Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5835–5843

  21. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 105–114

  22. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Loy C C. ESRGAN: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 2018, 63–79

  23. Lim B, Son S, Kim H, Nah S, Lee K M. Enhanced deep residual networks for single image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017, 1132–1140

  24. Yu J, Fan Y, Yang J, Xu N, Wang Z, Wang X, Huang T. Wide activation for efficient and accurate image super-resolution. 2018, arXiv preprint arXiv: 1808.08718

  25. Dai T, Cai J, Zhang Y, Xia S T, Zhang L. Second-order attention network for single image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 11057–11066

  26. Zhao H, Gallo O, Frosio I, Kautz J. Loss functions for neural networks for image processing. 2018, arXiv preprint arXiv: 1511.08861

  27. Timofte R, Agustsson E, Van Gool L, Yang M H, Zhang L, et al. NTIRE 2017 challenge on single image super-resolution: methods and results. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017, 1110–1121

  28. Bevilacqua M, Roumy A, Guillemot C, Morel M L A. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of British Machine Vision Conference. 2012

  29. Zeyde R, Elad M, Protter M. On single image scale-up using sparse-representations. In: Proceedings of the 7th International Conference on Curves and Surfaces. 2010, 711–730

  30. Martin D, Fowlkes C, Tal D, Malik J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 8th IEEE International Conference on Computer Vision. 2001, 416–423

  31. Huang J B, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5197–5206

  32. Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference for Learning Representations. 2015

  33. Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4): 600–612

    Article  Google Scholar 

  34. Dong C, Loy C C, He K, Tang X. Learning a deep convolutional network for image super-resolution. In: Proceedings of the 13th European Conference on Computer Vision. 2014, 184–199

  35. Tai Y, Yang J, Liu X, Xu C. MemNet: a persistent memory network for image restoration. In: Proceedings of IEEE International Conference on Computer Vision. 2017, 4549–4557

  36. Haris M, Shakhnarovich G, Ukita N. Deep back-projection networks for super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 1664–1673

  37. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W. Feedback network for image super-resolution. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 3862–3871

  38. Hui Z, Gao X, Yang Y, Wang X. Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM International Conference on Multimedia. 2019, 2024–2032

  39. Zhao H, Kong X, He J, Qiao Y, Dong C. Efficient image superresolution using pixel attention. In: Proceedings of the European Conference on Computer Vision. 2020, 56–72

  40. Wang C, Li Z, Shi J. Lightweight image super-resolution with adaptive weighted learning network. 2019, arXiv preprint arXiv: 1904.02358

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant Nos. 61603197 and 61772284), Natural Science Foundation of Nanjing University of Posts and Telecommunications (NY221071).

Author information

Authors and Affiliations

  1. School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing, 210023, China

    Ke-Jia Chen

  2. Jiangsu Key Laboratory of Big Data Security & Intelligent Processing, Nanjing University of Posts and Telecommunications, Nanjing, 210023, China

    Ke-Jia Chen

  3. College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China

    Mingyu Wu, Yibo Zhang & Zhiwei Chen

Authors
  1. Ke-Jia Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingyu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yibo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiwei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingyu Wu.

Additional information

Ke-Jia Chen is an associate professor in Nanjing University of Posts and Telecommunications, China. She received her PhD in Université de Technologie de Compiègne, France and her master’s degree in Nanjing University, China. She joined Jiangsu Key Laboratory of Big Data Security & Intelligent Processing, China in 2017. Her current research focuses on machine learning and its applications in complex network analysis.

Mingyu Wu received the BS degree in Electronic Information Engineering from Nanjing University of Posts and Telecommunications, China in 2021. He is working toward the master degree in Signal and information processing at Nanjing University of Posts and Telecommunications, China. His current research interests include casual inference, sequence modeling and cross-modal analysis.

Yibo Zhang received the BS degree in Electronic Information Engineering from Nanjing University of Posts and Telecommunications, China in 2021. His research interests include computer vision, UAV system and automatic driving.

Zhiwei Chen received the BS degree in Electronic Information Engineering from Nanjing University of Posts and Telecommunications, China in 2021. He is working toward the master degree in Electronic information at the South China Normal University, China. His current research interests include machine learning, computer vision and Neuromorphic computation.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, KJ., Wu, M., Zhang, Y. et al. SR-AFU: super-resolution network using adaptive frequency component upsampling and multi-resolution features. Front. Comput. Sci. 17, 171307 (2023). https://doi.org/10.1007/s11704-021-0562-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-021-0562-y

Keywords

Search

Navigation