ASDN: A Deep Convolutional Network for Arbitrary Scale Image Super-Resolution

Abstract

Deep convolutional neural networks have significantly improved the peak signal-to-noise ratio of Super-Resolution (SR). However, image viewer applications commonly allow users to zoom the images to arbitrary magnification scales, thus far imposing a large number of required training scales at a tremendous computational cost. To obtain a more computationally efficient model for arbitrary-scale SR, this paper employs a Laplacian pyramid method to reconstruct any-scale high-resolution (HR) images using the high-frequency image details in a Laplacian Frequency Representation. For SR of small-scales (between 1 and 2), images are constructed by interpolation from a sparse set of precalculated Laplacian pyramid levels. SR of larger scales is computed by recursion from small scales, which significantly reduces the computational cost. For a full comparison, fixed- and any-scale experiments are conducted using various benchmarks. At fixed scales, ASDN outperforms predefined upsampling methods (e.g., SRCNN, VDSR, DRRN) by about 1 dB in PSNR. At any-scale, ASDN generally exceeds Meta-SR on many scales.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: Dataset and study. In: The IEEE conference on computer vision and pattern recognition (CVPR) workshops, vol 3, p 2

  2. 2.

    Bevilacqua M, Roumy A, Guillemot C, Alberi-morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding

  3. 3.

    Burt PJ, Adelson EH (1987) The laplacian pyramid as a compact image code. In: Readings in computer vision, pp 671–679. Elsevier

  4. 4.

    Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision, pp 184–199. Springer

  5. 5.

    Gao G, Zhu D, Yang M, Lu H, Yang W, Gao H (2018) Face image super-resolution with pose via nuclear norm regularized structural orthogonal procrustes regression. Neural Comput & Applic, 1–11

  6. 6.

    Haris M, Shakhnarovich G, Ukita N (2018) Deep backprojection networks for super-resolution. In: Conference on computer vision and pattern recognition

  7. 7.

    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  8. 8.

    Hu X, Mu H, Zhang X, Wang Z, Tan T, Sun J (2019) Meta-sr: a magnification-arbitrary network for super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1575–1584

  9. 9.

    Hu Y, Li J, Huang Y, Gao X (2019) Channel-wise and spatial feature modulation network for single image super-resolution. IEEE Transactions on Circuits and Systems for Video Technology

  10. 10.

    Huang G, Sun Y, Liu Z, Sedra D, Weinberger KQ (2016) Deep networks with stochastic depth. In: European conference on computer vision, pp 646–661. Springer

  11. 11.

    Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 5197–5206

  12. 12.

    Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1646–1654

  13. 13.

    Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate superresolution. In: IEEE Conference on computer vision and pattern recognition, vol 2, p 5

  14. 14.

    Lai WS, Huang JB, Ahuja N, Yang MH (2018) Fast and accurate image super-resolution with deep laplacian pyramid networks IEEE transactions on pattern analysis and machine intelligence

  15. 15.

    Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR) workshops, vol 1, p 4

  16. 16.

    Liu Y, Wang Y, Li N, Cheng X, Zhang Y, Huang Y, Lu G (2018) An attention-based approach for single image super resolution. In: 2018 24Th international conference on pattern recognition (ICPR), pp 2777–2784. IEEE

  17. 17.

    Lu H, Li Y, Nakashima S, Kim H, Serikawa S (2017) Underwater image super-resolution by descattering and fusion. IEEE Access 5:670–679

    Article  Google Scholar 

  18. 18.

    Lu T, Wang J, Zhang Y, Wang Z, Jiang J (2019) Satellite image super-resolution via multi-scale residual deep neural network. Remote Sens 11(13):1588

    Article  Google Scholar 

  19. 19.

    Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838

    Article  Google Scholar 

  20. 20.

    Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3791–3799

  21. 21.

    Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 4539–4547

  22. 22.

    Timofte R, De Smet V, Van Gool L (2014) A+: Adjusted anchored neighborhood regression for fast super-resolution. In: Asian conference on computer vision, pp 111–126. Springer

  23. 23.

    Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: Computer vision (ICCV), 2017 IEEE international conference on, pp 4809–4817. IEEE

  24. 24.

    Wang D, Lu H, Yang MH (2015) Robust visual tracking via least soft-threshold squares. IEEE Trans Circuits Sys Vid Techn 26(9):1709–1721

    Article  Google Scholar 

  25. 25.

    Wang Y, Shen J, Zhang J (2018) Deep bi-dense networks for image super-resolution. In: 2018 Digital image computing: techniques and applications (DICTA), pp 1–8. IEEE

  26. 26.

    Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. International conference on curves and surfaces. 711–730

  27. 27.

    Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. In: IEEE Conference on computer vision and pattern recognition, vol 6

  28. 28.

    Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 286–301

  29. 29.

    Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jialiang Shen.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Shen, J., Wang, Y. & Zhang, J. ASDN: A Deep Convolutional Network for Arbitrary Scale Image Super-Resolution. Mobile Netw Appl (2021). https://doi.org/10.1007/s11036-020-01720-2

Download citation

Keywords

  • Image super-resolution
  • Any-scale SR
  • Convolutional neural network