Skip to main content
SpringerLink
Log in

A Novel Attention Enhanced Dense Network for Image Super-Resolution

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11961))

Included in the following conference series:

  • 2826 Accesses

Abstract

Deep convolutional neural networks (CNNs) have recently achieved impressive performance in image super-resolution (SR). However, they usually treat the spatial features and channel-wise features indiscriminatingly and fail to take full advantage of hierarchical features, restricting adaptive ability. To address these issues, we propose a novel attention enhanced dense network (AEDN) to adaptively recalibrate each kernel and feature for different inputs, by integrating both spatial attention (SA) and channel attention (CA) modules in the proposed network. In experiments, we explore the effect of attention mechanism and present quantitative and qualitative evaluations, where the results show that the proposed AEDN outperforms state-of-the-art methods by effectively suppressing the artifacts and faithfully recovering more high-frequency image details.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bevilacqua, M., Roumy, A., Guillemot, C., Morel, M.L.A.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the British Machine Vision Conference (BMVC) (2012)

    Google Scholar 

  2. Chen, L., et al.: SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5659–5667. IEEE (2017)

    Google Scholar 

  3. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  4. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural networkD. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 391–407. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_25

    Chapter  Google Scholar 

  5. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7132–7141. IEEE (2018)

    Google Scholar 

  6. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4700–4708. IEEE (2017)

    Google Scholar 

  7. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5197–5206. IEEE (2015)

    Google Scholar 

  8. Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654. IEEE (2016)

    Google Scholar 

  9. Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1637–1645. IEEE (2016)

    Google Scholar 

  10. Kingma, D., Ba., J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2015)

    Google Scholar 

  11. Ledig, C., Theis, L., Huszár, et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4681–4690. IEEE (2017)

    Google Scholar 

  12. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 136–144. IEEE (2017)

    Google Scholar 

  13. Mao, X., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Neural Information Processing Systems (NIPS), pp. 2802–2810 (2016)

    Google Scholar 

  14. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). IEEE (2001)

    Google Scholar 

  15. Matsui, Y., Ito, K., Aramaki, Y., Fujimoto, A., Ogawa, T., Yamasaki, T., Aizawa, K.: Sketch-based manga retrieval using Manga109 dataset. Multimedia Tools Appl. (MTA) 76(20), 21811–21838 (2017)

    Article  Google Scholar 

  16. Paszke, A., Gross, S., Chintala, S., et al.: Automatic differentiation in PyTorch (2017)

    Google Scholar 

  17. Shi, W., Caballero, J., Huszár, F., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1874–1883. IEEE (2016)

    Google Scholar 

  18. Shi, W., et al.: Cardiac image super-resolution with global correspondence using multi-atlas PatchMatch. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8151, pp. 9–16. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40760-4_2

    Chapter  Google Scholar 

  19. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3147–3155. IEEE (2017)

    Google Scholar 

  20. Tai, Y., Yang, J., Liu, X., Xu, C.: MemNet: a persistent memory network for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4539–4547. IEEE (2017)

    Google Scholar 

  21. Thornton, M.W., Atkinson, P.M., Holland, D.: Sub-pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super-resolution pixel-swapping. IJRS 27(3), 473–491 (2006)

    Google Scholar 

  22. Timofte, R., De Smet, V., Van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 111–126. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16817-3_8

    Chapter  Google Scholar 

  23. Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 4809–4817. IEEE (2017)

    Google Scholar 

  24. Wang, F., et al.: Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  25. Xu, H., Saenko, K.: Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 451–466. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_28

    Chapter  Google Scholar 

  26. Xu, K., Ba, J., Kiros, et al.: Show, attend and tell: neural image caption generation with visual attention. In: Proceedings of the 32nd International Conference on Machine Learning (ICML), pp. 2048–2057 (2015)

    Google Scholar 

  27. Yang, C.Y., Yang, M.H.: Fast direct super-resolution by simple functions. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 561–568. IEEE (2013)

    Google Scholar 

  28. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. TIP 19(11), 2861–2873 (2010)

    MathSciNet  MATH  Google Scholar 

  29. Yang, Z., He, X., Gao, J., Deng, L., Smola, A.: Stacked attention networks for image question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21–29. IEEE (2016)

    Google Scholar 

  30. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27413-8_47

    Chapter  Google Scholar 

  31. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3929–3938. IEEE (2017)

    Google Scholar 

  32. Zhang, K., Gao, X., Tao, D., Li, X., et al.: Single image super-resolution with non-local means and steering kernel regression. TIP 11, 12 (2012)

    MATH  Google Scholar 

  33. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_18

    Chapter  Google Scholar 

  34. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2472–2481. IEEE (2018)

    Google Scholar 

  35. Zou, W.W., Yuen, P.C.: Very low resolution face recognition problem. TIP 21(1), 327–340 (2012)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is funded by the National Natural Science Foundation of China (No. 61673204), and the Fundamental Research Funds for the Central Universities (No. 14380046).

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Zhong-Han Niu, Yang-Hao Zhou & Yu-Bin Yang

  2. Provincial Key Laboratory for Information Technology of Wisdom Mining of Shandong Province, Shandong University of Science and Technology, Qingdao, 266590, China

    Jian-Cong Fan

Authors
  1. Zhong-Han Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang-Hao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu-Bin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian-Cong Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhong-Han Niu or Yu-Bin Yang .

Editor information

Editors and Affiliations

  1. Korea Advanced Institute of Science and, Daejeon, Korea (Republic of)

    Yong Man Ro

  2. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  3. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Junmo Kim

  4. National Cheng Kung University, Tainan City, Taiwan

    Wei-Ta Chu

  5. Tsinghua University, Beijing, China

    Peng Cui

  6. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Jung-Woo Choi

  7. National Tsing Hua University, Hsinchu, Taiwan

    Min-Chun Hu

  8. Ghent University, Ghent, Belgium

    Wesley De Neve

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Niu, ZH., Zhou, YH., Yang, YB., Fan, JC. (2020). A Novel Attention Enhanced Dense Network for Image Super-Resolution. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11961. Springer, Cham. https://doi.org/10.1007/978-3-030-37731-1_46

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37731-1_46

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37730-4

  • Online ISBN: 978-3-030-37731-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation