Skip to main content
SpringerLink
Log in

Weak texture information map guided image super-resolution with deep residual networks

  • 1168: Deep Pattern Discovery for Big Multimedia Data
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Limited by the poor quality of the camera, transmission bandwidth, excessive compression and other factors, low-resolution images widely exist in our lives. Single image super-resolution method is a kind of image processing task which can obtain high-resolution image from corresponding source low-resolution image. With the development of deep learning technology, a series of deep learning methods have brought crucial improvement for SISR problem. However, we observe that no matter how deep the network structures for SISR are designed, they usually have poor performances on restoration of tiny or irregular details, we call them weak texture information. The main reason for this phenomenon is that weak texture information is not obvious relative to the salient features, so as weak texture information feature is less extracted in the process of neural network feature extraction. To address this problem, we propose a SISR method which owns unique weak texture information prediction module and call it as weak texture information map guided image super-resolution with deep residual networks. In our network structure, two auxiliary sub-networks work together to capture grab details information and predict weak texture information. Then, predicted weak texture information will be fused into main sub-network. Therefore, our network can obtain more irregular details which usually miss in deep learning based methods. Finally, both qualitative experiments and visual effects demonstrate the effectiveness of our proposed algorithm. Specifically, our method can restore more irregular image details.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ahn N, Kang B, Sohn K A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. 2018. European conference on computer vision, pp 252–268

  2. Bevilacqua M, Roumy A, Guillemot C et al (2012) Low complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference 135:1–10

  3. Dai S, Han M, Xu W, Gong Y (2007) Soft edge smoothness prior for Alpha Channel super resolution. 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8

  4. Dong C, Loy CC, He K, Yang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

  5. Fu B, Li Y, Wang XH (2019) Image super-resolution using TV priori guided convolutional network. Pattern Recognit Lett 125(JUL.):780–784

    Article  Google Scholar 

  6. Getreuer P (2011) Linear methods for image interpolation. Image Processing on Line 1:239–259

  7. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image, computer vision. 2009 IEEE 12th international conference on. IEEE, pp 349–356

  8. Ha VK, Ren JC, Xu XY, Zhao S, Hussain A (2019) Deep learning based single image super-resolution: a survey. Int J Autom Comput 16:413–426

    Article  Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 770–778

  10. Jiang J, Ma X, Chen C, Lu T, Wang Z, Ma J (2017) Single image super-resolution via locally regularized anchored neighborhood regression and nonlocal means. IEEE Trans Multimed 19(1):15–26

    Article  Google Scholar 

  11. Keys RG (2003) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29(6):1153–1160

    Article  MathSciNet  Google Scholar 

  12. Kim J, Lee J K, Lee K M (2016) Accurate image super-resolution using very deep convolutional networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1646–1654

  13. Kim J, Lee J K, Lee K M (2016) Deeply-recursive convolutional network for image super-resolution. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1637–1645

  14. Lai W, Huang J, Ahuja N, Yang M (2019) Fast and accurate image super-resolution with deep Laplacian pyramid networks. IEEE Trans Pattern Anal Mach Intell 41(11):2599–2613

    Article  Google Scholar 

  15. Lim B, Son S, Kim H, Nah S, Lee K M (2017) Enhanced deep residual networks for single image super-resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 1132–1140

  16. Jianchao Yang, John Wright, Thomas S Huang, Yi Ma (2010) Image super-resolution via sparse representation. IEEE Trans Image Process. 19 (11):2861–73. https://doi.org/10.1109/TIP.2010.2050625

  17. Mao XJ, Shen CH, Yang YB (2016) Image restoration using convolutional auto-encoders with symmetric skip connections 2016 proceedings of the 30th international conference on neural information processing SystemsDecember (NIPS) 2810–2818

  18. Sajjadi M S M, Schlkopf B, Hirsch M (2017) EnhanceNet: single image super-resolution through automated texture synthesis. 2017 IEEE International Conference on Computer Vision (ICCV), pp 4501–4510

  19. Sun J, Xu Z, Shum H Y (2008) Image super-resolution using gradient profile prior. IEEE Conference on Computer Vision & Pattern Recognition, pp 1–8

  20. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2790–2798

  21. Tai Y, Yang J, Liu X, Xu C (2017) MemNet: a persistent memory network for image restoration. 2017 IEEE International Conference on Computer Vision (ICCV), pp 4549–4557

  22. Timofte R, Smet V D, Gool L V (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. 2014 Asian Conference on Computer Vision, pp 111–126

  23. Timofte R, Rothe R, Van Gool L (2016) Seven ways to improve example-based single image super resolution. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1865–1873

  24. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. 2017 IEEE International Conference on Computer Vision (ICCV), pp 4809–4817

  25. Wang Y (2020) Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion, arXiv:2006.08159, ACM TOMM

  26. Wang Y, Lin X, Wu L, Zhang W, Zhang Q, Huang X (2015) Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Trans Image Process 24(11):3939–3949

    Article  MathSciNet  Google Scholar 

  27. Wu L, Wang Y, Shao L (2019) Cycle-consistent deep generative hashing for cross-modal retrieval. IEEE Trans Image Process 28(4):1602–1612

    Article  MathSciNet  Google Scholar 

  28. Wu L, Wang Y, Gao J, Li X (2019) Where-and-when to look: deep Siamese attention networks for video-based person re-identification. IEEE Trans Multimedia 21(6):1412–1424

    Article  Google Scholar 

  29. Wu L, Wang Y, Yin H, Wang M, Shao L (2020) Few-shot deep adversarial learning for video-based person re-identification. IEEE Trans Image Process 29:1233–1245

    Article  MathSciNet  Google Scholar 

  30. Wu L, Wang Y, Gao J et al (2020) Deep Coattention-based comparator for relative representation learning in person re-identification. IEEE Trans Neural Netw Learn Syst

  31. Yamanaka J, Kuwashima K, Kurita T (2017) Fast and accurate image super resolution by deep CNN with skip connection and network in network. 2017 in Proceedings of the 24th international conference on neural information processing, Springer, pp 217-225

  32. Yang C Y, Huang J B, Yang M H (2010) Exploiting self-similarities for single frame super-resolution. Asian Conference on Computer Vision Springer-Verlag, pp 497–510

  33. Yang W, Xia J, Guo Z (2018) Reference-guided deep super-resolution via manifold localized external compensation. IEEE Trans Circuits Syst Video Technol 29(5):1270–1283

    Article  Google Scholar 

  34. Yu J, Fan Y, Yang J et al (2019) Wide Activation for Efficient and Accurate Image Super-Resolution [Online]. https://arxiv.org/abs/1808.08718v1. Accessed 8 Apr 2019

  35. Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep CNN Denoiser prior for image restoration. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2808–2817

  36. Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3262–3271

  37. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2472–2481

  38. Zhang Y, Li K, Li K, Wang L, Fu Y (2018) Image super-resolution using very deep Residual Channel attention networks: 15th European conference, Munich, Germany, September 8–14, 2018, proceedings, part VII 286-301

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (NSFC) Grant No.61702246, No.61976109, China Postdoctoral Science Foundation, No. 2019 M651123 and Science and Technology Innovation Fund (Youth Science and Technology Star) of Dalian, China, No. 2018RQ65. Liaoning Natural Science Foundation (No.20180550542), Dalian Science and Technology Innovation Fund (No.2018J12GX047), Dalian Key Laboratory Special Fund.

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Liaoning Normal University, Dalian, China

    Bo Fu, Liyan Wang, Yuechu Wu, Yufeng Wu, Shilin Fu & Yonggong Ren

Authors
  1. Bo Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liyan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuechu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yufeng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shilin Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yonggong Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Fu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, B., Wang, L., Wu, Y. et al. Weak texture information map guided image super-resolution with deep residual networks. Multimed Tools Appl 81, 34281–34294 (2022). https://doi.org/10.1007/s11042-021-11085-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11085-7

Keywords

Search

Navigation