Skip to main content
SpringerLink
Log in

Dual-branch feature learning network for single image super-resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The feature extraction ability of some existing super-resolution networks is relatively weak. And these networks do not further process the extracted features. These problems make the networks often show limited performance, resulting in blurred details and unclear edges of the reconstructed images. Therefore, further research is needed to resolve these problems. In this paper, we propose a novel dual-branch feature learning super resolution network (DBSR). The core of DBSR is the dual-branch feature learning (DB) block. In order to enhance the ability of feature extraction, the block adopts a multi-level and dual-branch structure. At the same time, some components for further processing features are introduced in each branch to maximize the learning ability of the block. The reconstructed images of DBSR are clearer than other networks in line and contour, and better results are obtained in peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). For example, when the scaling factor is 2, the PSNR/SSIM on each test dataset is 38.25dB/0.9614, 34.11dB/0.9218, 32.35dB/0.9018, 32.94dB/0.9354 and 39.46dB/0.9783 respectively. The experimental results demonstrate that DBSR achieves better accuracy and visually pleasing than the current excellent methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The data that support the findings of this study is available from Key Laboratory of Advanced Marine Communication and Information Technology of Ministry of Industry and Information Technology but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available. Data is however available from the authors upon reasonable request and with permission of Key Laboratory of Advanced Marine Communication and Information Technology of Ministry of Industry and Information Technology.

References

  1. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: dataset and study. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1122–1131. https://doi.org/10.1109/CVPRW.2017.150

  2. Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV)

  3. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British machine vision conference (BMVC)

  4. Chang H, Yeung D-Y, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004., vol 1. https://doi.org/10.1109/CVPR.2004.1315043

  5. Deqiang C, Xin G, Liangliang C, Qiqi K, Kai Z, Rui G (2021) Image super-resolution reconstruction from multi-channel recursive residual network. J Image Graph 26(3):605–618. https://doi.org/10.11834/jig.200108

    Google Scholar 

  6. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Computer vision – ECCV 2014, pp 184–199

  7. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision – ECCV 2016, pp 391–407

  8. Freedman G, Fattal R (2011) Image and video upscaling from local self-examples. ACM Trans Graph 30(2):1–11. https://doi.org/10.1145/1944846.1944852

    Article  Google Scholar 

  9. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65. https://doi.org/10.1109/38.988747

    Article  Google Scholar 

  10. Fujimoto A, Ogawa T, Yamamoto K, Matsui Y, Yamasaki T, Aizawa K (2016) Manga109 dataset and creation of metadata. In: Proceedings of the 1st international workshop on CoMics ANalysis, processing and understanding. https://doi.org/10.1145/3011549.3011551

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  12. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 5197–5206. https://doi.org/10.1109/CVPR.2015.7299156

  13. Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 723–731. https://doi.org/10.1109/CVPR.2018.00082

  14. Irani M, Peleg S (1991) Improving resolution by image registration. CVGIP: Graph Model Image Process 53(3):231–239. https://doi.org/10.1016/1049-9652(91)90045-L

    Google Scholar 

  15. Keys R (1981) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29 (6):1153–1160. https://doi.org/10.1109/TASSP.1981.1163711

    Article  MathSciNet  MATH  Google Scholar 

  16. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1646–1654. https://doi.org/10.1109/CVPR.2016.182

  17. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1637–1645. https://doi.org/10.1109/CVPR.2016.181

  18. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 5835–5843. https://doi.org/10.1109/CVPR.2017.618

  19. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 3867–3876

  20. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops(CVPRW)

  21. Liu J, Tang J, Wu G (2020) Residual feature distillation network for lightweight image super-resolution. arXiv:2009.11551

  22. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE international conference on computer vision. ICCV 2001, vol 2, pp 416–423. https://doi.org/10.1109/ICCV.2001.937655

  23. Matsune A, Cheng G, Zhan S (2019) Dual branches network for image super-resolution. Electron Lett 55(23):1229–1231. https://doi.org/10.1049/el.2018.7562

    Article  Google Scholar 

  24. Qin J, Huang Y, Wen W (2020) Multi-scale feature fusion residual network for single image super-resolution. Neurocomputing 379:334–342. https://doi.org/10.1016/j.neucom.2019.10.076

    Article  Google Scholar 

  25. Rostami M, Berahmand K, Nasiri E, Forouzandeh S (2021) Review of swarm intelligence-based feature selection methods. Eng Appl Artif Intell 100:104210. https://doi.org/10.1016/j.engappai.2021.104210

    Article  Google Scholar 

  26. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016). In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1874–1883. https://doi.org/10.1109/CVPR.2016.207

  27. Sun J, Xu Z, Shum H-Y (2008) Image super-resolution using gradient profile prior. In: 2008 IEEE conference on computer vision and pattern recognition, pp 1–8. https://doi.org/10.1109/CVPR.2008.4587659

  28. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2790–2798. https://doi.org/10.1109/CVPR.2017.298

  29. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: 2017 IEEE international conference on computer vision (ICCV), pp 4549–4557. https://doi.org/10.1109/ICCV.2017.486

  30. Tian C, Zhuge R, Wu Z, Xu Y, Zuo W, Chen C, Lin C-W (2020) Lightweight image super-resolution with enhanced cnn. Knowl-Based Syst 205:106235. https://doi.org/10.1016/j.knosys.2020.106235

    Article  Google Scholar 

  31. Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Boissonnat J-D, Chenin P, Cohen A, Gout C, Lyche T, Mazure M-L, Schumaker L (eds) Curves and surfaces. Springer, Berlin, pp 711-730

  32. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), pp 286–301

  33. Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 3262–3271. https://doi.org/10.1109/CVPR.2018.00344

  34. Zhao H, Gallo O, Frosio I, Kautz J (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57. https://doi.org/10.1109/TCI.2016.2644865

    Article  Google Scholar 

Download references

Acknowledgements

Thanks a lot to Miss Zheng Liying for her brilliant help to the research. This work was supported by National Natural Science Foundation of China (61771155,52001096), China Postdoctoral Science Foundation (2020M681079), the Fundamental Research Funds for the Central Universities (3072021CFT0803), Heilongjiang Postdoctoral Financial Assistance (LBH-Z20128).

Author information

Authors and Affiliations

  1. College of Information and Communication Engineering, Harbin Engineering University, Harbin, 150001, China

    Lei Yu, Qiuyue Deng, Bin Liu, Haoyu Wu & Haicheng Hu

Authors
  1. Lei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiuyue Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haoyu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haicheng Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Liu.

Ethics declarations

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

For better reading of this paper, we provide some symbols used in this paper and their detailed definitions in Table 6.

Table 6 Some symbols used in the paper
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, L., Deng, Q., Liu, B. et al. Dual-branch feature learning network for single image super-resolution. Multimed Tools Appl 82, 43297–43314 (2023). https://doi.org/10.1007/s11042-023-14742-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14742-1

Keywords

Search

Navigation