Skip to main content
SpringerLink
Log in

Deep primitive convolutional neural network for image super resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep networks have emerged as a dominant solution in many research areas recently. Numerous approaches based on deep networks have been developed for image super-resolution problems with good performance. The Super-resolution Convolutional Neural Network model attempts to direct feature learning from low to high-resolution images but sometimes fails in network training when noisy examples are presented. In this work, we propose a super-resolution model that exploits an image represented by the deep structures while characterized by the primitive prior. More specifically, we discuss the use of traditional sparse representation being still sensible and combine processing of primitive prior with deep learning structure to attain further enhanced results. In addition, we apply the sparsity of primitive prior to the super-resolution problem and generate a sharp-edged high-resolution image from low-resolution image. The primitive priors are more informative structures, which can retain the high-resolution image effectively. The network simulation based on primitive prior leads to more effective network training and fine-tuning of the network. On evaluating the super-resolution model on different low-resolution images, an enhanced performance over existing algorithms is achieved in quantitative and qualitative validations. The quantitative results show that the proposed network outshines the other state-of-the-art approaches, surpassing the metric values. Qualitative analysis using the primitive network model achieves good visual quality concerning ridge and corner details.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Ahn N, Kang B, Sohn K (2022) Efficient deep neural network for photo-realistic image super-resolution. Pattern Recognit. 127:108649. https://doi.org/10.1016/j.patcog.2022.108649

    Article  Google Scholar 

  2. Benecki P, Kawulok M, Kostrzewa D, Skonieczny L (2018) Evsaluating super-resolution reconstruction of satellite images. Acta Astronaut 153:15–25. https://doi.org/10.1016/j.actaastro.2018.07.035

    Article  Google Scholar 

  3. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel M (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. BMVC. https://doi.org/10.5244/C.26.135

    Article  Google Scholar 

  4. Chang H, Yeung D, Xiong Y (2004) Super-resolution through neighbor embedding. In: 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) I-I

  5. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image superresolution

  6. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307. https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  7. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Springer, 391–407

  8. Duong V V, Huu TN, Yim J, Jeon B (2021) A fast and efficient super-resolution network using hierarchical dense residual learning. In: 2021 IEEE International Conference on Image Processing (ICIP), 1809–1813 https://doi.org/10.1109/ICIP42928.2021.9506786

  9. Esmaeilzehi A, Ahmad M O, Swamy M (2021) SRNHARB: A deep light-weight image super resolution network using hybrid activation residual blocks. Signal Process: Image Commun. https://doi.org/10.1016/j.image.2021.116509

  10. Fang F, Li J, Zeng T (2020) Soft-Edge Assisted Network for Single Image Super-Resolution. IEEE Trans Image Process 29:4656–4668. https://doi.org/10.1109/TIP.2020.2973769

    Article  Google Scholar 

  11. Fattal R (2007) Image upsampling via imposed edge statistics. In ACM Trans on Graphics 26:95–102

    Article  Google Scholar 

  12. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65. https://doi.org/10.1109/38.988747

    Article  Google Scholar 

  13. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: 2009 IEEE 12th International Conference on Computer Vision, 349–356 https://doi.org/10.1109/ICCV.2009.5459271.

  14. M. S. Greeshma and V. R. Bindu. (2017) Single image super-resolution: An efficient approach using auto-learning and filter pooling. In: IEEE International Conference on Intelligent Sustainable Systems (ICISS), 20–27

  15. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV), 1026–1034. https://doi.org/10.1109/ICCV.2015.123

  16. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R.B , Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: 2014 22nd ACM international conference on Multimedia. https://doi.org/10.1145/2647868.2654889

  17. Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133. https://doi.org/10.1109/TPAMI.2010.25

    Article  Google Scholar 

  18. Kim J, Lee J K, Lee K M (2016) Accurate image super-resolution using very deep convolutional networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1646–1654. https://doi.org/10.1109/CVPR.2016.182

  19. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1637–1645. https://doi.org/10.1109/CVPR.2016.181

  20. Lai W, Huang J, Ahuja N, Yang M (2017) Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5835–5843 https://doi.org/10.1109/CVPR.2017.618

  21. Li X, Orchard M (2001) New edge-directed interpolation. IEEE trans on image process : a publication of the IEEE Signal Processing Society 10:1521–1527

    Article  Google Scholar 

  22. Liu H, Lu Z, Shi W, Tu J (2020) A fast and accurate super-resolution network using progressive residual learning. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1818–1822

  23. Lu J, Sun Y (2015) Context-aware single image super-resolution using sparse representation and cross-scale similarity. Signal Process: Image Commun 32:40–53. https://doi.org/10.1016/j.image.2015.01.005

    Article  Google Scholar 

  24. Lv Z, Jia Y, Zhang, (2017) Joint image registration and point spread function estimation for the super-resolution of satellite images. Signal Processing: Image Commun 53:199–221. https://doi.org/10.1016/j.image.2017.08.006

    Article  Google Scholar 

  25. Lyu K, Pan S, Li Y, Zhang Z (2022) JSENet: A deep convolutional neural network for joint image super-resolution and enhancement. Neurocomputing 489:570–583. https://doi.org/10.1016/j.neucom.2021.12.071

    Article  Google Scholar 

  26. Ma C, Rao Y, Cheng Y, Chen C, Lu J, Zhou J (2020) Structure-preserving super resolution with gradient guidance. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7766–7775. https://doi.org/10.1109/cvpr42600.2020.00779

  27. Matsui Y, Ito K, Aramaki Y, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76. https://doi.org/10.1007/s11042-016-4020-z

  28. Mittal A, Moorthy AK, Bovik AC (2012) No-Reference Image Quality Assessment in the Spatial Domain. IEEE Trans Image Process 21:4695–4708. https://doi.org/10.1109/TIP.2012.2214050

    Article  MathSciNet  Google Scholar 

  29. Nair V, Hinton G E (2010) Rectified linear units improve restricted boltzmann machines. ICML. 807–814

  30. Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468

    Article  Google Scholar 

  31. Qifang X, Guoqing Y, Pin L (2017) Super-resolution Reconstruction of Satellite Video Images Based on Interpolation Method. Procedia Computer Science 107:454–459. https://doi.org/10.1016/j.procs.2017.03.089

    Article  Google Scholar 

  32. Qiu D, Zhang S, Liu Y, Zhu J, Zheng L (2020) Super-resolution reconstruction of knee magnetic resonance imaging based on deep learning. Comput Methods Programs Biomed 187:105509

    Article  Google Scholar 

  33. Shocher A, Cohen N, Irani M (2018) Zero-shot super-resolution using deep internal learning. CVPR. https://doi.org/10.1109/CVPR.2018.00329

    Article  Google Scholar 

  34. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556

  35. Song W, Li L, Ren Z (2019) Ultrasonic image processing based on fusion super-resolution reconstruction of familiar models. Journal of Visual Commun and Image Representation 64:102–633. https://doi.org/10.1016/j.jvcir.2019.102633

    Article  Google Scholar 

  36. Sun J, Sun J, Xu Z, Shum H (2011) Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans on Image Process 20:1529–1542

    Article  MathSciNet  Google Scholar 

  37. Sun J, Xu Z, Shum H (2008) Image super-resolution using gradient profile prior. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2471– 2478 https://doi.org/10.1109/CVPR.2008.4587659

  38. Sun J, Zheng N, Tao H, Shum H (2003) Image hallucination with primal sketch priors. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. II-729

  39. Tai Y, Liu S, Brown M S, Lin S (2010) Super resolution using edge prior and single image detail synthesis. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2400–2407

  40. Tian C, Xu Y, Zuo W, Zhang B, Fei L, Lin C-W (2020) Coarse-to-fine cnn for image super-resolution. IEEE Trans Multimed:1–1. https://doi.org/10.1109/TMM.2020.2999182

  41. Timofte R, De V, Gool LV (2013) Anchored neighborhood regression for fast example-based superresolution. In: 2013 IEEE international conference on computer vision, 1920–1927. https://doi.org/10.1109/ICCV.2013.241

  42. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  43. Wang Z, Liu D, Yang J, Han W, Huang TS (2015) Deep networks for image super-resolution with sparse prior. IEEE International Conference on Computer Vision (ICCV) 2015:370–378

    Article  Google Scholar 

  44. Wang C, Ning X, Sun L, Zhang L, Li W, Bai X (2022) Learning discriminative features by covering local geometric space for point cloud analysis. IEEE Trans on Geoscience and Remote Sens 60:1–15. https://doi.org/10.1109/tgrs.2022.3170493

    Article  Google Scholar 

  45. Wang C, Wang X, Zhang J, Zhang L, Bai X, Ning X, Zhou J, Hancock ER (2022) Uncertainty estimation for stereo matching based on evidential deep learning. Pattern Recognit. 124:108498. https://doi.org/10.1016/j.patcog.2021.108498

    Article  Google Scholar 

  46. Wang S, Zhou T, Lu Y, Di H (2022) Detail-preserving transformer for light field image super-resolution. AAAI. https://doi.org/10.1609/aaai.v36i3.20153

    Article  Google Scholar 

  47. Wang S, Zhou T, Lu Y, Di H (2022) Contextual transformation network for lightweight remote-sensing image super-resolution. IEEE Trans Geoscience and Remote Sensing 60:1–13. https://doi.org/10.1109/tgrs.2021.3132093

    Article  Google Scholar 

  48. Wei S, Zhou X, Wu W, QiangPu QW, Yang X (2018) Medical images per-resolution by using multi-dictionary and random forest. Sustain Cities Soc 37:358–370. https://doi.org/10.1016/j.scs.2017.11.012

    Article  Google Scholar 

  49. Yang J, Wright J, Huang T S, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1–8

  50. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  Google Scholar 

  51. Yang J, Wang Z, Lin Z, Cohen S, Huang T (2012) Coupled dictionary training for image super-resolution. IEEE Trans Image Process 21(8):3467–3478. https://doi.org/10.1109/TIP.2012.2192127

    Article  MathSciNet  Google Scholar 

  52. Yang W, Feng J, Yang J, Zhao F, Liu J, Guo Z, Yan S (2017) Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans on Image Process 26:5895–5907. https://doi.org/10.1109/TIP.2017.2750403

    Article  MathSciNet  Google Scholar 

  53. Yang W, Zhou F, Zhu R, Fukui K, Wang G, Xue J (2020) Deep learning for image super-resolution. Neurocomputing 398:291–292. https://doi.org/10.1016/j.neucom.2019.09.091

    Article  Google Scholar 

  54. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. 711–730. https://doi.org/10.1007/978-3-642-27413-847

  55. Zhou Y, Du X, Wang M, Huo S, Zhang Y, Kung SY (2022) Cross-Scale Residual Network: A General Framework for Image Super-Resolution, Denoising, and Deblocking. IEEE Trans Cybernetics 52:5855–5867. https://doi.org/10.1109/TCYB.2020.3044374

Download references

Acknowledgements

The authors acknowledge the support extended by the Department of Science & Technology- Promotion of University Research and Scientific Excellence (PURSE) Phase II, Govt of India.

Author information

Authors and Affiliations

  1. School of Computer Sciences, Mahatma Gandhi University, Kottayam, Kerala, India

    Greeshma M. S. & Bindu V. R.

Authors
  1. Greeshma M. S.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bindu V. R.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bindu V. R..

Ethics declarations

Conflict of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

S., G.M., R., B.V. Deep primitive convolutional neural network for image super resolution. Multimed Tools Appl 83, 253–278 (2024). https://doi.org/10.1007/s11042-023-15661-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15661-x

Keywords

Search

Navigation