Skip to main content
SpringerLink
Log in

A New Single-Image Super-Resolution Using Efficient Feature Fusion and Patch Similarity in Non-Euclidean Space

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Efficient trade-off between the reconstruction qualities and the processing time of any single-image super-resolution reconstruction (SISRR) approach is critically influenced by two major aspects. These aspects are (i) appropriate representation of image patch in feature space and (ii) effective searching of candidate patches from the pool of training patches or learned dictionary. This paper proposes a neighbor embedding-based SISRR method. Novelties of our work include integration of (i) efficient feature mapping scheme which fuses multiple correlated features naturally, (ii) faster searching of candidate patches by measuring the patch correlation in non-Euclidean space and (iii) adaptive selection of neighborhood size using patch characteristic. Correlation among features is modeled via global covariance matrix, and the fusion process enables to preserve sufficient structural, spatial correlation among patches. Distance functions based on notion of generalized eigenvalue are used for measuring patch similarity which support faster searching of candidate patches. Performance analysis of the suggested method is compared with some of the competent state-of-the-art methodologies. From the simulated result analysis, proposed work is found to be outperforming in terms of sharpened image details with diminished effect of artifacts at a reasonable computational burden.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. \({\alpha ^* }\), optimal sparse coefficient; LASSO, least absolute shrinkage and selection operator, NCC, normalized cross-correlation, G-SOMP+, generalized simultaneous OMP, HSI, hyper-spectral image, IFCM, improved fuzzy C-means.

References

  1. Hamza, R.; Hassan, A.; Huang, T.; Ke, L.; Yan, H.: An efficient cryptosystem for video surveillance in the internet of things environment. Complexity 2019, (2019)

  2. Hamza, R.; Hewage, C.; Titouna, F.: Investigation of three-dimentional image security based on improved image randomised encryption method. NED Univ. J. Res. (2018)

  3. Singh, D.; Kumar, V.; Kaur, M.: Single image dehazing using gradient channel prior. Appl. Intell. 49(12), 4276–4293 (2019)

    Article  Google Scholar 

  4. Singh, D.; Kumar, V.: Single image defogging by gain gradient image filter. Sci. China Inf. Sci. 62(7), 79101 (2019)

    Article  MathSciNet  Google Scholar 

  5. Jiang, K.; Wang, Z.; Yi, P.; Jiang, J.: A progressively enhanced network for video satellite imagery superresolution. IEEE Signal Process. Lett. 25(11), 1630–1634 (2018)

    Article  Google Scholar 

  6. Jurek, J.; Kociński, M.; Materka, A.; Elgalal, M.; Majos, A.: Cnn-based superresolution reconstruction of 3D MR images using thick-slice scans. Biocybernet. Biomed. Eng. 40(1), 111–125 (2020)

    Article  Google Scholar 

  7. Zhang, L.; Zhang, H.; Shen, H.; Li, P.: A super-resolution reconstruction algorithm for surveillance images. Signal Process. 90(3), 848–859 (2010)

    Article  MATH  Google Scholar 

  8. Freeman, W.T.; Jones, T.R.; Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  9. Sun, J.; Zheng, N.-N.; Tao, H.; Shum, H.-Y.: Image hallucination with primal sketch priors. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, vol. 2, pp. II–729 IEEE (2003)

  10. Huang, D.J.; Siebert, J.P.; Cockshott, W.P.; Xiao, Y.J.: Super-resolution of face images based on adaptive markov network. In: Third International IEEE Conference on Signal-Image Technologies and Internet-Based System, 2007, pp. 742–747. IEEE (2007)

  11. Banerjee, J.; Namboodiri, A.M.; Jawahar, C.; Contextual restoration of severely degraded document images. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, vol. 2009, 517–524. IEEE (2009)

  12. Wu, W.; Liu, Z.; Gueaieb, W.; He, X.: Single-image super-resolution based on markov random field and contourlet transform. J. Electron. Imaging 20(2), 023005–023005 (2011)

    Article  Google Scholar 

  13. Chen, X.; Qi, C.: Document image super-resolution using structural similarity and markov random field. IET Image Process. 8(12), 687–698 (2014)

    Article  Google Scholar 

  14. Nayak, R.; Krishna, L.S.; Patra, D.: Enhanced super-resolution image reconstruction using MRF model. In: Progress in Intelligent Computing Techniques: Theory, Practice, and Applications, pp. 207–215. Springer (2018)

  15. Nayak, R.; Patra, D.: New single-image super-resolution reconstruction using MRF model. Neurocomputing 293, 108–129 (2018)

    Article  Google Scholar 

  16. Chang, H.; Yeung, D.-Y.; Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, vol. 1, pp. I–I. IEEE (2004)

  17. Chan, T.-M.; Zhang, J.: Improved super-resolution through residual neighbor embedding. J. Guangxi Normal Univ. 24(4), 1–8 (2006)

    Google Scholar 

  18. Chan, T.-M.; Zhang, J.; Pu, J.; Huang, H.: Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recognit. Lett. 30(5), 494–502 (2009)

    Article  Google Scholar 

  19. Zhang, K.; Gao, X.; Li, X.; Tao, D.: Partially supervised neighbor embedding for example-based image super-resolution. IEEE J. Sel. Top. Signal Process. 5(2), 230–239 (2010)

    Article  Google Scholar 

  20. Cao, M.; Gan, Z.; Zhu, X.: Super-resolution algorithm through neighbor embedding with new feature selection and example training. In: 2012 IEEE 11th International Conference on Signal Processing (ICSP), vol. 2, pp. 825–828. IEEE (2012)

  21. Gao, X.; Zhang, K.; Tao, D.; Li, X.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21(7), 3194–3205 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Zhu, Q.; Sun, L.; Cai, C.: Non-local neighbor embedding for image super-resolution through FoE features. Neurocomputing 141, 211–222 (2014)

    Article  Google Scholar 

  23. Mishra, D.; Majhi, B.; Sa, P.K.; Dash, R.: Development of robust neighbor embedding based super-resolution scheme. Neurocomputing 202, 49–66 (2016)

    Article  Google Scholar 

  24. Mishra, D.; Majhi, B.; Sa, P.K.: Improved feature selection for neighbor embedding super-resolution using zernike moments. In: Proceedings of International Conference on Computer Vision and Image Processing, pp. 13–24. Springer (2017)

  25. Yang, J.; Wright, J.; Huang, T.S.; Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Zeyde, R.; Elad, M.; Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730. Springer, Berlin (2010)

  27. Yang, J.; Wang, Z.; Lin, Z.; Cohen, S.; Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21(8), 3467–3478 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  28. Wang, S.; Zhang, L.; Liang, Y.; Pan, Q.: Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2216–2223. IEEE (2012)

  29. Timofte, R.; De Smet, V.; Van Gool, L.: A+: Adjusted anchored neighborhood regression for fast super-resolution. In: Asian Conference on Computer Vision, pp. 111–126. Springer, Berlin (2014)

  30. Wang, L.; Geng, H.; Liu, P.; Lu, K.; Kolodziej, J.; Ranjan, R.; Zomaya, A.Y.: Particle swarm optimization based dictionary learning for remote sensing big data. Knowl.-Based Syst. 79, 43–50 (2015)

    Article  Google Scholar 

  31. Nayak, R.; Patra, D.; Harshavardhan, S.: Sparse representation based image super resolution reconstruction. In: TENCON 2015–2015 IEEE Region 10 Conference, pp. 1–6. IEEE (2015)

  32. Ahmadi, K.; Salari, E.: Single-image super resolution using evolutionary sparse coding technique. IET Image Process. 11(1), 13–21 (2016)

    Article  Google Scholar 

  33. Fang, L.; Zhuo, H.; Li, S.: Super-resolution of hyperspectral image via superpixel-based sparse representation. Neurocomputing 273, 171–177 (2018)

    Article  Google Scholar 

  34. Yang, X.; Wu, W.; Liu, K.; Chen, W.; Zhou, Z.: Multiple dictionary pairs learning and sparse representation-based infrared image super-resolution with improved fuzzy clustering. Soft Comput. 22(5), 1385–1398 (2018)

    Article  Google Scholar 

  35. Liu, N.; Xu, X.; Li, Y.; Zhu, A.: Sparse representation based image super-resolution on the knn based dictionaries. Optics Laser Technol. 110, 135–144 (2019)

    Article  Google Scholar 

  36. Dong, C.; Loy, C.C.; He, K.; Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)

    Article  Google Scholar 

  37. Kim, J.; Lee, J.K.; Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

  38. Ledig, C.; Theis, L.; Huszár, F.; Caballero, J.; Cunningham, A.; Acosta, A.; Aitken, A.; Tejani, A.; Totz, J.; Wang, Z.; et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)

  39. Wang, Y.; Wang, L.; Wang, H.; Li, P.: End-to-end image super-resolution via deep and shallow convolutional networks. IEEE Access 7, 31959–31970 (2019)

    Article  Google Scholar 

  40. Wang, Z.; Chen, J.; Hoi, S.C.: Deep learning for image super-resolution: a survey. arXiv preprint arXiv:1902.06068 (2019)

  41. Tuzel, O.; Porikli, F.; Meer, P.: Region covariance: A fast descriptor for detection and classification. In: Computer Vision-ECCV, vol. 2006, pp. 589–600 (2006)

  42. Nayak, R.; Patra, D.: Super resolution image reconstruction using penalized-spline and phase congruency. Comput. Electr. Eng. 62, 232–248 (2017)

    Article  Google Scholar 

  43. Ou, Y.; Sotiras, A.; Paragios, N.; Davatzikos, C.: Dramms: deformable registration via attribute matching and mutual-saliency weighting. Medical Image Anal. 15(4), 622–639 (2011)

    Article  Google Scholar 

  44. Zhan, Y.; Shen, D.: Deformable segmentation of 3-D ultrasound prostate images using statistical texture matching method. IEEE Trans. Med. Imaging 25(3), 256–272 (2006)

    Article  Google Scholar 

  45. Marziliano, P.; Dufaux, F.; Winkler, S.; Ebrahimi, T.: Perceptual blur and ringing metrics: application to jpeg2000. Signal Process. Image Commun. 19(2), 163–172 (2004)

    Article  Google Scholar 

  46. Tang, L.; Sun, K.; Liu, L.; Wang, G.; Liu, Y.: A reduced-reference quality assessment metric for super-resolution reconstructed images with information gain and texture similarity. Signal Process. Image Commun. 79, 32–39 (2019)

    Article  Google Scholar 

  47. Bevilacqua, M.; Roumy, A.; Guillemot, C.; Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)

  48. Martin, D.; Fowlkes, C.; Tal, D.; Malik J.; et al.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Iccv Vancouver (2001)

  49. Milanfar, P.; Farsiu, S., Robinson, D.: MDSP super-resolution and demosaicing datasets (2016)

  50. Moorthy, A.K.; Bovik, A.C.: A two-step framework for constructing blind image quality indices. IEEE Signal Process Lett. 17(5), 513–516 (2010)

    Article  Google Scholar 

  51. Mittal, A.; Soundararajan, R.; Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Institute of Technology Meghalaya, Shillong, India

    Rajashree Nayak

  2. Department of Computer Science Engineering, National Institute of Technology Meghalaya, Shillong, India

    Bunil Kumar Balabantaray

  3. Department of Electrical Engineering, National Institute of Technology Rourkela, Rourkela, India

    Dipti Patra

Authors
  1. Rajashree Nayak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bunil Kumar Balabantaray
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dipti Patra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajashree Nayak.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nayak, R., Balabantaray, B.K. & Patra, D. A New Single-Image Super-Resolution Using Efficient Feature Fusion and Patch Similarity in Non-Euclidean Space. Arab J Sci Eng 45, 10261–10285 (2020). https://doi.org/10.1007/s13369-020-04662-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-020-04662-9

Keywords

Search

Navigation