Skip to main content
SpringerLink
Log in

A guided optimized recursive least square adaptive filtering based multi-variate dense fusion network model for image interpolation

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Recently, learning-based image interpolation methods have gained significant popularity in the field of surveillance image processing due to their promising results. Notably, deep neural networks have shown considerable improvements in image super-resolution. To enhance the performance of image interpolation for surveillance images, researchers often employ deep convolutional neural Networks. However, merely increasing the network depth may not lead to substantial improvements and could even introduce new training-related challenges, necessitating novel training approaches. Therefore, in this proposed work, a new deep learning-based model is developed specifically tailored for effective surveillance image interpolation. The approach begins with the Optimized Recursive Least Square Adaptive Filter (ORLSAF) technique for image filtering. This step involves calculating the error signal and estimating weight factors to generate noise-free images. To reconstruct the interpolated surveillance images, the innovative Multi-Variate Dense Fusion Network (MVDFN) methodology is utilized, which incorporates feature fusion, augmentation, and loss regularization processes. Particularly, the loss factor is optimally calculated using the Hybrid Butterfly Optimization (HBO) algorithm. To evaluate the performance of the proposed technique, extensive experiments are conducted using a benchmarking dataset commonly employed in surveillance image processing. The evaluation metrics used include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), loss factor, and normalized similarity index. Overall, this research aims to advance surveillance image interpolation using a novel deep learning-based approach, combining ORLSAF, MVDFN, and the HBO algorithm to achieve superior results compared to existing methods. The potential impact of this work includes enhancing the quality and clarity of surveillance images, contributing to improved surveillance systems and applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availability

This Research, analysis uses the popular benchmarking datasets such as Cave, IAPR TC-12, DIV 2 K and CVDS. In order to validate the results, the parameters such as Peak Signal to Noise Ratio (PSNR), loss value, Structural Similarity Index Measure (SSMI), and Feature Similarity Index Measure (FSIM) have been considered.

References

  1. Mishra, D., Hadar, O.: Self-fusenet: data free unsupervised remote sensing image super-resolution. IEEE J. Select. Topics Appl. Earth Obs. Remote Sens. 16, 1710–1727 (2023)

    Article  ADS  Google Scholar 

  2. Seo, J., Kim, I., Seok, J.: Grid-wise simulation acceleration of the electromagnetic fields of 2D optical devices using super-resolution. Sci. Rep. 13, 435 (2023)

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  3. Ashiba, H.: Acquisition super resolution from infrared images using proposed techniques. Multimed. Tools Appl. 82, 2329–2348 (2023)

    Article  Google Scholar 

  4. Xiu M., Nie Y., Song Q., and Liu C., CoT-MISR: marrying convolution and transformer for multi-image super-resolution. arXiv preprint arXiv:2303.06548, (2023)

  5. Liu, F., Yang, X., De Baets, B.: A deep recursive multi-scale feature fusion network for image super-resolution. J. Vis. Commun. Image Repres. 90, 103730 (2023)

    Article  Google Scholar 

  6. HuangB., Yan J., Morris M., Sinnett V., Somaiah N., and Tang M.-X., acceleration-based kalman tracking for super-resolution ultrasound imaging in vivo, arXiv preprint: arXiv:2304.00819, (2023)

  7. Fu, L., Jiang, H., Wu, H., Yan, S., Wang, J., Wang, D.: Image super-resolution reconstruction based on instance spatial feature modulation and feedback mechanism. Appl. Intell. 53, 601–615 (2023)

    Article  Google Scholar 

  8. Wang, C., Lv, X., Shao, M., Qian, Y., Zhang, Y.: A novel fuzzy hierarchical fusion attention convolution neural network for medical image super-resolution reconstruction. Inf. Sci. 622, 424–436 (2023)

    Article  Google Scholar 

  9. Narayanan, N.V., Arjun, T., Logeshwari, R.: Surveillance image super resolution using SR-generative adversarial network. Adv. Sci. Technol. 124, 125–136 (2023)

    Article  Google Scholar 

  10. Li, H.-A., Wang, D., Zhang, J., Li, Z., Ma, T.: Image super-resolution reconstruction based on multi-scale dual-attention. Connect. Sci. 2, 1–19 (2023)

    Google Scholar 

  11. Kezzoula Z., Gaceb D., Akli Z., Kahouli A., Titoun A., and Touazi F., Bi-ESRGAN: A New Approach of Document Image Super-Resolution Based on Dual Deep Transfer Learning. In: Artificial Intelligence: Theories and Applications: First International Conference, ICAITA 2022, Mascara, Algeria, November 7–8, 2022, Revised Selected Papers, 2023, pp. 110-122

  12. Nguyen Q. H., and Beksi W. J., Single Image Super-Resolution via a Dual Interactive Implicit Neural Network. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 4936–4945.

  13. Wang Y. and Du H., Image Interpolation Algorithm Based on Texture Complexity and Gradient Optimization. In: Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery: Proceedings of the ICNC-FSKD 2022, ed.: Springer, 2023, pp. 682–692.

  14. Bouffard, A., Pop, M., Ebrahimi, M.: Multi-step reinforcement learning for medical image super-resolutionin Medical Imaging. Image Process. 2023, 444–450 (2023)

    Google Scholar 

  15. Xu, Y., Dai, S., Song, H., Du, L., Chen, Y.: Multi-modal brain MRI images enhancement based on framelet and local weights super-resolution. Math. Biosci. Eng. 20, 4258–4273 (2023)

    Article  MathSciNet  PubMed  Google Scholar 

  16. Yao, D., Liang, H., Campos, J., Yan, L., Yan, C., Jiang, C., et al.: Calculation and restoration of lost spatial information in division-of-focal-plane polarization remote sensing using polarization super-resolution technology. Int. J. Appl. Earth Obs. Geoinf. 116, 103155 (2023)

    Google Scholar 

  17. Ismail, I., Eltoukhy, M.M., Eltaweel, G.: Super-resolution based on curvelet transform and sparse representation. Comput. Syst. Sci. Eng. 45, 167–181 (2023)

    Article  Google Scholar 

  18. Shen, Y., Zheng, W., Chen, L., Huang, F.: RSHAN: image super-resolution network based on residual separation hybrid attention module. Eng. Appl. Artif. Intell. 122, 106072 (2023)

    Article  Google Scholar 

  19. Ge, R., Shi, F., Chen, Y., Tang, S., Zhang, H., Lou, X., et al.: Improving anisotropy resolution of computed tomography and annotation using 3D super-resolution network. Biomed. Signal Process. Control 82, 104590 (2023)

    Article  Google Scholar 

  20. Hu, H., Yang, S., Li, X., Cheng, Z., Liu, T., Zhai, J.: Polarized image super-resolution via a deep convolutional neural network. Opt. Express 31, 8535–8547 (2023)

    Article  ADS  PubMed  Google Scholar 

  21. Li, K., Yang, S., Dong, R., Wang, X., Huang, J.: Survey of single image super-resolution reconstruction. IET Image Proc. 14, 2273–2290 (2020)

    Article  Google Scholar 

  22. Singh, A., Singh, J.: Content adaptive single image interpolation based super resolution of compressed images. Int. J. Electr. Comput. Eng. (IJECE) 10, 3014–3021 (2020)

    Article  Google Scholar 

  23. Hung, K.-W., Wang, K., Jiang, J.: Image interpolation using convolutional neural networks with deep recursive residual learning. Multimed. Tools Appl. 78, 22813–22831 (2019)

    Article  Google Scholar 

  24. Chen, L., Liu, H., Yang, M., Qian, Y., Xiao, Z., Zhong, X.: Remote sensing image super-resolution via residual aggregation and split attentional fusion network. IEEE J. Selected Topics Appl. Earth Observat. Remote Sens. 14, 9546–9556 (2021)

    Article  ADS  Google Scholar 

  25. Qiu, D., Zheng, L., Zhu, J., Huang, D.: Multiple improved residual networks for medical image super-resolution. Futur. Gener. Comput. Syst. 116, 200–208 (2021)

    Article  Google Scholar 

  26. Zhang, Y., Wang, P., Bao, F., Yao, X., Zhang, C., Lin, H.: A single-image super-resolution method based on progressive-iterative approximation. IEEE Trans. Multimedia 22, 1407–1422 (2019)

    Article  Google Scholar 

  27. Z. Hui, X. Gao, Y. Yang, and X. Wang, "Lightweight image super-resolution with information multi-distillation network," in Proceedings of the 27th acm international conference on multimedia, 2019, pp. 2024–2032.

  28. Aydin, O., Cinbiş, R.G.: Single-image super-resolution analysis in DCT spectral domain. Balkan J. Elect. Comput. Eng. 8, 209–217 (2020)

    Article  Google Scholar 

  29. Zheng, J., Song, W., Wu, Y., Liu, F.: Weighted direct nonlinear regression for effective image interpolation. IEEE Access 7, 8646–8659 (2019)

    Article  Google Scholar 

  30. Ahmad, W., Ali, H., Shah, Z., Azmat, S.: A new generative adversarial network for medical images super resolution. Sci. Rep. 12, 9533 (2022)

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  31. Reid, E.J., Drummy, L.F., Bouman, C.A., Buzzard, G.T.: Multi-resolution data fusion for super resolution imaging. IEEE Trans. Computat. Imaging 8, 81–95 (2022)

    Article  MathSciNet  Google Scholar 

  32. Khan, S., Lee, D.-H., Khan, M.A., Gilal, A.R., Mujtaba, G.: Efficient edge-based image interpolation method using neighboring slope information. IEEE Access 7, 133539–133548 (2019)

    Article  Google Scholar 

  33. Zhang, Y., Li, K., Li, K., Zhong, B., & Fu, Y. (2019). Residual non-local attention networks for image restoration. arXiv preprint arXiv:1903.10082.

  34. Liu, Z.Y., Liu, J.W., Zuo, X., Hu, M.F.: Multi-scale iterative refinement network for RGB-D salient object detection. Eng. Appl. Artif. Intell. 106, 104473 (2021)

    Article  Google Scholar 

  35. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image Super-Resolution using Very Deep Residual Channel Attention Networks. In Proceedings of the European conference on computer vision (ECCV) (pp. 286–301)

  36. Zhao, M., Liu, X., Liu, H., Wong, K.K.: Super-resolution of cardiac magnetic resonance images using Laplacian pyramid based on generative adversarial networks. Comput. Med. Imaging Graph. 80, 101698 (2020)

    Article  PubMed  Google Scholar 

  37. Zhang, Y., Chen, H., Ren, C., Zhu, C.: Depth map super-resolution via shape-adaptive non-local regression and direction-based local smoothness. Electron. Lett. 57(12), 475–477 (2021)

    Article  ADS  Google Scholar 

  38. Hu, H.-T., Hsu, L.-Y., Wu, S.-T.: Blind watermarking for hiding color images in color images with super-resolution enhancement. Sensors 23, 370 (2023)

    Article  ADS  Google Scholar 

  39. Dong, W., Zhou, C., Wu, F., Wu, J., Shi, G., Li, X.: Model-guided deep hyperspectral image super-resolution. IEEE Trans. Image Process. 30, 5754–5768 (2021)

    Article  ADS  PubMed  Google Scholar 

  40. Zhang, S., Fu, G., Wang, H., Zhao, Y.: Degradation learning for unsupervised hyperspectral image super-resolution based on generative adversarial network. SIViP 15, 1695–1703 (2021)

    Article  Google Scholar 

  41. Catalbas, M.C.: Modified VDSR-based single image super-resolution using naturalness image quality evaluator. SIViP 16, 661–668 (2022)

    Article  Google Scholar 

  42. Weng, Y., Chen, Z., Zhou, T.: Improved differentiable neural architecture search for single image super-resolution. Peer-to-Peer Netw. Appl. 14, 1806–1815 (2021)

    Article  Google Scholar 

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Sathyabama Institute of Science and Technology, Chennai, Tamil Nadu, 600119, India

    V. Diana Earshia & M. Sumathi

  2. Department of Electronics and Communication Engineering, Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology Chennai, Chennai, Tamil Nadu, 600062, India

    V. Diana Earshia

Authors
  1. V. Diana Earshia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Sumathi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors confirm their contribution to the article as follows: VDE, MS have conceived the idea; VDE has devel oped the theory and conducted research and analyzed the data; MS has supervised the findings; VDE,MS have drafted and finalized the article.

Corresponding author

Correspondence to V. Diana Earshia.

Ethics declarations

Conflict of interest

The authors have No relevant financial or Non-financial interests to disclose.

Ethical approval

The study submitted to Signal, Image and Video Processing have been conducted in accordance with the Declaration of Helsinki and according to requirements of all applicable local and international standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Diana Earshia, V., Sumathi, M. A guided optimized recursive least square adaptive filtering based multi-variate dense fusion network model for image interpolation. SIViP 18, 991–1005 (2024). https://doi.org/10.1007/s11760-023-02805-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02805-7

Keywords

Search

Navigation