Abstract
Recently, learning-based image interpolation methods have gained significant popularity in the field of surveillance image processing due to their promising results. Notably, deep neural networks have shown considerable improvements in image super-resolution. To enhance the performance of image interpolation for surveillance images, researchers often employ deep convolutional neural Networks. However, merely increasing the network depth may not lead to substantial improvements and could even introduce new training-related challenges, necessitating novel training approaches. Therefore, in this proposed work, a new deep learning-based model is developed specifically tailored for effective surveillance image interpolation. The approach begins with the Optimized Recursive Least Square Adaptive Filter (ORLSAF) technique for image filtering. This step involves calculating the error signal and estimating weight factors to generate noise-free images. To reconstruct the interpolated surveillance images, the innovative Multi-Variate Dense Fusion Network (MVDFN) methodology is utilized, which incorporates feature fusion, augmentation, and loss regularization processes. Particularly, the loss factor is optimally calculated using the Hybrid Butterfly Optimization (HBO) algorithm. To evaluate the performance of the proposed technique, extensive experiments are conducted using a benchmarking dataset commonly employed in surveillance image processing. The evaluation metrics used include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), loss factor, and normalized similarity index. Overall, this research aims to advance surveillance image interpolation using a novel deep learning-based approach, combining ORLSAF, MVDFN, and the HBO algorithm to achieve superior results compared to existing methods. The potential impact of this work includes enhancing the quality and clarity of surveillance images, contributing to improved surveillance systems and applications.
Similar content being viewed by others
Data availability
This Research, analysis uses the popular benchmarking datasets such as Cave, IAPR TC-12, DIV 2 K and CVDS. In order to validate the results, the parameters such as Peak Signal to Noise Ratio (PSNR), loss value, Structural Similarity Index Measure (SSMI), and Feature Similarity Index Measure (FSIM) have been considered.
References
Mishra, D., Hadar, O.: Self-fusenet: data free unsupervised remote sensing image super-resolution. IEEE J. Select. Topics Appl. Earth Obs. Remote Sens. 16, 1710–1727 (2023)
Seo, J., Kim, I., Seok, J.: Grid-wise simulation acceleration of the electromagnetic fields of 2D optical devices using super-resolution. Sci. Rep. 13, 435 (2023)
Ashiba, H.: Acquisition super resolution from infrared images using proposed techniques. Multimed. Tools Appl. 82, 2329–2348 (2023)
Xiu M., Nie Y., Song Q., and Liu C., CoT-MISR: marrying convolution and transformer for multi-image super-resolution. arXiv preprint arXiv:2303.06548, (2023)
Liu, F., Yang, X., De Baets, B.: A deep recursive multi-scale feature fusion network for image super-resolution. J. Vis. Commun. Image Repres. 90, 103730 (2023)
HuangB., Yan J., Morris M., Sinnett V., Somaiah N., and Tang M.-X., acceleration-based kalman tracking for super-resolution ultrasound imaging in vivo, arXiv preprint: arXiv:2304.00819, (2023)
Fu, L., Jiang, H., Wu, H., Yan, S., Wang, J., Wang, D.: Image super-resolution reconstruction based on instance spatial feature modulation and feedback mechanism. Appl. Intell. 53, 601–615 (2023)
Wang, C., Lv, X., Shao, M., Qian, Y., Zhang, Y.: A novel fuzzy hierarchical fusion attention convolution neural network for medical image super-resolution reconstruction. Inf. Sci. 622, 424–436 (2023)
Narayanan, N.V., Arjun, T., Logeshwari, R.: Surveillance image super resolution using SR-generative adversarial network. Adv. Sci. Technol. 124, 125–136 (2023)
Li, H.-A., Wang, D., Zhang, J., Li, Z., Ma, T.: Image super-resolution reconstruction based on multi-scale dual-attention. Connect. Sci. 2, 1–19 (2023)
Kezzoula Z., Gaceb D., Akli Z., Kahouli A., Titoun A., and Touazi F., Bi-ESRGAN: A New Approach of Document Image Super-Resolution Based on Dual Deep Transfer Learning. In: Artificial Intelligence: Theories and Applications: First International Conference, ICAITA 2022, Mascara, Algeria, November 7–8, 2022, Revised Selected Papers, 2023, pp. 110-122
Nguyen Q. H., and Beksi W. J., Single Image Super-Resolution via a Dual Interactive Implicit Neural Network. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 4936–4945.
Wang Y. and Du H., Image Interpolation Algorithm Based on Texture Complexity and Gradient Optimization. In: Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery: Proceedings of the ICNC-FSKD 2022, ed.: Springer, 2023, pp. 682–692.
Bouffard, A., Pop, M., Ebrahimi, M.: Multi-step reinforcement learning for medical image super-resolutionin Medical Imaging. Image Process. 2023, 444–450 (2023)
Xu, Y., Dai, S., Song, H., Du, L., Chen, Y.: Multi-modal brain MRI images enhancement based on framelet and local weights super-resolution. Math. Biosci. Eng. 20, 4258–4273 (2023)
Yao, D., Liang, H., Campos, J., Yan, L., Yan, C., Jiang, C., et al.: Calculation and restoration of lost spatial information in division-of-focal-plane polarization remote sensing using polarization super-resolution technology. Int. J. Appl. Earth Obs. Geoinf. 116, 103155 (2023)
Ismail, I., Eltoukhy, M.M., Eltaweel, G.: Super-resolution based on curvelet transform and sparse representation. Comput. Syst. Sci. Eng. 45, 167–181 (2023)
Shen, Y., Zheng, W., Chen, L., Huang, F.: RSHAN: image super-resolution network based on residual separation hybrid attention module. Eng. Appl. Artif. Intell. 122, 106072 (2023)
Ge, R., Shi, F., Chen, Y., Tang, S., Zhang, H., Lou, X., et al.: Improving anisotropy resolution of computed tomography and annotation using 3D super-resolution network. Biomed. Signal Process. Control 82, 104590 (2023)
Hu, H., Yang, S., Li, X., Cheng, Z., Liu, T., Zhai, J.: Polarized image super-resolution via a deep convolutional neural network. Opt. Express 31, 8535–8547 (2023)
Li, K., Yang, S., Dong, R., Wang, X., Huang, J.: Survey of single image super-resolution reconstruction. IET Image Proc. 14, 2273–2290 (2020)
Singh, A., Singh, J.: Content adaptive single image interpolation based super resolution of compressed images. Int. J. Electr. Comput. Eng. (IJECE) 10, 3014–3021 (2020)
Hung, K.-W., Wang, K., Jiang, J.: Image interpolation using convolutional neural networks with deep recursive residual learning. Multimed. Tools Appl. 78, 22813–22831 (2019)
Chen, L., Liu, H., Yang, M., Qian, Y., Xiao, Z., Zhong, X.: Remote sensing image super-resolution via residual aggregation and split attentional fusion network. IEEE J. Selected Topics Appl. Earth Observat. Remote Sens. 14, 9546–9556 (2021)
Qiu, D., Zheng, L., Zhu, J., Huang, D.: Multiple improved residual networks for medical image super-resolution. Futur. Gener. Comput. Syst. 116, 200–208 (2021)
Zhang, Y., Wang, P., Bao, F., Yao, X., Zhang, C., Lin, H.: A single-image super-resolution method based on progressive-iterative approximation. IEEE Trans. Multimedia 22, 1407–1422 (2019)
Z. Hui, X. Gao, Y. Yang, and X. Wang, "Lightweight image super-resolution with information multi-distillation network," in Proceedings of the 27th acm international conference on multimedia, 2019, pp. 2024–2032.
Aydin, O., Cinbiş, R.G.: Single-image super-resolution analysis in DCT spectral domain. Balkan J. Elect. Comput. Eng. 8, 209–217 (2020)
Zheng, J., Song, W., Wu, Y., Liu, F.: Weighted direct nonlinear regression for effective image interpolation. IEEE Access 7, 8646–8659 (2019)
Ahmad, W., Ali, H., Shah, Z., Azmat, S.: A new generative adversarial network for medical images super resolution. Sci. Rep. 12, 9533 (2022)
Reid, E.J., Drummy, L.F., Bouman, C.A., Buzzard, G.T.: Multi-resolution data fusion for super resolution imaging. IEEE Trans. Computat. Imaging 8, 81–95 (2022)
Khan, S., Lee, D.-H., Khan, M.A., Gilal, A.R., Mujtaba, G.: Efficient edge-based image interpolation method using neighboring slope information. IEEE Access 7, 133539–133548 (2019)
Zhang, Y., Li, K., Li, K., Zhong, B., & Fu, Y. (2019). Residual non-local attention networks for image restoration. arXiv preprint arXiv:1903.10082.
Liu, Z.Y., Liu, J.W., Zuo, X., Hu, M.F.: Multi-scale iterative refinement network for RGB-D salient object detection. Eng. Appl. Artif. Intell. 106, 104473 (2021)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image Super-Resolution using Very Deep Residual Channel Attention Networks. In Proceedings of the European conference on computer vision (ECCV) (pp. 286–301)
Zhao, M., Liu, X., Liu, H., Wong, K.K.: Super-resolution of cardiac magnetic resonance images using Laplacian pyramid based on generative adversarial networks. Comput. Med. Imaging Graph. 80, 101698 (2020)
Zhang, Y., Chen, H., Ren, C., Zhu, C.: Depth map super-resolution via shape-adaptive non-local regression and direction-based local smoothness. Electron. Lett. 57(12), 475–477 (2021)
Hu, H.-T., Hsu, L.-Y., Wu, S.-T.: Blind watermarking for hiding color images in color images with super-resolution enhancement. Sensors 23, 370 (2023)
Dong, W., Zhou, C., Wu, F., Wu, J., Shi, G., Li, X.: Model-guided deep hyperspectral image super-resolution. IEEE Trans. Image Process. 30, 5754–5768 (2021)
Zhang, S., Fu, G., Wang, H., Zhao, Y.: Degradation learning for unsupervised hyperspectral image super-resolution based on generative adversarial network. SIViP 15, 1695–1703 (2021)
Catalbas, M.C.: Modified VDSR-based single image super-resolution using naturalness image quality evaluator. SIViP 16, 661–668 (2022)
Weng, Y., Chen, Z., Zhou, T.: Improved differentiable neural architecture search for single image super-resolution. Peer-to-Peer Netw. Appl. 14, 1806–1815 (2021)
Funding
No funding was received to assist with the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
The authors confirm their contribution to the article as follows: VDE, MS have conceived the idea; VDE has devel oped the theory and conducted research and analyzed the data; MS has supervised the findings; VDE,MS have drafted and finalized the article.
Corresponding author
Ethics declarations
Conflict of interest
The authors have No relevant financial or Non-financial interests to disclose.
Ethical approval
The study submitted to Signal, Image and Video Processing have been conducted in accordance with the Declaration of Helsinki and according to requirements of all applicable local and international standards.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Diana Earshia, V., Sumathi, M. A guided optimized recursive least square adaptive filtering based multi-variate dense fusion network model for image interpolation. SIViP 18, 991–1005 (2024). https://doi.org/10.1007/s11760-023-02805-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-023-02805-7