Skip to main content
SpringerLink
Log in

Particle filter based multi-frame image super resolution

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In most multi-frame image super-resolution (SR) studies, the process that models the construction of the low-resolution (LR) images from high resolution (HR) one includes geometric transformation, blurring, downsampling, and adding noise. A simple linear approximation of this model is used in most super-resolution frameworks. The approximation error in this popular model appears as an additive noise that is generally a non-Gaussian noise. Most state-of-the-art estimators, such as the Kalman filter as a linear estimator used in super-resolution problems, suffer from this non-Gaussian noise caused by approximation error. We present a novel multi-frame super-resolution using the particle filter method that performs well in a non-Gaussian nonlinear dynamic system to produce the high-resolution output. One key innovation is that the approximation error in the super-resolution model can be handled well by using a particle filter. In this study, the optimal importance function is used to improve particle filter efficiency determined with the Taylor expansion of observation and measurement function. Experiments with simulated and real-image sequences yield that our proposed super-resolution method has good results in suppressing noise and reconstructing details in high-resolution images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Harikrishnan, P.M., Thomas, A., Gopi, V.P., Palanisamy, P.: Fast approach for moving vehicle localization and zbounding box estimation in highway traffic videos. SIViP 15(5), 1041–1048 (2021)

    Article  Google Scholar 

  2. Kanakaraj, S., Nair, M.S., Kalady, S.: SAR image super resolution using importance sampling unscented Kalman filter. IEEE J. Sel. Top Appl. Earth Obs. Remote Sens. 11(2), 562–571 (2017)

    Article  Google Scholar 

  3. Shao, Z., Wang, L., Wang, Z., Deng, J.: Remote sensing image super-resolution using sparse representation and coupled sparse autoencoder. IEEE J. Sel. Top Appl. Earth Obs. Remote Sens. 12(8), 2663–2674 (2019)

    Article  Google Scholar 

  4. Dimililer, K.: DCT-based medical image compression using machine learning. SIViP 16(1), 55–62 (2022)

    Article  Google Scholar 

  5. Ashok Kumar, P.M., Maddala, J.B., Sagayam, K.M.: Enhanced facial emotion recognition by optimal descriptor selection with neural network. IETE J. Res. (2021). https://doi.org/10.1080/03772063.2021.1902868

    Article  Google Scholar 

  6. Huang, T.S., Tsai, R.Y.: Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process. 1, 317–339 (1984)

    Google Scholar 

  7. Irani, M., Peleg, S.: Improving resolution by image registration. Graph. Model Image Process. 53, 231–239 (1991)

    Article  Google Scholar 

  8. Elad, M., Feuer, A.: Super-resolution reconstruction of continuous image sequences. In: Proceedings 1999 International Conference on Image Processing, 3, 459–463 (1999).

  9. Cheeseman, P., Kanefsky, B., Kraft, R., Stutz, J., Hanson, R.: Superresolved surface reconstruction from multiple images. In: Maximum Entropy and Bayesian Methods, pp. 293–308. Springer, Cham (1996)

    Chapter  Google Scholar 

  10. Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multi-frame super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)

    Article  Google Scholar 

  11. Zeng, X., Yang, L.: A robust multiframe super-resolution algorithm based on half-quadratic estimation with modified BTV regularization. Digit. Signal Process. 23, 98–109 (2013)

    Article  MathSciNet  Google Scholar 

  12. Liu, X., Chen, L., Wang, W., Zhao, J.: Robust multi-frame super-resolution based on spatially weighted half-quadratic estimation and adaptive BTV regularization. IEEE Trans. Image Process. 27(10), 4971–4986 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. Rahimi, A., Moallem, P., Shahtalebi, K., Momeni, M.: Using Kalman filter in the frequency domain for multi-frame scalable super resolution. Signal Process. 155, 108–129 (2019)

    Article  Google Scholar 

  14. Wei, Z., Tao, F., Jun, W.: Kalman filter-based method for image superresolution using a sequence of low-resolution images. J. Electron. Imaging 23(1), 1–12 (2014)

    Article  Google Scholar 

  15. Farsiu, S., Elad, M., Milanfar, P.: Video-to-video dynamic superresolution for grayscale and color sequences. EURASIP J. Adv. Signal Process. (2006). https://doi.org/10.1155/ASP/2006/61859

    Article  Google Scholar 

  16. Liu, C., Sun, D.: On Bayesian adaptive video super resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 346–360 (2013)

    Article  Google Scholar 

  17. Zhang, T., Gao, K., Ni, G., Fan, G., Lu, Y.: Spatio-temporal super-resolution for multi-videos based on belief propagation. Signal Process. Image Commun. 68, 1–12 (2018)

    Article  Google Scholar 

  18. Baker, S., Kanade, T.: Hallucinating faces. In: IEEE Conference on Automatic Face and Gesture Recognition, pp. 83–88 (2000).

  19. Sun, Y., Qin, J., Gao, X., Chai, S., Chen, B.: Attention-enhanced multi-scale residual network for single image super-resolution. SIViP (2022). https://doi.org/10.1007/s11760-021-02095-x

    Article  Google Scholar 

  20. Hu, Y., Gao, X., Li, J., Huang, Y., Wang, H.: Single image super-resolution with multi-scale information cross-fusion network. Signal Process. 179, 107831 (2021)

    Article  Google Scholar 

  21. Wang, Y., Wang, L., Wang, H., Li, P.: End-to-end image super-resolution via deep and shallow convolutional networks. IEEE Access 7, 31959–31970 (2019)

    Article  Google Scholar 

  22. Li, Z., Kuang, Z.S., Zhu, Z.L., Wang, H.P., Shao, X.L.: Wavelet-based texture reformation network for image super-resolution. IEEE Trans. Image Process. 31, 2647–2660 (2022)

    Article  Google Scholar 

  23. Wang, Z., Chen, J., Hoi, S.C.: Deep learning for image super-resolution: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3365–3387 (2020)

    Article  Google Scholar 

  24. Zhang, X., Liu, D., Lei, B., Liang, J., Ji, R.: An intelligent particle filter with resampling of multi-population cooperation. Digit. Signal Process. 115, 103084 (2021)

    Article  Google Scholar 

  25. Wang, F., Zhang, J., Lin, B., Li, X.: Two stage particle filter for nonlinear Bayesian estimation. IEEE Access 6, 13803–13809 (2018)

    Article  Google Scholar 

  26. Doucet, A., Godsill, S., Andrieu, C.: On sequential monte carlo sampling methods for baysian filtering. Stat. Comput. 10(3), 197–208 (2000)

    Article  Google Scholar 

  27. Kwon, J., Lee, K.M., Park, F.C.: Visual tracking via geometric particle filtering on the affine group with optimal importance functions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2009).

  28. Lv, X.: A Kalman filter with noise covariances estimation and its stability analysis. Int. J. Control Autom 10(10), 63–76 (2017)

    Article  Google Scholar 

  29. University of california, santa cruz, milanfar software. [Online] Available: https://users.soe.ucsc.edu/~milanfar/software/sr-datasets.html.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Faculty of Engineering, University of Isfahan, Isfahan, Iran

    Negin Ghasemi-Falavarjani & Payman Moallem

  2. Department of Electro-Ceramics and Electrical Engineering, Malek Ashtar University, Isfahan, Iran

    Akbar Rahimi

Authors
  1. Negin Ghasemi-Falavarjani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Payman Moallem
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Akbar Rahimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Payman Moallem.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 2414 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghasemi-Falavarjani, N., Moallem, P. & Rahimi, A. Particle filter based multi-frame image super resolution. SIViP 17, 3247–3254 (2023). https://doi.org/10.1007/s11760-022-02406-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02406-w

Keywords

Search

Navigation