Skip to main content
SpringerLink
Log in

Multi-frame image super-resolution reconstruction based on spatial information weighted fields of experts

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

To overcome the limitations of the traditional fields of experts (FoE) model, which will blur image edges and texture during the denoising processing, a spatial information weighted FoE (WFoE) model has been presented to introduce the image spatial structure information into the FoE model. A monotone decreasing function is based on the curvature difference to control the filter weight in the edge and smooth region. The proposed WFoE model can better remove noise while preserving edges. Additionally, the proposed WFoE model is designed as a regularization term in the maximum a posteriori-based multi-frame image super-resolution (SR) reconstruction algorithm, enabling the development of a new SR method. Since the WFoE model is more inclined to keep image edges, the proposed WFoE-based SR reconstruction method can obtain better results than traditional FoE model with respect to preserving image edges. Experimental results demonstrate that our method has better peak signal-to-noise ratio and visual verisimilitude compared with some existing SR methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. P. Milanfar. MDSP Super-Resolution and Demosaicing Datasets Online: http://users.soe.ucsc.edu/∼milanfar/software/srdatasets.html.

  2. P. Vandewalle and S. Süsstrunk Super-Resolution Data Sets Online: http://users.soe.ucsc.edu/∼milanfar/software/srdatasets.html.

References

  • Bareja, M. N., & Modi, C. K. (2012). An effective iterative back projection based single image super resolution approach. IEEE Computer Society,14(4), 95–99.

    Google Scholar 

  • Chen, P., Nelson, J., & Tourneret, J. Y. (2017). Toward a sparse Bayesian Markov random field approach to hyperspectral unmixing and classification. IEEE Transactions on Image Processing,26(1), 426–438.

    Article  MathSciNet  MATH  Google Scholar 

  • Chen, J., Nunez-Yanez, J., & Achim, A. (2011). Video super-resolution using generalized Gaussian Markov random fields. IEEE Signal Processing Letters,19(2), 63–66.

    Article  Google Scholar 

  • Cheng, P., Qiu, Y., Wang, X., & Zhao, K. (2017). A new single image super-resolution method based on the infinite mixture model. IEEE Access,5(9), 2228–2240.

    Article  Google Scholar 

  • Demirel, H., Izadpanahi, S., & Anbarjafari, G. (2009). Improved motion-based localized super resolution technique using discrete wavelet transform for low resolution video enhancement. In Proceedings of European signal processing conference (pp. 1097–1101).

  • Deng, L. J., Guo, W., & Huang, T. Z. (2016). Single image super-resolution by approximated Heaviside functions. Information Sciences,348, 107–123.

    Article  MathSciNet  MATH  Google Scholar 

  • Elad, M., & Feuer, A. (1997). Restoration of a single super-resolution image from several blurred, noisy, and under-sampled measured images. IEEE Transactions on Image Processing,6(12), 1646–1658.

    Article  Google Scholar 

  • Farsiu, S., Robinson, M. D., Elad, M., & Milanfar, P. (2004). Fast and robust multi-frame super resolution. IEEE Transactions on Image Processing,13(10), 1327–1344.

    Article  Google Scholar 

  • Geman, S., & Geman, D. (1984). Stochastic relaxation Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence,6, 721–741.

    Article  MATH  Google Scholar 

  • Huang, S., Sun, J., Yang, Y., Fang, Y., & Lin, P. (2017). Multi-frame super-resolution reconstruction based on gradient vector flow hybrid field. IEEE Access,5, 21669–21683.

    Article  Google Scholar 

  • Huang, S., & Yang, Y. (2013). Super-resolution reconstruction sensor using adaptively combined partial differential equations. Sensor Letters,11(11), 2126–2130.

    Article  Google Scholar 

  • Kanemura, A., Maeda, S. I., & Ishii, S. (2009). Super-resolution with compound Markov random fields via the variational EM algorithm. IEEE Transactions on Neural Networks,22(7), 1025–1034.

    Article  MATH  Google Scholar 

  • Li, F., Xin, L., Guo, Y., Gao, J., & Jia, X. (2017). A framework of mixed sparse representations for remote sensing images. IEEE Geoscience and Remote Sensing Letters,55(2), 1210–1221.

    Article  Google Scholar 

  • Lukeš, T., Křížek, P., Švindrych, Z., Benda, J., Ovesný, M., Fliegel, K., et al. (2014). Three-dimensional super-resolution structured illumination microscopy with maximum a posteriori probability image estimation. Optics Express,22(24), 29805–29817.

    Article  Google Scholar 

  • Marquina, A., & Osher, S. J. (2008). Image super-resolution by TV-regularization and bregman iteration. Journal of Scientific Computing,37(3), 367–382.

    Article  MathSciNet  MATH  Google Scholar 

  • Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2002). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of IEEE international conference on computer and vision (Vol. 2(11), pp. 416–423).

  • Pan, R., & Reeves, S. J. (2006). Efficient Huber–Markov edge-preserving image restoration. IEEE Transactions on Image Processing,15(12), 3728–3735.

    Article  MathSciNet  Google Scholar 

  • Portilla, J., Strela, V., Wainwright, M. J., & Simoncelli, E. P. (2001). Adaptive Wiener denoising using a Gaussian scale mixture model in the wavelet domain. International Conference on Image Processing,2(2), 37–40.

    Google Scholar 

  • Ren, C., He, X., & Nguyen, T. (2017). Single image super-resolution via adaptive high-dimensional non-local total variation and adaptive geometric feature. IEEE Transactions on Image Processing,26(1), 90–106.

    MathSciNet  MATH  Google Scholar 

  • Rhee, S., & Kang, M. G. (1999). Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Optical Engineering,38(8), 1348–1356.

    Article  Google Scholar 

  • Robinson, M. D., Toth, C. A., Lo, J. Y., & Farsiu, S. (2010). Efficient fourier-wavelet super-resolution. IEEE Transactions on Image Processing,19(10), 2669–2681.

    Article  MathSciNet  MATH  Google Scholar 

  • Roth, S., & Black, M. J. (2005). Fields of experts: A frame work for learning image priors. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2(2), pp. 860–867).

  • Roth, S., & Black, M. J. (2007). High-order markov random fields for low-level vision. Providence: Brown University.

    Google Scholar 

  • Sharmin, N., & Brad, R. (2012). Optimal filter estimation for Lucas–Kanade optical flow. Sensors,12(9), 12694–12709.

    Article  Google Scholar 

  • Soccorsi, M., Gleich, D., & Datcu, M. (2010). Huber–Markov model for complex SAR image restoration. IEEE Geoscience and Remote Sensing Letters,7(1), 63–67.

    Article  Google Scholar 

  • Stark, H., & Oskoui, P. (1989). High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America. A, Optics, Image Science,6(11), 1715.

    Article  Google Scholar 

  • Tom, B. C., Galatsanos, N. P., & Katsaggelos, A. K. (1994). Reconstruction of a high resolution image from multiple low resolution images. Visual Communications and Image Processing’,2308, 971–981.

    Article  Google Scholar 

  • Tsai, R. Y., & Huang, T. S. (1984). Multi-frame image restoration and registration. Advance Computer Visual and Image Processing,1(2), 317–339.

    Google Scholar 

  • Wang, Q., & Shi, W. (2014). Utilizing multiple sub-pixel shifted images in sub-pixel mapping with image interpolation. IEEE Geoscience and Remote Sensing Letters,11(4), 798–802.

    Article  Google Scholar 

  • Wang, W., & Yuan, X. (2017). Recent advances in image dehazing. IEEE/CAA Journal of Automatica, Sinica,4(3), 410–436.

    Article  MathSciNet  Google Scholar 

  • Xiao, J., Pang, G., Zhang, Y., Kuang, Y., Yan, Y., & Wang, Y. (2016). Adaptive shock filter for image super-resolution and enhancement. Journal of Visual Communication and Image Representation,40, 168–177.

    Article  Google Scholar 

  • Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., & Zhang, L. (2016). Image super-resolution: The techniques, applications, and future. Signal Processing,128, 389–408.

    Article  Google Scholar 

  • Zhang, X., Lam, E. Y., Wu, E. X., & Wong, K. K. Y. (2008). Application of tikhonov regularization to super-resolution reconstruction of brain MRI images. Lecture Notes in Computer Science,4987(23), 51–56.

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Nos. 61862030, 61662026 and 61462031), by the Natural Science Foundation of Jiangxi Province (Nos. 20182BCB22006, 20181BAB202010), and by the Project of the Education Department of Jiangxi Province (Nos. GJJ170318, GJJ170312 and KJLD14031).

Author information

Authors and Affiliations

  1. School of Software and Communication Engineering, Jiangxi University of Finance and Economics, Nanchang, 330032, China

    Shuying Huang & Jiajun Wu

  2. School of Information Technology, Jiangxi University of Finance and Economics, Nanchang, 330032, China

    Yong Yang

  3. College of Biomedical Engineering, South-Central University for Nationalities, Wuhan, 430074, China

    Pan Lin

Authors
  1. Shuying Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiajun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Yang.

Ethics declarations

Conflict of interest

The authors declare no conflict of interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, S., Wu, J., Yang, Y. et al. Multi-frame image super-resolution reconstruction based on spatial information weighted fields of experts. Multidim Syst Sign Process 31, 1–20 (2020). https://doi.org/10.1007/s11045-019-00648-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-019-00648-5

Keywords

Search

Navigation