Skip to main content
SpringerLink
Log in

Super-resolving blurry multiframe images through multiframe blind deblurring using ADMM

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multiframe super resolution (MFSR) aims to reconstruct a high resolution (HR) image from a set of low resolution (LR) images. However, the MFSR is an ill-posed problem and typically computational costly. In this paper, we propose to super-resolve multiple degraded LR frames of the original scene through multiframe blind deblurring (MFDB). First, we propose a new MFSR forward model and reformulate the MFSR problem into a MFDB problem which is easier to be solved than the former. We further solve the MFBD problem in which, the optimization problems with respect to the unknown image and with respect to the unknown blur are efficiently addressed by the alternating direction method of multipliers (ADMM). Our approach bridges the gap between MFSR and MFBD, taking advantages of existing MFBD methods to handle MFSR. Experiments on synthetic and real images show that the proposed method is competitive and effective in terms of speed and restoration quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Afonso MV, Bioucas-Dias JM, Figueiredo MAT (2010) Fast Image Recovery Using Variable Splitting and Constrained Optimization. IEEE Transactions on Image Process 19(9):2345–2356

    Article  MathSciNet  Google Scholar 

  2. Baker S, Scharstein D, Lewis J, Roth S, Black MJ, Szeliski R (2007) A database and evaluation methodology for optical flow, In ICCV

  3. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning 3:1–122

    Article  MATH  Google Scholar 

  4. Capel. D (2004) Image Mosaicing and Super-resolution. Springer, London

    Book  MATH  Google Scholar 

  5. Carles G, Downing J, Harvey AR (2014) Super-resolution imaging using a camera array. Opt Lett 39(7):1889–1892

  6. Chaudhuri S (2001) Super-Resolution Imaging. Kluwer Academic Publishers, Boston

    Google Scholar 

  7. Chiang MC, Boulte TE (2000) Efficient super-resolution via image warping. Image Vis Comput 18(10):761–771

    Article  Google Scholar 

  8. Combettes P, Pesquet J-C (2011) Proximal splitting methods in signal processing, in fixed-point algorithms for inverse problems in science and engineering Bauschke H et al (Eds), pp. 185–212, Springer

  9. Dong WS, Shi GM, Hu XC, Ma Y (2014) Nonlocal Sparse and Low-rank Regularization for Optical Flow Estimation. IEEE Transactions on Image Process. 23(10):4527–4538

    Article  MathSciNet  Google Scholar 

  10. Duchi J, Shalev-Shwartz S, Singer Y, Chandra T (2008) Efficient projections onto the l1-ball for learning in high dimensions, in Proc. 25th international conference on Machine learning, pp. 272–279.

  11. Farsiu S, Dirk Robinson M, Elad M, Milanfar P (2004) Fast and robust multiframe super resolution. IEEE Trans Image Process 13(10):1327–1344

    Article  Google Scholar 

  12. Ferreira RU, Hung EM, Queiroz RLD (2015) Clustering of matched features and gradient matching for mixed-resolution video super-resolution. IEEE international symposium on circuits & systems, pp. 1202–1205

  13. Gabay D, Mercier B (1976) A dual algorithm for the solution of nonlinear variational problems via nite element approximations. Computers and Mathematics with Applications 2:17–40

    Article  MATH  Google Scholar 

  14. Goldstein T, Osher S (2009) The split bregman method for l1-regularized problems. SIAM Journal on Imaging Sciences 2:323–343

    Article  MathSciNet  MATH  Google Scholar 

  15. Hardie RC, Barnard KJ, Armstrong EE (1997) Join MAP registration and high resolution image estimation using a sequence of undersampled images. IEEE Trans Image Process 6(12):1621–1633

    Article  Google Scholar 

  16. Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision, 2n edition. Cambridge University Press, New York

  17. He Y, Yap KH, Chen L, Chau LP (2007) Image registration and super-resolution. IEEE Trans Image Process 16(11):2830–2841

  18. Horn B, Schunck B (1981) Determining optical flow. Artif Intell 16:185–203

    Article  Google Scholar 

  19. Huang T, Tsai R (1984) Multi-frame image restoration and registration. Advances in Computer Vision and Image Processing 1:317–339

    Google Scholar 

  20. Huangpeng Q, Zeng X, Fan J, Feng J Sun Q (2015) Improved multi-frame super resolution via multi-frame blind deblurring using the alternating direction method of multipliers, IEEE International Conference on Progress in Informatics and Computing, Nanjing, 18–20 December, 2015, pp 281–285

  21. Irani M, Peleg S (1990) Super resolution from image sequences. In Proceedings of 10th International Conference on Pattern Recognition, 2:115–120

  22. Katsaggelos AK , Molina R, Mateos J (2007) Super resolution of images and video, Morgan and Claypool, San Rafael

  23. Keren D, Peleg S, Brada R (1988) Image sequence enhancement using subpixel displacements. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 742–746

  24. Kim SP, Bose NK, Valenzuela HM (1990) Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans Acoust Speech Signal Process 38:1013–1027

    Article  Google Scholar 

  25. Kotera J, Šroubek F, Milanfar P (2013) Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors. Lectures Notes in Computer Science 8048:59–66

    Article  MathSciNet  Google Scholar 

  26. Krishnan D, Fergus R (2009a) Fast image deconvolution using hyper-laplacian priors

  27. Krishnan D, Fergus R (2009b) Fast image deconvolution using hyper-laplacian priors

  28. Lee ES, Kang MG (2003) Regularized Adaptive High-Resolution Image Reconstruction Considering Inaccurate Subpixel Registration. IEEE Trans Image Process 12(7):826–837

    Article  Google Scholar 

  29. Liu L, Liu B,Huang H, Bovik AC (2014a) “SSEQ Software Release”, URL: http://live.ece.utexas.edu/research/quality/SSEQ_release.zip

  30. Liu L, Liu B, Huang H, Bovik AC (2014b) "No-reference Image Quality Assessment Based on Spatial and Spectral Entropies",. Signal Process Image Commun 29(8):856–863

    Article  Google Scholar 

  31. Milanfar P (2010) Super-Resolution imaging, digital imaging and computer vision. Taylor&Francis, Boca Raton

    Google Scholar 

  32. Mitzel D, Pock T, Schoenemann T, Cremers D (2009) Video Super Resolution using Duality Based TV-L1 Optical Flow. Lect Notes Comput Sci. 5748:432–441

  33. Shen H, Zhang L, Huang B, Li P (2007) A MAP approach for joint motion estimation, segmentation and super-resolution. IEEE Trans Image Process 16(2):479–490

    Article  MathSciNet  Google Scholar 

  34. Šroubek F, Cristóbal G, Flusser J (2007) A Unified Approach to Superresolution and Multichannel Blind Deconvolution. IEEE Trans Image Process 16(9):2322–2332

    Article  MathSciNet  Google Scholar 

  35. Šroubek F, Flusser J, Cristóbal G Multiframe blind deconvolution coupled with frame registration and resolution enhancement, Blind Image Deconvolution: Theory Appl, pp. 317–348 , Eds: Campisi P, Egiazarian K

  36. Stark H, Oskoui P (1989) High resolution image recovery from imageplane arrays, using convex projections. J. Opt. Soc. Amer A 6:1715–1726

    Article  Google Scholar 

  37. Takeda H, Farsiu S, Milanfar P (2007) Kernel regression for image processing and reconstruction. IEEE Trans Image Process 16(2):349–366

    Article  MathSciNet  Google Scholar 

  38. Tom BC, Katsaggelos AK, Galatsanos NP (1994) Reconstruction of a high resolution image from registration and restoration of low resolution images. In Proceedings of IEEE international conference on image processing, pp. 553–557

  39. Tom BC, Katsaggelos AK, Galatsanos NP (1994) Reconstruction of a high resolution image from registration and restoration of low resolution images. In Proceedings of IEEE international conference on image processing, pp 553–557

  40. Wedel A, Pock T, Cremers D (2009) Structure- and motion-adaptive regularization for high accuracy optic flow, In ICCV

  41. Zhao WY, Sawhney H (2002) Is super-resolution with optical flow feasible? Computer vision — ECCV Volume 2350 of the series lecture notes in computer science pp 599–613

  42. Villena S, Vega M, Babacan SD, Molina R, Katsaggelos AK (2013) Bayesian combination of sparse and non sparse priors in image super resolution. Digital Signal Processing 23(2):530–541

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information System and Management, National University of Defense Technology, Changsha, People’s Republic of China

    Qizi Huangpeng, Xiangrong Zeng, Quan Sun, Jun Fan, Jing Feng & Zhengqiang Pan

Authors
  1. Qizi Huangpeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiangrong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Quan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jing Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhengqiang Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qizi Huangpeng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huangpeng, Q., Zeng, X., Sun, Q. et al. Super-resolving blurry multiframe images through multiframe blind deblurring using ADMM. Multimed Tools Appl 76, 13563–13579 (2017). https://doi.org/10.1007/s11042-016-3770-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3770-y

Keywords

Search

Navigation