Advertisement

International Journal of Computer Vision

, Volume 115, Issue 3, pp 253–278 | Cite as

Fast Approximations of Shift-Variant Blur

  • Loïc DenisEmail author
  • Eric Thiébaut
  • Ferréol Soulez
  • Jean-Marie Becker
  • Rahul Mourya
Article

Abstract

Image deblurring is essential in high resolution imaging, e.g., astronomy, microscopy or computational photography. Shift-invariant blur is fully characterized by a single point-spread-function (PSF). Blurring is then modeled by a convolution, leading to efficient algorithms for blur simulation and removal that rely on fast Fourier transforms. However, in many different contexts, blur cannot be considered constant throughout the field-of-view, and thus necessitates to model variations of the PSF with the location. These models must achieve a trade-off between the accuracy that can be reached with their flexibility, and their computational efficiency. Several fast approximations of blur have been proposed in the literature. We give a unified presentation of these methods in the light of matrix decompositions of the blurring operator. We establish the connection between different computational tricks that can be found in the literature and the physical sense of corresponding approximations in terms of equivalent PSFs, physically-based approximations being preferable. We derive an improved approximation that preserves the same desirable low complexity as other fast algorithms while reaching a minimal approximation error. Comparison of theoretical properties and empirical performances of each blur approximation suggests that the proposed general model is preferable for approximation and inversion of a known shift-variant blur.

Keywords

Blur Deconvolution Inverse problems Image restoration PSF 

Notes

Acknowledgments

This work has been supported by Project MiTiV funded by the French National Research Agency (ANR DEFI 09-EMER-008-01). It has been performed in part within the framework of the LABEX PRIMES (ANR-11-LABX-0063) of Université de Lyon, within the program “Investissements d’Avenir” (ANR-11-IDEX-0007) operated by the French National Research Agency (ANR). Rahul Mourya acknowledges a PhD Grant funded by the Région Rhône-Alpes.

References

  1. Almeida, M., & Almeida, L. (2009). Blind deblurring of foreground-background images. In 2009 16th IEEE International Conference on Image Processing (ICIP), IEEE, (pp. 1301–1304).Google Scholar
  2. Ayers, G., & Dainty, J. C. (1988). Iterative blind deconvolution method and its applications. Optics Letters, 13(7), 547–549.CrossRefGoogle Scholar
  3. Bardsley, J., Jefferies, S., Nagy, J., & Plemmons, R. (2006). A computational method for the restoration of images with an unknown, spatially-varying blur. Optics Express, 14(5), 1767–1782.CrossRefGoogle Scholar
  4. Ben Hadj, S., Blanc-Feraud, L., Aubert, G., & Engler, G. (2013). Blind restoration of confocal microscopy images in presence of a depth-variant blur and poisson noise. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, (pp. 915–919).Google Scholar
  5. Blomgren, P., & Chan, T. F. (1998). Color Tv: total variation methods for restoration of vector-valued images. IEEE Transactions on Image Processing, 7(3), 304–309.CrossRefGoogle Scholar
  6. Calvetti, D., Lewis, B., & Reichel, L. (2000). Restoration of images with spatially variant blur by the gmres method. In Proceedings of SPIE, (vol. 4116, p. 364).Google Scholar
  7. Campisi, P., & Egiazarian, K. (Eds.). (2007). Blind image deconvolution: theory and applications. Boca Raton: CRC Press.Google Scholar
  8. Chacko, N., Liebling, M., & Blu, T. (2013). Discretization of continuous convolution operators for accurate modeling of wave propagation in digital holography. JOSA A, 30(10), 2012–2020.CrossRefGoogle Scholar
  9. Chakrabarti, A., Zickler, T., & Freeman, W. (2010). Analyzing spatially-varying blur. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, (pp. 2512–2519).Google Scholar
  10. Chen, Y., Ranftl, R., & Pock, T. (2014). Insights into analysis operator learning: A view from higher-order filter-based mrf model. IEEE Transactions on Image Processing, 23(3), 1060–1072.MathSciNetCrossRefGoogle Scholar
  11. Chuang, Y., Curless, B., Salesin, D., & Szeliski, R. (2001). A bayesian approach to digital matting. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, IEEE, (vol. 2, pp. II-264).Google Scholar
  12. Cresci, G., Davies, R., Baker, A., & Lehnert, M. (2005). Accounting for the anisoplanatic point spread function in deep wide-field adaptive optics images. A & A, 438, 757–767. doi: 10.1051/0004-6361:20052890. arXiv:astro-ph/0504467.
  13. Delbracio, M., Musé, P., Almansa, A., & Morel, J. M. (2012). The non-parametric sub-pixel local point spread function estimation is a well posed problem. International Journal of Computer Vision, 96, 175–194. doi: 10.1007/s11263-011-0460-0.zbMATHMathSciNetCrossRefGoogle Scholar
  14. Demoment, G. (1989). Image reconstruction and restoration: Overview of common estimation structures and problems. IEEE Transactions on Acoustics, Speech and Signal Processing, 37(12), 2024–2036.CrossRefGoogle Scholar
  15. Denis, L., Thiébaut, E., & Soulez, F. (2011). Fast model of space-variant blurring and its application to deconvolution in astronomy. In IEEE International Conference on Image Process (ICIP).Google Scholar
  16. Escande, P., & Weiss, P. (2014). Numerical computation of spatially varying blur operators a review of existing approaches with a new one. arXiv preprint arXiv:1404.1023.
  17. Fish, D. A., Grochmalicki, J., & Pike, E. R. (1996). Scanning singular-value-decomposition method for restoration of images with space-variant blur. Journal of the Optical Society of America A, 13(3), 464–469. doi: 10.1364/JOSAA.13.000464.CrossRefGoogle Scholar
  18. Flicker, R., & Rigaut, F. J. (2005). Anisoplanatic deconvolution of adaptive optics images. Journal of the Optical Society of America A, 22(3), 504–513.CrossRefGoogle Scholar
  19. Freeman, W. T., & Adelson, E. H. (1991). The design and use of steerable filters. IEEE Transactions on Pattern analysis and machine intelligence, 13(9), 891–906.CrossRefGoogle Scholar
  20. Frigo, M. (1999). A fast Fourier transform compiler. In ACM Sigplan Notices, (vol. 34, pp. 169–180).Google Scholar
  21. Gilad, E., & Hardenberg, J. (2006). A fast algorithm for convolution integrals with space and time variant kernels. Journal of Computational Physics, 216(1), 326–336.zbMATHMathSciNetCrossRefGoogle Scholar
  22. Goodman, J. (2008). Introduction to Fourier optics. New York: McGraw-hill.Google Scholar
  23. Gorski, J., Pfeuffer, F., & Klamroth, K. (2007). Biconvex sets and optimization with biconvex functions: a survey and extensions. Mathematical Methods of Operations Research, 66, 373–407. doi: 10.1007/s00186-007-0161-1.zbMATHMathSciNetCrossRefGoogle Scholar
  24. Greenspan, H., Belongie, S., Goodman, R., Perona, P., Rakshit, S., & Anderson, C.H. (1994). Overcomplete steerable pyramid filters and rotation invariance. In Proceedings CVPR’94, 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994, IEEE, (pp. 222–228).Google Scholar
  25. Hirsch, M., Sra, S., Scholkopf, B., & Harmeling, S. (2010). Efficient filter flow for space-variant multiframe blind deconvolution. In IEEE Computer Vision Pattern Recognition, (pp 607–614).Google Scholar
  26. Levin, A. (2007). Blind motion deblurring using image statistics. Advances in Neural Information Processing Systems, 19, 841.Google Scholar
  27. Maalouf, E., Colicchio, B., & Dieterlen, A. (2011). Fluorescence microscopy three-dimensional depth variant point spread function interpolation using Zernike moments. JOSA A, 28(9), 1864–1870.CrossRefGoogle Scholar
  28. Mahajan, V. N. (1994). Zernike circle polynomials and optical aberrations of systems with circular pupils. Applied Optics, 33(34), 8121–8121.CrossRefGoogle Scholar
  29. Martin, C. D., & Porter, M. A. (2012). The extraordinary svd. The American Mathematical Monthly, 119(10), 838–851.zbMATHMathSciNetCrossRefGoogle Scholar
  30. Matakos, A., Ramani, S., & Fessler, J. (2013). Accelerated edge-preserving image restoration without boundary artifacts. IEEE Transactions on Image Processing, 22(5), 2019–2029. doi: 10.1109/TIP.2013.2244218.MathSciNetCrossRefGoogle Scholar
  31. Miraut, D., & Portilla, J. (2012). Efficient shift-variant image restoration using deformable filtering (part i). EURASIP Journal on Advances in Signal Processing, 2012(1), 1–20.CrossRefGoogle Scholar
  32. Mugnier, L. M., Robert, C., Conan, J. M., Michau, V., & Salem, S. (2001). Myopic deconvolution from wave-front sensing. JOSA A, 18(4), 862–872.CrossRefGoogle Scholar
  33. Nagy, J., & O’Leary, D. (1998). Restoring images degraded by spatially variant blur. IAM Journal on Scientific Computing, 19, 1063.zbMATHMathSciNetCrossRefGoogle Scholar
  34. Nagy, J. G., Palmer, K., & Perrone, L. (2004). Iterative methods for image deblurring: A Matlab object-oriented approach. Numerical Algorithms, 36, 73–93. doi: 10.1023/B:NUMA.0000027762.08431.64.zbMATHMathSciNetCrossRefGoogle Scholar
  35. Ng, J., Prager, R., Kingsbury, N., Treece, G., & Gee, A. (2007). Wavelet restoration of medical pulse-echo ultrasound images in an em framework. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, 54(3), 550–568.CrossRefGoogle Scholar
  36. Nocedal, J. (1980). Updating quasi-Newton matrices with limited storage. Mathematics of Computation, 35(151), 773–782. doi: 10.1090/S0025-5718-1980-0572855-7.
  37. Perona, P. (1995). Deformable kernels for early vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(5), 488–499.CrossRefGoogle Scholar
  38. Porter, T., & Duff, T. (1984). Compositing digital images. ACM Siggraph Computer Graphics, 18(3), 253–259.CrossRefGoogle Scholar
  39. Preza, C., & Conchello, J. A. (2004). Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy. Journal of the Optical Society of America A, 21(9), 1593–1601. doi: 10.1364/JOSAA.21.001593.CrossRefGoogle Scholar
  40. Reeves, S. J. (2005). Fast image restoration without boundary artifacts. IEEE Transactions on Image Processing, 14(10), 1448–1453.CrossRefGoogle Scholar
  41. Richardson, W. H. (1972). Bayesian-based iterative method of image restoration. JOSA, 62(1), 55–59.CrossRefGoogle Scholar
  42. Rogers, A., & Fiege, J. (2011). Strong gravitational lens modeling with spatially variant point-spread functions. The Astrophysical Journal, 743, 68.CrossRefGoogle Scholar
  43. Rudin, L., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60, 259–268. doi: 10.1016/0167-2789(92)90242-F.zbMATHCrossRefGoogle Scholar
  44. Sorel, M., & Flusser, J. (2008). Space-variant restoration of images degraded by camera motion blur. IEEE Transactions on Image Processing, 17(2), 105–116.MathSciNetCrossRefGoogle Scholar
  45. Soulez, F., Denis, L., Thiébaut, É., Fournier, C., & Goepfert, C. (2007). Inverse problem approach in particle digital holography: Out-of-field particle detection made possible. JOSA A, 24(12), 3708–3716.CrossRefGoogle Scholar
  46. Soulez, F., Denis, L., Tourneur, Y., & Thiébaut, E. (2012). Blind deconvolution of 3d data in wide field fluorescence microscopy. In 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, (pp 1735–1738).Google Scholar
  47. Titterington, D. M. (1985). General structure of regularization procedures in image reconstruction. Astron & Astrophys, 144, 381–387.MathSciNetGoogle Scholar
  48. Wang, J., & Cohen, M. (2007). Image and video matting: a survey. Foundations and Trends in Computer Graphics and Vision, 3(2), 97–175.CrossRefGoogle Scholar
  49. Wei, J., Bouman, C., & Allebach, J. (2014). Fast space-varying convolution using matrix source coding with applications to camera stray light reduction. IEEE Transactions on Image Processing, 23(5), 1965–1979. doi: 10.1109/TIP.2014.2311657.MathSciNetCrossRefGoogle Scholar
  50. Whyte, O., Sivic, J., Zisserman, A., & Ponce, J. (2010). Non-uniform deblurring for shaken images. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, (pp 491–498).Google Scholar
  51. Zoran, D., & Weiss, Y. (2011). From learning models of natural image patches to whole image restoration. In 2011 IEEE International Conference on Computer Vision (ICCV), IEEE, (pp. 479–486).Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Loïc Denis
    • 1
    • 2
    • 3
    Email author
  • Eric Thiébaut
    • 4
    • 5
    • 6
    • 7
  • Ferréol Soulez
    • 4
    • 5
    • 6
    • 7
  • Jean-Marie Becker
    • 1
    • 2
    • 3
  • Rahul Mourya
    • 1
    • 2
    • 3
  1. 1.Université de LyonSaint-ÉtienneFrance
  2. 2.Laboratoire Hubert CurienCNRS, UMR 5516Saint-ÉtienneFrance
  3. 3.Université de Saint-Etienne, Jean MonnetSaint-ÉtienneFrance
  4. 4.Université de LyonLyonFrance
  5. 5.Université de Lyon 1VilleurbanneFrance
  6. 6.Centre de Recherche Astrophysique de LyonObservatoire de LyonSaint-Genis Laval CEDEXFrance
  7. 7.CNRS, UMR 5574, Ecole Normale Supérieure de LyonLyonFrance

Personalised recommendations