Advertisement

Journal of Mathematical Imaging and Vision

, Volume 48, Issue 1, pp 185–201 | Cite as

A Linear Systems Approach to Imaging Through Turbulence

  • Mario MicheliEmail author
  • Yifei Lou
  • Stefano Soatto
  • Andrea L. Bertozzi
Article

Abstract

In this paper we address the problem of recovering an image from a sequence of distorted versions of it, where the distortion is caused by what is commonly referred to as ground-level turbulence. In mathematical terms, such distortion can be described as the cumulative effect of a blurring kernel and a time-dependent deformation of the image domain. We introduce a statistical dynamic model for the generation of turbulence based on linear dynamical systems (LDS). We expand the model to include the unknown image as part of the unobserved state and apply Kalman filtering to estimate such state. This operation yields a blurry image where the blurring kernel is effectively isoplanatic. Applying blind nonlocal Total Variation (NL-TV) deconvolution yields a sharp final result.

Keywords

Image processing Atmospheric turbulence Deblurring Kalman filter 

Notes

Acknowledgements

Mario Micheli’s research was partially supported by ONR grant N000140910256. Yifei Lou and Andrea Bertozzi were partially supported by ONR grants N00014101022, N000141210040 and by NSF grants DMS-0914856, DMS-1118971. Stefano Soatto was partially supported by ONR grant N0001411100863. The authors would like to thank Dr. Alan Van Nevel at the U.S. Naval Air Warfare Center (China Lake, California) for providing the image data. We are also deeply grateful to Stanley Osher and Jérôme Gilles of UCLA, and to Angelo Cenedese of Università di Padova (Italy) for the insightful discussions on the topic. We would also like to thank Xiaoqun Zhang, formerly at UCLA and now at Shanghai Jiao Tong University, for providing the nonlocal Total Variation (NL-TV) deconvolution Matlab code.

References

  1. 1.
    Andrews, H.C., Hunt, B.R.: Digital Image Restoration. Prentice Hall, Englewood Cliffs (1977) Google Scholar
  2. 2.
    Aubailly, M., Vorontsov, M.A., Carhart, G.W., Valley, M.T.: Automated video enhancement from a stream of atmospherically-distorted images: the lucky-region fusion approach. In: SPIE Atmospheric Optics: Models, Measurements, and Target-in-the-Loop Propagation, San Diego, CA, Aug. (2009) Google Scholar
  3. 3.
    Bascle, B., Blake, A., Zisserman, A.: Motion deblurring and super-resolution from an image sequence. In: Proceedings of the Fourth European Conference on Computer Vision, pp. 573–582. Springer, Berlin (1996) Google Scholar
  4. 4.
    Beg, M.F., Miller, M.I., Trouvé, A., Younes, L.: Computing large deformation metric mappings via geodesic flows of diffeomorphisms. Int. J. Comput. Vis. 61(2), 139–157 (2005) CrossRefGoogle Scholar
  5. 5.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996) CrossRefGoogle Scholar
  6. 6.
    Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA (2005) Google Scholar
  7. 7.
    Buades, A., Coll, B., Morel, J.-M.: A review of image denoising methods, with a new one. Multiscale Model. Simul. 4(3), 490–530 (2005) CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004) CrossRefMathSciNetGoogle Scholar
  9. 9.
    Chan, A.B., Mahadevan, V., Vasconcelos, N.: Generalized Stauffer–Grimson background subtraction for dynamic scenes. Mach. Vis. Appl. 22(5), 751–766 (2011) CrossRefGoogle Scholar
  10. 10.
    Chan, T.F., Wong, C.-K.: Total variation blind deconvolution. IEEE Trans. Image Process. 7(3), 370–375 (1998) CrossRefGoogle Scholar
  11. 11.
    Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003) CrossRefzbMATHGoogle Scholar
  12. 12.
    Fried, D.L.: Statistics of a geometric representation of wavefront distortion. J. Opt. Soc. Am. 55(11), 1427–1435 (1965) CrossRefMathSciNetGoogle Scholar
  13. 13.
    Fried, D.L.: Optical resolution through a randomly inhomogeneous medium for very long and very short exposures. J. Opt. Soc. Am. 56(10), 1372–1379 (1966) CrossRefMathSciNetGoogle Scholar
  14. 14.
    Fried, D.L.: Probability of getting a lucky short-exposure image through turbulence. J. Opt. Soc. Am. 68, 1651–1657 (1978) CrossRefGoogle Scholar
  15. 15.
    Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7(3), 1005–1028 (2008) CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Gilles, J., Dagobert, T., De Franchis, C.: Atmospheric turbulence restoration by diffeomorphic image registration and blind deconvolution. In: Advanced Concepts for Intelligent Vision Systems. Lecture Notes in Computer Science, vol. 5259, pp. 400–409. Springer, Berlin (2008) CrossRefGoogle Scholar
  17. 17.
    Gilles, J., Osher, S.: Fried deconvolution. CAM Technical Report 11-62, UCLA Department of Mathematics, Dec. (2011) Google Scholar
  18. 18.
    Goldstein, T., Osher, S.: The split Bregman method for L 1 regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009) CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996) zbMATHGoogle Scholar
  20. 20.
    He, L., Marquina, A., Osher, S.: Blind deconvolution using TV regularization and Bregman iteration. Int. J. Imaging Syst. Technol. 15(1), 74–83 (2005) CrossRefGoogle Scholar
  21. 21.
    Hirsch, M., Sra, S., Schölkopf, B., Harmeling, S.: Efficient filter flow for space-variant multiframe blind deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2010), San Francisco, CA, June, pp. 607–614 (2010) Google Scholar
  22. 22.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981) CrossRefGoogle Scholar
  23. 23.
    Kumar, P.R., Varaiya, P.: Stochastic Systems: Estimation, Identification, and Adaptive Control. Prentice Hall, Englewood Cliffs (1986) zbMATHGoogle Scholar
  24. 24.
    Li, D., Mersereau, R.M., Simske, S.: Atmospheric turbulence-degraded image restoration using principal components analysis. IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007) CrossRefGoogle Scholar
  25. 25.
    Lin, H.T., Tai, Y.-W., Brown, M.S.: Motion regularization for matting motion blurred objects. IEEE Trans. Pattern Anal. Mach. Intell. (2011, to appear) Google Scholar
  26. 26.
    Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice Hall, Englewood Cliffs (1999) Google Scholar
  27. 27.
    Lou, Y., Zhang, X., Osher, S., Bertozzi, A.: Image recovery via nonlocal operators. Journal of Scientific Computing 2(42) (2010) Google Scholar
  28. 28.
    Mao, Y., Gilles, J.: Non-rigid geometric distortions corrections – application to atmospheric turbulence stabilization. Inverse Probl. Imaging 6(3) (2012, to appear) Google Scholar
  29. 29.
    Metari, S., Deschênes, F.: A new convolution kernel for atmospheric point spread function applied to computer vision. In: Proceedings of the IEEE 11th International Conference on Computer Vision (ICCV 2007), Rio de Janeiro, Brazil, Oct. 2007 Google Scholar
  30. 30.
    Miller, M.I., Younes, L.: Group actions, homeomorphisms, and matching: a general framework. Int. J. Comput. Vis. 41(1/2), 61–84 (2001) CrossRefzbMATHGoogle Scholar
  31. 31.
    Paxman, R.G., Seldin, J.H., Löfdahl, M.G., Scharmer, G.B., Keller, C.U.: Evaluation of phase-diversity techniques for solar-image restoration. Astrophys. J. 466, 1087–1099 (1996) CrossRefGoogle Scholar
  32. 32.
    Paxman, R.G., Thelen, B.J., Seldin, J.H.: Phase-diversity correction of turbulence-induced space-variant blur. Opt. Lett. 19(16), 1231–1233 (1994) CrossRefGoogle Scholar
  33. 33.
    Pennec, X.: Probabilities and Statistics on Riemannian Manifolds: a Geometric Approach. Research Report 5093, INRIA, January 2004 Google Scholar
  34. 34.
    Roddier, F.: Adaptive Optics in Astronomy. Cambridge University Press, Cambridge (2004) Google Scholar
  35. 35.
    Roggemann, M.C., Stoudt, C.A., Welsh, B.M.: Image-spectrum signal-to-noise-ratio improvements by statistical frame selection for adaptive-optics imaging through atmospheric turbulence. Opt. Eng. 33(10), 3254–3264 (1994) CrossRefGoogle Scholar
  36. 36.
    Roggemann, M.C., Welsh, B.M.: Imaging Through Turbulence. CRC Press, Boca Raton (1996) Google Scholar
  37. 37.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992) CrossRefzbMATHGoogle Scholar
  38. 38.
    Shimizu, M., Yoshimura, S., Tanaka, M., Okutomi, M.: Super-resolution from image sequence under influence of hot-air optical turbulence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, Alaska, June 2008 Google Scholar
  39. 39.
    Shumway, R.H., Stoffer, D.S.: An approach to time series smoothing and forecasting using the EM algorithm. J. Time Ser. Anal. 3(4), 253–264 (1982) CrossRefzbMATHGoogle Scholar
  40. 40.
    Stone, J., Hu, P.H., Mills, S.P., Ma, S.: Anisoplanatic effects in finite-aperture optical systems. J. Opt. Soc. Am. A 11(1), 347–357 (1994) CrossRefGoogle Scholar
  41. 41.
    Tahtali, M., Lambert, A., Fraser, D.: Self-tuning Kalman filter estimation of atmospheric warp. In: SPIE Image Reconstruction from Incomplete Data, San Diego, CA, Aug. 2008 Google Scholar
  42. 42.
    Tai, Y.-W., Tan, P., Brown, M.S.: Richardson-Lucy deblurring for scenes under a projective motion path. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1603–1618 (2011) CrossRefGoogle Scholar
  43. 43.
    Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problems, Winston, Washington DC (1977) Google Scholar
  44. 44.
    Van Overschee, P., De Moor, B.: N4sid: subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica 30(1), 75–93 (1994) CrossRefzbMATHMathSciNetGoogle Scholar
  45. 45.
    Welsh, B.M., Gardner, C.S.: Effects of turbulence-induced anisoplanatism on the imaging performance of adaptive-astronomical telescopes using laser guide stars. J. Opt. Soc. Am. A 8(1), 69–81 (1991) CrossRefGoogle Scholar
  46. 46.
    Younes, L.: Shapes and Diffeomorphisms. Applied Mathematical Sciences, vol. 171. Springer, Berlin (2010) CrossRefzbMATHGoogle Scholar
  47. 47.
    Zhu, X., Milanfar, P.: Image reconstruction from videos distorted by atmospheric turbulence. In: SPIE Electronic Imaging, San Jose, CA, Jan. 2010 Google Scholar
  48. 48.
    Zhu, X., Milanfar, P.: Stabilizing and deblurring atmospheric turbulence. In: Proceedings of the of the IEEE International Conference on Computational Photography (ICCP 2011), Carnegie Mellon University, Pittsburgh (2011) Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Mario Micheli
    • 1
    Email author
  • Yifei Lou
    • 2
  • Stefano Soatto
    • 3
  • Andrea L. Bertozzi
    • 1
  1. 1.Department of MathematicsUCLALos AngelesUSA
  2. 2.School of Electrical and Computer EngineeringGeorgia Inst. of TechnologyAtlantaUSA
  3. 3.Department of Computer ScienceUCLALos AngelesUSA

Personalised recommendations