Journal of Real-Time Image Processing

, Volume 2, Issue 1, pp 11–22 | Cite as

Real-time stabilization of long range observation system turbulent video

  • Barak Fishbain
  • Leonid P. Yaroslavsky
  • Ianir A. Ideses
Original Research Paper

Abstract

The paper presents a real-time algorithm that compensates image distortions due to atmospheric turbulence in video sequences, while keeping the real moving objects in the video unharmed. The algorithm involves (1) generation of a “reference” frame, (2) estimation, for each incoming video frame, of a local image displacement map with respect to the reference frame, (3) segmentation of the displacement map into two classes: stationary and moving objects; (4) turbulence compensation of stationary objects. Experiments with both simulated and real-life sequences have shown that the restored videos, generated in real-time using standard computer hardware, exhibit excellent stability for stationary objects while retaining real motion.

Keywords

LOROS Turbulence compensation Real-time processing Rank filtering Optical-flow 

References

  1. 1.
    Roggermann, M.C., Welsh, B.: Imaging Through Turbulence, Chap. 3, CRC Press, USA, pp. 57–115 (1996)Google Scholar
  2. 2.
    Farmer, W.M.: The Atmospheric Filter, vol. I Sources. JCD Publishing, (2001)Google Scholar
  3. 3.
    Welsh, B.M., Gardner, C.S.: Performance analysis of adaptive optics systems using slope sensors. J. Opt. Soc. Am. A 6:1913–1923 (1989)Google Scholar
  4. 4.
    Thorpe, G., Lambert, A., Fraser, D.: Atmospheric turbulence visualization through image time-sequence registration. In: Proceedings International Conference on Pattern Recognition, vol. 2, pp. 1768–1770. IEEE Computer Society, Brisbane (1998)Google Scholar
  5. 5.
    Ellerbroek, B.: First-order performance evaluation of adaptive-optics systems for atmospheric-turbulence compensation in extended-field-of-view astronomical telescopes. J. Opt. Soc. Am. A, 11(2):783 (1994)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Farmer, W.M.: The Atmospheric Filter, vol. II Sources. JCD Publishing, Winter Park (2001)Google Scholar
  7. 7.
    Sheppard, D.G., Hunt, B.R., Marcellin, M.W.: Iterative multiframe super-resolution algorithms for atmospheric turbulence-degraded imagery. J. Opt. Soc. Am. A, 15(4):972–992 (1998)CrossRefGoogle Scholar
  8. 8.
    Sadot, D., Kopeika, N.: Imaging through the atmosphere: practical instrumentation-based theory and verification of aerosol modulation transfer function. J. Opt. Soc. Am. A, 10(1):172 (1993)Google Scholar
  9. 9.
    Cohen, B., Avrin, V., Belitsky, M., Dinstein, I.: Generation of a restored image from a videosequence recorded under turbulence effects. Opt. Eng. 36(12):3312–3317 (1997)CrossRefGoogle Scholar
  10. 10.
    Frieden, B.R.: Turbulent image reconstruction using object power spectrum information. Opt. Commun. 109(3–4): 227–230 (1994)CrossRefGoogle Scholar
  11. 11.
    Wang, Y., Frieden, B.R.: Minimum entropy-neural network approach to turbulent-image reconstruction. Appl. Opt. 34(26):5938–5944 (1995)Google Scholar
  12. 12.
    Mohammed, A.T., Burge, R.E.: Short-exposure turbulent image reconstructions. J. Phys. D Appl. Phys. 21(7):1067–1077 (1988)CrossRefGoogle Scholar
  13. 13.
    David, H.F., Joseph, W.M., Mark J.T.S.: Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach. In: Wee, S.J., Apostolopoulos J.G. (eds.) IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP01), vol. 3, pp. 1881–1884. IEEE International Society, Salt Lake City (2001)Google Scholar
  14. 14.
    van der Elst, H., van Schalkwyk, J.J.D.: Modelling and restoring images distorted by atmospheric turbulence. South African Symposium on Communications and Signal Processing (COMSIG-94), pp. 162–167. IEEE International Society, Stellenbosch (1994)Google Scholar
  15. 15.
    Gepshtein, Sh., Shteinman, A., Fishbain, B., Yaroslavsky, L.: Restoration of atmospheric turbulent video containing real motion using elastic image registration. In: The 2004 European Signal Processing Conference (EUSIPCO-2004), John Platt, pp. 477–480. Vienna, Austria, (2004)Google Scholar
  16. 16.
    Yaroslavsky, L., Fishbain, B., Shteinman, A., Gepshtein, Sh.: Processing and fusion of thermal and video sequences for terrestrial long range observation systems. In: Johan, S (ed.) The 7th International Conference on Information Fusion, pp. 848–855, International Society of Information Fusion, Stockholm, Sweden, June 2004Google Scholar
  17. 17.
    Yaroslavsky, L.P., Fishbain, B., Ideses, I., Slasky, D., Hadas, Z.: Simple methods for real-time stabilization of turbulent video. In: Calvo, M.L., Pavlov, A.V., Jahns, J. (eds.) Proceeding of ICO topical meeting on optoinformatics/information photonics, ITMO, pp. 138–140. St Petersburg, Russia (2006)Google Scholar
  18. 18.
    Fishbain, B., Yaroslavsky, L.P., Ideses, I.A., Shtern, A., Ben-Zvi, O.: Real-time stabilization of long-range observation system turbulent video. In: Proceedings of Real-Time Image Processing/Electronic Imaging 2007, SPIE Vol. 6496, San-Jose, CA, USA, 28 January–1 February 2007Google Scholar
  19. 19.
    Yaroslavsky, L.P.: Digital Holography and Digital Image Processing. Kluwer, Boston (2003)Google Scholar
  20. 20.
    Bondeau, C., Bourennane, E.: Restoration of images degraded by the atmospheric turbulence. In: Proceedings of the 4th International Conference on Signal Processing (ICSP), vol.2, pp. 1056–1059. Beijing China (1998)Google Scholar
  21. 21.
    Glick, Y., Baram, A., Loebenstein, H.M., Azar Z:. Restoration of turbulence-degraded images by the most-common method. Appl. Opt. 30(27):3924–3929 (1991)Google Scholar
  22. 22.
    Cheung, S.C., Kamath, C.: Robust techniques for background subtraction in urban traffic video. Video Communications and Image Processing, SPIE Electronic Imaging, San Jose (2004)Google Scholar
  23. 23.
  24. 24.
    Huang, T.S., Yang, G.J., Yang, G.Y.: A Fast Two-Dimensional Median Filtering Algorithm. IEEE Trans. Acoust. Speech Signal Process. ASSP-27, 13 (1979)Google Scholar
  25. 25.
    Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. J Imaging Vis. 14(3), 245–255 (2001)MATHCrossRefGoogle Scholar
  26. 26.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings Of the 7th International Joint Conference on Artificial Intelligence (IJCAI), pp. 674–679. Vancouver BC (1981)Google Scholar
  27. 27.
    Horn, B., Schunck, B.: Determining optical flow. Art. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  28. 28.
    Mitiche, A., Bouthemy, P.: Computation and analysis of image motion: a synopsis of current problems and methods. Int. J. Comput. Vis. 19(1), 29–55 (1996)CrossRefGoogle Scholar
  29. 29.
    Barron, L.J., Fleet, D.J., Beachemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)CrossRefGoogle Scholar
  30. 30.
    Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacement. Int. J. Comput. Vis. 39(1), 41–56 (2000)MATHCrossRefGoogle Scholar
  31. 31.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation based on Theory for Wrapping. European Conference on Computer Vision (ECCV04), Vaclav Hlavac, vol. 4, pp. 25–36. Springer LNCS, Prague Czech Republic (2004)Google Scholar
  32. 32.
    Ben-Ari, R., Sochen, N.: A general framework for regularization in PDE based computation of optical flow via embedded maps and minimal surfaces. In: Proceedings of IEEE Computer Vision and Pattern Recognition Conference, Dan Huttenlocher and David Forsyth, pp. 529–536. IEEE Computer Society, New-York (2006)Google Scholar
  33. 33.
    Nagel, H.H., Enkelman, W.: An investigation of smoothness constrains for estimation of displacement vector fields from image sequences. IEEE Trans. Patt. Anal. Mach. Intell. 8(5), 565–593 (1986)Google Scholar
  34. 34.
    Deriche, R., Kornprobst, P., Aubert, G.: Optical flow estimation while preserving its discontinuities: a variational approach. Lecture Notes in Computer Science, vol. 1035. pp. 71–80 (1996)Google Scholar
  35. 35.
    Yaroslavsky, L.P., Eden, M.: Fundamental of Digital Optics. Birkhäuser, Boston (1996)Google Scholar
  36. 36.
    Stefan W.: Digital Video Quality—Vision Models and Metrics. Wiley, New York (2005)Google Scholar
  37. 37.
    Recommendation H.261 of the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T), Line Transmission of Non-Telephone Signals—Video Codec for Audiovisual Services at p x 64 kb (1993)Google Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  • Barak Fishbain
    • 1
  • Leonid P. Yaroslavsky
    • 1
  • Ianir A. Ideses
    • 1
  1. 1.Department of Interdisciplinary Studies, the Iby and Aladar Fleischman Faculty of EngineeringTel-Aviv UniversityTel AvivIsrael

Personalised recommendations