Pattern Recognition and Tracking in Forward Looking Infrared Imagery

Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 1)


In this chapter, we review the recent trends and advancements on pattern recognition and tracking in forward looking infrared (FLIR) imagery. In particular, we discuss several target detection and tracking algorithms for single/multiple target detection and tracking purposes. Each detection and tracking algorithm utilizes various properties of targets and image frames of a given sequence. At first we discuss a Fukunga–Kuntz Transform and template matching based algorithm for target detection and tracking. Then, we described a novel algorithm for target detection and tracking using fringe-adjusted joint transform correlation (JTC) and template matching. Finally, we discussed an invariant detection and tracking algorithm using a combination of fringe-adjusted JTC and a composited weighted reference function. The impact of sensor ego motion and possible compensation techniques as well as the role of image segmentation towards enhancing the accuracy of target detection and tracking is also described. The aforementioned techniques can detect and track small objects comprising of only a few pixels and is capable of compensating the high ego-motion of the sensor for various challenging scenarios. Test results obtained using real life FLIR image sequences are included to verify the effectiveness of the above mentioned algorithms for target detection and tracking in FLIR imagery.


Pattern recognition Forward-looking infrared imagery Target detection Target tracking Fringe-adjusted JTC Correlation discrimination Synthetic discriminant function Invariant pattern recognition Global motion compensation Subframe segmentation 


  1. 1.
    Alam, M.S., Haque, M., Khan, J., Kettani, H.: Target tracking in forward looking infrared imagery using fringe-adjusted joint transform correlation. Opt. Eng. 43, 1407–1413 (2004)CrossRefGoogle Scholar
  2. 2.
    Longmire, M.S., Takken, E.H.: LMS and matched digital filters for optical clutter suppression. Appl. Opt. 27, 1141–1159 (1988)CrossRefGoogle Scholar
  3. 3.
    Chen, J.Y., Reed, I.S.: A detection algorithm for optical targets in clutter. IEEE Trans. Aerosp. Electron. Syst. 23, 46–59 (1987)CrossRefGoogle Scholar
  4. 4.
    Bal, A., Alam, M.S.: Dynamic target tracking using fringe-adjusted joint transform correlation and template matching. Appl. Opt. 43, 4874–4881 (2004)CrossRefGoogle Scholar
  5. 5.
    Yilmaz, A., Shafique, K., Shah, M.: Target tracking in airborne forward looking infrared imagery. Image Vis. Comput. 21, 623–635 (2003)CrossRefGoogle Scholar
  6. 6.
    Loo, H.C., Alam, M.S.: Invariant object tracking using fringe-adjusted joint transform correlation. J. Opt. Eng. 43, 2175–2183 (2004)CrossRefGoogle Scholar
  7. 7.
    Bharadwaj, P., Carin, L.: Infrared-image classification using hidden Markov trees. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1394–1398 (2002)CrossRefGoogle Scholar
  8. 8.
    Cooper, M.L., Miller, M.I.: Information measures for object recognition accommodating signature variability. IEEE Trans. Inf. Theory 46, 1896–1907 (2000)MATHCrossRefGoogle Scholar
  9. 9.
    Alam, M.S., Bal, A.: Improved multiple target tracking via global motion compensation and optoelectronic correlation. IEEE Trans. Ind. Electron. 54, 522–529 (2007)CrossRefGoogle Scholar
  10. 10.
    Dawoud, A., Alam, M.S., Bal, A., Loo, C.: Decision fusion algorithm for target tracking in infrared imagery. Opt. Eng. 44, 026401(1)–026401(8) (2005)CrossRefGoogle Scholar
  11. 11.
    Lee, M., Kim, Y.: An efficient multitarget tracking algorithm for car applications. IEEE Trans. Ind. Electron. 50, 397–400 (2003)CrossRefGoogle Scholar
  12. 12.
    Nguyen, H.T., Smeulders, A.W.M.: Fast occluded object tracking by a robust appearance filter. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1099–1104 (2004)CrossRefGoogle Scholar
  13. 13.
    Tao, H., Sawhney, H.S., Kumar, R.: Object tracking with Bayesian estimation of dynamic layer representations. IEEE Trans. Pattern Anal. Mach. Intell. 24, 75–89 (2002)CrossRefGoogle Scholar
  14. 14.
    Strehl, A., Aggarwal, J.K.: Detecting moving objects in airborne forward looking infra-red sequences. Mach. Vis. Appl. J. 11, 267–276 (2000)CrossRefGoogle Scholar
  15. 15.
    Mahalanobis, A., Sims, A.R., Nevel, A.V.: Signal-to-clutter measure for measuring automatic target recognition performance using complimentary eigenvalue distribution analysis. Opt. Eng. 42, 1144–1151 (2003)CrossRefGoogle Scholar
  16. 16.
    Mahalanobis, A., Muise, R.R., Stanfill, S.R., Nevel, A.V.: Design and application of quadratic correlation filters for target detection. IEEE Trans. Aerosp. Electron. Syst. 40, 837–850 (2004)CrossRefGoogle Scholar
  17. 17.
    Cheng, F., Yu, F.T.S., Gregory, D.A.: Multitarget detection using spatial synthesis joint transform correlator. Appl. Opt. 32, 6521–6526 (1993)CrossRefGoogle Scholar
  18. 18.
    Alam, M.S., Khan, J., Bal, A.: Hetero associative multiple target tracking using fringe-adjusted joint transform correlation. Appl. Opt. 43, 328–365 (2004)Google Scholar
  19. 19.
    Miller, P.C., Royce, M., Virgo, P., Fiebig, M., Hamlyn, G.: Evaluation of an optical correlator automatic target recognition system for acquisition and tracking in densely cluttered natural scenes. Opt. Eng. 38, 1814–1825 (1999)CrossRefGoogle Scholar
  20. 20.
    Huo, X.: A statistical analysis of Fukunaga–Koontz transform. IEEE Signal Process. Lett. 11, 123–126 (2004)CrossRefGoogle Scholar
  21. 21.
    Castellano, G., Boyce, J., Sandler, M.: Regularized CDWT optical flow applied to moving-target detection in IR imagery. Mach. Vis. Appl. 11, 277–288 (2000)CrossRefGoogle Scholar
  22. 22.
    Dahyot, R., Charbonnier, P., Heitz, F.: Unsupervised statistical detection of changing objects in camera-in-motion video. In: Proceedings of the IEEE International Conference on Image Processing (ICIP’01), Greece, October (2001)Google Scholar
  23. 23.
    Elnagar, A., Basu, A.: Motion detection using background constraints. Pattern Recognition 28, 1537–1554 (1995)CrossRefGoogle Scholar
  24. 24.
    Bruno, M.G.S.: Sequential importance sampling filtering for target tracking in image sequences. IEEE Trans. Ind. Electron. 10, 246–300 (2003)Google Scholar
  25. 25.
    Davies, D., Palmer, P., Mirmehdi, M.: Detection and tracking of very small low contrast objects. In: Ninth British Machine Vision Conference, September 1998 (1998)Google Scholar
  26. 26.
    VanderLugt, A.: Signal detection by complex spatial filtering. IEEE Trans. Inf. Theory 10, 139–145 (1964)Google Scholar
  27. 27.
    Weaver, C.S., Goodman, J.W.: A technique for optically convolving two functions. Appl. Opt. 5, 1246–1249 (1966)CrossRefGoogle Scholar
  28. 28.
    Yu, F.T.S., Lu, X.J.: A real-time programmable joint transform correlator. Opt. Commun. 52, 10–16 (1984)CrossRefGoogle Scholar
  29. 29.
    Alam, M.S., Chain, D.: Efficient multiple target recognition using a wavelet transform processor. Opt. Eng. 39, 1203–1210 (2000)CrossRefGoogle Scholar
  30. 30.
    Alam, M.S., Karim, M.A.: Multiple target detection using a modified fringe-adjusted joint transform correlator. Opt. Eng. 33, 1610–1617 (1994)CrossRefGoogle Scholar
  31. 31.
    Alam, M.S.: Phase-encoded fringe-adjusted joint transform correlation. Opt. Eng. 39, 1169–1176 (2000)CrossRefGoogle Scholar
  32. 32.
    Briechle, K., Hanebeck, U.D.: Template matching using fast normalized cross correlation. In: Optical Pattern Recognition XII, Proceeding of SPIE, vol. 4387, pp. 95–102 (2001)Google Scholar
  33. 33.
    Javidi, B., Kuo, C.: Joint transform image correlation using a binary spatial light modulator at the Fourier plane. Appl. Opt. 27, 663–665 (1988)CrossRefGoogle Scholar
  34. 34.
    Alam, M.S.: Deblurring using fringe-adjusted joint transform correlation. Opt. Eng. 37, 556–564 (1998)CrossRefGoogle Scholar
  35. 35.
    Alam, M.S., Karim, M.A.: Improved correlation discrimination in a multiobject bipolar joint transform correlator. Opt. Laser Tech. 24, 45–50 (1992)CrossRefGoogle Scholar
  36. 36.
    Yu, F.T.S., Cheng, F., Nagata, T., Gregory, D.A.: Effects of fringe binarization of multi-object joint transform correlation. Appl. Opt. 28, 2988–2990 (1989)CrossRefGoogle Scholar
  37. 37.
    Wu, Y., Huang, T.S.: Non-stationary color tracking for vision-based human–computer interaction. IEEE Trans. Neural Network 13, 948–960 (2002)CrossRefGoogle Scholar
  38. 38.
    Oron, E., Kumar, A., Bar-Shalom, Y.: Precision tracking with segmentation for imaging sensor. IEEE Trans. Aerosp. Electron. Syst. 29, 977–987 (1993)CrossRefGoogle Scholar
  39. 39.
    Rastogi, K., Chatterji, B.N., Ray, A.K.: Design of real-time tracking system for fast moving objects. IETE J. Res. 43, 359–369 (1997)Google Scholar
  40. 40.
    Gudmundsson, K., Awwal, A.A.S.: Sub-imaging technique to improve phase only filter search capability. Appl. Opt. 42, 4709–4717 (2003)CrossRefGoogle Scholar
  41. 41.
    Alam, M.S., Bognar, J.G., Hardie, R.C., Yasuda, B.J.: Infrared image registration and high resolution reconstruction using multiple translationally shifted aliased video frames. IEEE Trans. Instrum. Meas. 49, 915–923 (2000)CrossRefGoogle Scholar
  42. 42.
    Bal, A., Alam, M.S.: Automatic target tracking in FLIR image sequences. In: Proceedings of the SPIE Conference on Automatic Target Recognition XIV, vol. 5426, pp. 30–36 (2004)Google Scholar
  43. 43.
    Shekarforoush, H., Chellappa, R.: A multi-fractal formalism for stabilization, object detection and tracking in FLIR sequences. In: IEEE International Conference on Image Processing, vol. 3, pp. 78–81 (2000)Google Scholar
  44. 44.
    Braga-Neto, U., Goutsias, J.: Automatic target detection and tracking in forward-looking infrared image sequences using morphological connected operators. In: 33rd Conference of Information Sciences and Systems, vol. 1, pp. 173–178, March 1999 (1999)Google Scholar
  45. 45.
    Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 142–149 (2000)Google Scholar
  46. 46.
    Feng, D., Zhao, H., Xia, S.: Amplitude-modulated JTC for improving correlation discrimination. Opt. Commun. 86, 260–264 (1991)CrossRefGoogle Scholar
  47. 47.
    Alam, M.S., Karim, M.A.: Fringe-adjusted joint transform correlation. Appl. Opt. 32(23), 4344–4350 (1993)CrossRefGoogle Scholar
  48. 48.
    Alam, M.S., Chen, X.W., Karim, M.A.: Distortion-invariant fringe-adjusted joint transform correlation. Appl. Opt. 36, 7422–7427 (1997)CrossRefGoogle Scholar
  49. 49.
    Grycewicz, T.J.: Applying time modulation to the joint transform correlator. Opt. Eng. 33(6), 1813–1830 (1994)CrossRefGoogle Scholar
  50. 50.
    Casasent, D., Chang, W.T.: Correlation synthetic discriminant functions. Appl. Opt. 25, 2343–2350 (1986)CrossRefGoogle Scholar
  51. 51.
    Wu, Y., Huang, T.S.: Nonstationary color tracking for vision-based human–computer interaction. IEEE Trans. Neural Networks 13, 948–960 (2002)CrossRefGoogle Scholar
  52. 52.
    Alam, M.S., et al.: Fringe-adjusted JTC based target detection and tracking using subframes from a video sequence. In: Proceedings of the SPIE, vol. 5201, pp. 85–96, San Diego, 3–8 August 2003Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringUniversity of South AlabamaMobile, ALUSA

Personalised recommendations