Moving Object Detection and Tracking in Forward Looking Infra-Red Aerial Imagery

  • Subhabrata Bhattacharya
  • Haroon Idrees
  • Imran Saleemi
  • Saad Ali
  • Mubarak Shah
Part of the Augmented Vision and Reality book series (Augment Vis Real, volume 1)


This chapter discusses the challenges of automating surveillance and reconnaissance tasks for infra-red visual data obtained from aerial platforms. These problems have gained significant importance over the years, especially with the advent of lightweight and reliable imaging devices. Detection and tracking of objects of interest has traditionally been an area of interest in the computer vision literature. These tasks are rendered especially challenging in aerial sequences of infra red modality. The chapter gives an overview of these problems, and the associated limitations of some of the conventional techniques typically employed for these applications. We begin with a study of various image registration techniques that are required to eliminate motion induced by the motion of the aerial sensor. Next, we present a technique for detecting moving objects from the ego-motion compensated input sequence. Finally, we describe a methodology for tracking already detected objects using their motion history. We substantiate our claims with results on a wide range of aerial video sequences.


Aerial image registration Object detection Tracking 


  1. 1.
    Ali, S., Shah, M.: Cocoa—tracking in aerial imagery. In: SPIE Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications (2006)Google Scholar
  2. 2.
    Andriluka, M., Roth, S., Schiele, B.: People-tracking-by-detection and people-detection-by-tracking. In: CVPR (2008)Google Scholar
  3. 3.
    Arambel, P., Antone, M., Landau, R.H.: A multiple-hypothesis tracking of multiple ground targets from aerial video with dynamic sensor control. In: Proceedings of SPIE, Signal Processing, Sensor Fusion, and Target Recognition XIII, vol. 5429, pp. 23–32 (2004)Google Scholar
  4. 4.
    Bay, H., Tuytelaars, T., Gool, L.V.: Surf: speeded up robust features. In: ECCV (2006)Google Scholar
  5. 5.
    Berclaz, J., Fleuret, F., Fua, P.: Robust people tracking with global trajectory optimization. In: CVPR (2006)Google Scholar
  6. 6.
    Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple object tracking performance metrics and evaluation in a smart room environment (2006)Google Scholar
  7. 7.
    Blackman, S., Popoli, R.: Design and Analysis of Modern Tracking Systems. Artech House, Boston (1999)MATHGoogle Scholar
  8. 8.
    Bouguet, J.: Pyramidal implementation of the Lucas–Kanade feature tracker: description of the algorithm. TR, Intel Microprocessor Research Labs (2000)Google Scholar
  9. 9.
    Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)CrossRefGoogle Scholar
  10. 10.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)Google Scholar
  11. 11.
    Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Gandhi, T., Devadiga, S., Kasturi, R., Camps, O.: Detection of obstacles on runway using ego-motion compensation and tracking of significant features. In: Proceedings of the 3rd IEEE Workshop on Applications of Computer Vision, p. 168Google Scholar
  13. 13.
    Heitz, G., Koller, D.: Learning spatial context: using stuff to find things. In: ECCV (2008)Google Scholar
  14. 14.
    Isard, M., Blake, A.: Condensation: conditional density propagation for visual tracking. In: IJCV (1998)Google Scholar
  15. 15.
    Jepson, A., Fleet, D., El-Maraghi, T.: Robust online appearance models for visual tracking. In: IEEE TPAMI (2003)Google Scholar
  16. 16.
    Kumar, R., Sawhney, H., Samarasekera, S., Hsu, S., Tao, H., Guo, Y., Hanna, K., Pope, A., Wildes, R., Hirvonen, D., Hansen, M., Burt, P.: Aerial video surveillance and exploitation. IEEE Proc. 89, 1518–1539 (2001)CrossRefGoogle Scholar
  17. 17.
    Leibe, B., Schindler, K., Gool, L.V.: Coupled detection and trajectory estimation for multi-object tracking. In: ICCV (2007)Google Scholar
  18. 18.
    Lin, R., Cao, X., Xu, Y., Wu, C., Qiao, H.: Airborne moving vehicle detection for video surveillance of urban traffic. In: IEEE Intelligent Vehicles Symposium, pp. 203–208 (2009)Google Scholar
  19. 19.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  20. 20.
    Mann, S., Picard, R.W.: Video orbits of the projective group: a simple approach to featureless estimation of parameters. IEEE Trans. Image Process. 6, 1281–1295 (1997)CrossRefGoogle Scholar
  21. 21.
    Matas, J., Chum, O., Martin, U., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: BMVC (2002)Google Scholar
  22. 22.
    Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: VISAPP (2009)Google Scholar
  23. 23.
    Olson, C.F., Huttenlocher, D.P.: Automatic target recognition by matching oriented edge pixels. IEEE Trans. Image Process. 6, 103–113 (1997)CrossRefGoogle Scholar
  24. 24.
    Perera, A., Srinivas, C., Hoogs, A., Brooksby, G., Hu, W.: Multi-object tracking through simultaneous long occlusions and split–merge conditions. In: CVPR (2006)Google Scholar
  25. 25.
    Piccardi, M.: Background subtraction techniques: a review. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3099–3104 (2004)Google Scholar
  26. 26.
    Shah, M., Kumar, R.: Video Registration. Kluwer Academic Publishers, Dordrecht (2003)MATHGoogle Scholar
  27. 27.
    Shi, J., Tomasi, C.: Good features to track. In: CVPR, pp. 593–600 (1994)Google Scholar
  28. 28.
    Spencer, L., Shah, M.: Temporal synchronization from camera motion. In: ACCV (2004)Google Scholar
  29. 29.
    Xiao, J., Cheng, H., Han, F., Sawhney, H.: Geo-spatial aerial video processing for scene understanding. In: CVPR (2008)Google Scholar
  30. 30.
    Xiao, J., Yang, C., Han, F., Cheng, H.: Vehicle and person tracking in aerial videos. In: Multimodal Technologies for Perception of Humans: International Evaluation Workshops CLEAR 2007 and RT 2007, pp. 203–214 (2008)Google Scholar
  31. 31.
    Yalcin, H., Collins, R., Black, M., Hebert, M.: A flow-based approach to vehicle detection and background mosaicking in airborne video. In: CVPR, p. 1202 (2005)Google Scholar
  32. 32.
    Yalcin, H., Collins, R., Hebert, M.: Background estimation under rapid gain change in thermal imagery. In: OTCBVS (2005)Google Scholar
  33. 33.
    Yilmaz, A.: Target tracking in airborne forward looking infrared imagery. Image Vis. Comput. 21(7), 623–635 (2003)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), 1–45 (2006)CrossRefGoogle Scholar
  35. 35.
    Yilmaz, A., Shafique, K., Lobo, N., Li, X., Olson, T., Shah, M.A.: Target-tracking in flir imagery using mean-shift and global motion compensation. In: Workshop on Computer Vision Beyond the Visible Spectrum, pp. 54–58 (2001)Google Scholar
  36. 36.
    Yin, Z., Collins, R.: Moving object localization in thermal imagery by forward–backward mhi. In: OTCBVS (2006)Google Scholar
  37. 37.
    Yuan, C., Medioni, G., Kang, J., Cohen, I.: Detecting motion regions in presence of strong parallax from a moving camera by multi-view geometric constraints. IEEE TPAMI 29, 1627–1641 (2007)Google Scholar
  38. 38.
    Zhang, H., Yuan, F.: Vehicle tracking based on image alignment in aerial videos. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, vol. 4679, pp. 295–302 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Subhabrata Bhattacharya
    • 1
  • Haroon Idrees
    • 1
  • Imran Saleemi
    • 1
  • Saad Ali
    • 2
  • Mubarak Shah
    • 1
  1. 1.University of Central FloridaFLUSA
  2. 2.Sarnoff CorporationPrincetonUSA

Personalised recommendations