Fusion of Thermal Infrared and Visible Spectrum Video for Robust Surveillance

  • Praveen Kumar
  • Ankush Mittal
  • Padam Kumar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4338)


This paper presents an approach of fusing the information provided by visible spectrum video with that of thermal infrared video to tackle video processing challenges such as object detection and tracking for increasing the performance and robustness of the surveillance system. An enhanced object detection strategy using gradient information along with background subtraction is implemented with efficient fusion based approach to handle typical problems in both the domains. An intelligent fusion approach using Fuzzy logic and Kalman filtering technique is proposed to track objects and obtain fused estimate according to the reliability of the sensors. Appropriate measurement parameters are identified to determine the measurement accuracy of each sensor. Experimental results are shown on some typical scenarios of detection and tracking of pedestrians.


Object Detection Foreground Object Efficient Fusion Thermal Imagery Fuzzy Logic Technique 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Conaire, C., Transfer, O.: report-Phd register: Thermal Infrared and Visible Spectrum Fusion for Multi-modal Video Analysis. Dublin City University, July 28 (2005)Google Scholar
  2. 2.
    Hu, W., Tan, T., Wang, L., Maybank., S.: A survey on visual surveillance of object motion and behaviors. IEEE Transactions on Systems, Man and Cybernetics 34(3), 334–350 (2004)CrossRefGoogle Scholar
  3. 3.
    Cucchiara., R.: Multimedia surveillance systems. In: VSSN 2005: Proceedings of the third ACM international workshop on Video surveillance & sensor networks, New York, pp. 3–10 (2005)Google Scholar
  4. 4.
    Bhanu., B., Han., J.: Kinematic-based human motion analysis in infrared sequences. In: Proceedings Workshop Applications of Computer Vision, pp. 208–212 (2002)Google Scholar
  5. 5.
    Davis, J., Sharma, V.: Robust detection of people in thermal imagery. In: Proceedings of International Conference on Pattern Recognition, pp. 713–716 (2004)Google Scholar
  6. 6.
    McDaniel, R., Scribner, D., Krebs, W., Warren, P., Ockman, N., McCarley, J.: Image fusion for tactical applications. In: Proceedings of the SPIE - Infrared Technology and Applications XXIV, vol. 3436, pp. 685–695 (1998)Google Scholar
  7. 7.
    Torresan, H., Turgeon, B., Ibarra-Castanedo, C., Hébert, P., Maldague, X.: Advanced Surveillance Systems: Combining Video and Thermal Imagery for Pedestrian Detection. In: Proceedings of SPIE, Thermosense XXVI, SPIE, vol. 5405, pp. 506–515 (2004)Google Scholar
  8. 8.
    Davis, J., Sharma, V.: Fusion-Based Background-Subtraction using Contour Saliency. Computer Vision and Pattern Recognition, 20–26 (June 2005)Google Scholar
  9. 9.
    Snidaro, L., Niu, R., Varshney, P.K., Foresti, G.L.: Automatic camera selection and fusion for outdoor surveillance under changing weather conditions. In: IEEE Conference on Advanced Video and Signal based Surveillance, Florida, pp. 364–370 (2003)Google Scholar
  10. 10.
    Escamilla-Ambrosio, P.J., Mort, N.: A Hybrid Kalman Filter - Fuzzy Logic Architecture for Multisensor Data Fusion. In: Proceedings of the 2001 IEEE International Symposium on Intelligent Control, pp. 364–369 (2001)Google Scholar
  11. 11.
    Regazzoni, C., Ramesh, V., Foresti, G.L.: Special issue on video communications, processing, and understanding for third generation surveillance systems. Proceedings of the IEEE 89(10) (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Praveen Kumar
    • 1
  • Ankush Mittal
    • 1
  • Padam Kumar
    • 1
  1. 1.Department of Electronics and Computer EngineeringIndian Institute of TechnologyRoorkeeIndia

Personalised recommendations