Robust Tracking with and Beyond Visible Spectrum: A Four-Layer Data Fusion Framework

  • Jianru Xue
  • Nanning Zheng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4153)


Developing robust visual tracking algorithms for real-world applications is still a major challenge today. In this paper,we focus on robust object tracking with multiple spectrum imaging sensors. We propose a four-layer probabilistic fusion framework for visual tracking with and beyond visible spectrum imaging sensors. The framework consists of four different layers of a bottom-up fusion process. These four layers are defined as: visual cues layer fusing visual modalities via an adaptive fusion strategy, models layer fusing prior motion information via interactive multi-model method(IMM), trackers layer fusing results from multiple trackers via adaptive tracking mode switching, and sensors layer fusing multiple sensors in a distributed way. It requires only state distributions in the input and output of each layer to ensure consistency of so many visual modules within the framework. Furthermore, the proposed framework is general and allows augmenting and pruning of fusing layers according to visual environment at hand. We test the proposed framework in various complex scenarios where a single sensor based tracker may fail, and obtain satisfying tracking results.


Tracking Algorithm Data Fusion Product Rule Visual Tracking Robust Tracking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bhanu, B., Pavlidis, I.: Computer Vision Beyond the Visible Spectrum. Springer, Heidelberg (2004)Google Scholar
  2. 2.
    Dai, C., Zheng, Y., Li, X.: Layered Representation for Pedestrian Detection and Tracking in Infrared Imagery. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 3 (2005)Google Scholar
  3. 3.
    Perez, P., Vermaak, J., Blake, A.: Data fusion for visual tracking with particles. Proceedings of the IEEE 92(3), 495–513 (2004)CrossRefGoogle Scholar
  4. 4.
    Li, X., Jilkov, V.: Survey of Maneuvering Target Tracking. Part V: Multiple-Model Methods. IEEE Transactions on Aerospace and Electronic Systems 41(4), 1255 (2005)CrossRefGoogle Scholar
  5. 5.
    Liu, J., Chen, R.: Sequential Monte Carlo Methods for Dynamic Systems. Journal of the American Statistical Association 93(443), 1032–1044 (1998)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Isard, M., Blake, A.: CONDENSATION Conditional Density Propagation for Visual Tracking. International Journal of Computer Vision 29(1), 5–28 (1998)CrossRefGoogle Scholar
  7. 7.
    Wu, Y., Huang, T.: A co-inference approach to robust visual tracking. In: Proc. Int. Conf. Computer Vision (2001)Google Scholar
  8. 8.
    Vermaak, J., Pérez, P., Gangnet, M., Blake, A.: Towards improved observation models for visual tracking: Selective adaptation. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 645–660. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Leichter, I., Lindenbaum, M., Rivlin, E.: A probabilistic framework for combining tracking algorithms. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004) (2) (2004)Google Scholar
  10. 10.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(5), 603–619 (2002)CrossRefGoogle Scholar
  11. 11.
    Jacobs, R.: What determines visual cue reliability. Trends Cogn. Sci. 6, 345–350 (2002)CrossRefGoogle Scholar
  12. 12.
    Bar-Shalom, Y., Li, X., Kirubarajan, T.: Estimation with applications to tracking and navigation. Wiley, New York (2001)CrossRefGoogle Scholar
  13. 13.
    Sung, S., Chien, S., Kim, M., Kim, J.: Adaptive window algorithm with four-direction sizing factors for robust correlation-based tracking. In: Proceedings of Ninth IEEE International Conference on Tools with Artificial Intelligence 1997, pp. 208–215 (1997)Google Scholar
  14. 14.
    Bar-Shalom, Y., Li, X.: Multitarget-multisensor tracking: Principles and techniques. Storrs, CT: University of Connecticut, 1995 (1995)Google Scholar
  15. 15.
    Hall, D., McMullen, S.: Mathematical Techniques in Multisensor Data Fusion. Artech House Publishers (2004)Google Scholar
  16. 16.
    Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-based probabilistic tracking. In: Proceedings of Europe Conference on Computer Vision, 661–675 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jianru Xue
    • 1
  • Nanning Zheng
    • 1
  1. 1.Institute of Artificial Intelligence and RoboticsXi’an Jiaotong UniversityXi’an, ShaanxiChina

Personalised recommendations