Skip to main content

Toward a sentient environment: real-time wide area multiple human tracking with identities

Abstract

In this paper, we presented a fully integratedreal-time computer vision system that can detect and track multiple humans in a wide-area using a network of stereo cameras. Continuous human identities are achieved by fusing video tracking with different kinds of biometric devices. The system also provides immersive visualization which enables the users to conveniently navigate through space and time and query useful events. The key innovations include stereo-based multi-object detection and tracking, a unified approach for fusing multiple sensors of different modalities, visualization and user interface design.

This is a preview of subscription content, access via your institution.

References

  1. Haritaoglu, S., Harwood, D., Davis, L.S.: W4: Real-time surveillance of people and their activities. IEEE Trans. Pattern Recogn. Mach. Intell. 22(8) (2000)

  2. Isard, M., MacCormick, J.: Bramble: a bayesian multiple-blob tracker. In: Proceedings of the International Conference On Computer Vision (2001)

  3. Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. Washington DC, pp. II:406–413 (2004)

  4. Ting, Yu., Ying, Wu.: Collaborative tracking of multiple targets. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. Washington DC, pp. I:834–841 (2004)

  5. Tao H., Sawhney H.S., Kumar R. (2002) Object tracking with bayesian estimation of dynamic layer representations. IEEE Trans. Pattern Recogn. Mach. Intell. 24(1): 75–89

    Article  Google Scholar 

  6. Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust online appearance models for visual tracking. IEEE Trans. Pattern Recogn. Mach. Intell. 25(10) (2003)

  7. Darrell T., Gordon G., Harville M., Woodfill J. (2000) Integrated person tracking using stereo, color, and pattern detection. Int. J. Comput. Vis. 37(2): 175–185

    MATH  Article  Google Scholar 

  8. Beymer, D.: Person counting using stereo. In: Proceedings IEEE Workshop on Human Motion (2000)

  9. Harville, M., Li, D.: Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2004)

  10. Collins, R., Lipton, A., Fujiyoshi, H., Kanade, T.: Algorithms for cooperative multi-sensor surveillance. In: Proceedings of IEEE, 89, p. 1456 (2001)

  11. Khan, S., Shah, M.: Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. IEEE Trans. Pattern Recogn. Mach. Intell. 25(10) (2003)

  12. Cupillard, F., Bremond, F., Thonnat, M.: Group behavior recognition with multiple cameras. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (2002)

  13. Pasula, H., Russell, S., Ostland, M., Ritov, Y.: Tracking many objects with many sensors. In: Proceedings of the International Joint Conference on Artificial Intelligence (1999)

  14. Kettnaker, V., Zabih, R.: Bayesian multi-camera surveillance. In: Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (1999)

  15. Javed, O., Rasheed, Z., Shafique, K., Shah, M.: Tracking across multiple cameras with disjoint views. In: Proceedings of the International Conference on Computer Vision (2003)

  16. Haralick, R.M.: Propagating covariance in computer vision. In: Proceedings of the Workshop on Performance Characteristics of Vision Algorithms (1996)

  17. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Recogn. Mach. Intell. 24(5) (2002)

  18. Mittal, A., Davis, L.S.: M2tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int. J. Comput. Vis. 51(3) (2003)

  19. Isard M., Blake A. (1998) Condensation-conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1): 5–28

    Article  Google Scholar 

  20. Dempster A.P., Laird N.M., Rubin D.B. (1977) Maximum likelihood from incomplete data via the em algorithm. J R. Stat. Soc. B 39(1): 1–38

    MATH  MathSciNet  Google Scholar 

  21. Kalman R. (1960) A new approach to linear filtering and prediction problems. J. Basic Eng. 82: 35–45

    Google Scholar 

  22. Goodwin and Sin: Adaptive Filtering, Prediction, and Control. Prentice Hall, Englewood Cliffs (1984)

  23. Chris Stauffer.: Estimating tracking sources and sinks. In: Proceedings IEEE Workshop on Event Mining (2003)

  24. LG Electronics USA, Inc.: LG IrisAccess(TM) 3000 system

  25. Identix Inc.: FaceIt (R) ARGUS system

  26. Duda O., Hart P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2000)

    Google Scholar 

  27. Sawnhey, H.S., Arpa, A., Kumar, R., Samarasekera, S., Aggarwal, M., Hsu, S.C., Nister, D., Hanna, K.J.: Video flashlights: real time rendering of multiple videos for immersive model visualization. In: Rendering Techniques, Pisa, Italy, pp. 157–168 (2002)

  28. Multispectral Solutions, Inc.: Sapphire DART UWB-based Precision Asset Location System(TM)

  29. Hartley, R., Zisserman A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tao Zhao.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Zhao, T., Aggarwal, M., Germano, T. et al. Toward a sentient environment: real-time wide area multiple human tracking with identities. Machine Vision and Applications 19, 301–314 (2008). https://doi.org/10.1007/s00138-008-0154-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-008-0154-y

Keywords

  • Video surveillance
  • Visual tracking
  • Stereo vision
  • Camera network
  • Human activity recognition
  • Surveillance system