Skip to main content

Toward a sentient environment: real-time wide area multiple human tracking with identities

Machine Vision and Applications volume 19, pages 301–314 (2008)Cite this article

Abstract

In this paper, we presented a fully integratedreal-time computer vision system that can detect and track multiple humans in a wide-area using a network of stereo cameras. Continuous human identities are achieved by fusing video tracking with different kinds of biometric devices. The system also provides immersive visualization which enables the users to conveniently navigate through space and time and query useful events. The key innovations include stereo-based multi-object detection and tracking, a unified approach for fusing multiple sensors of different modalities, visualization and user interface design.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Haritaoglu, S., Harwood, D., Davis, L.S.: W4: Real-time surveillance of people and their activities. IEEE Trans. Pattern Recogn. Mach. Intell. 22(8) (2000)

  2. Isard, M., MacCormick, J.: Bramble: a bayesian multiple-blob tracker. In: Proceedings of the International Conference On Computer Vision (2001)

  3. Zhao, T., Nevatia, R.: Tracking multiple humans in crowded environment. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. Washington DC, pp. II:406–413 (2004)

  4. Ting, Yu., Ying, Wu.: Collaborative tracking of multiple targets. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. Washington DC, pp. I:834–841 (2004)

  5. Tao H., Sawhney H.S., Kumar R. (2002) Object tracking with bayesian estimation of dynamic layer representations. IEEE Trans. Pattern Recogn. Mach. Intell. 24(1): 75–89

    Article  Google Scholar 

  6. Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust online appearance models for visual tracking. IEEE Trans. Pattern Recogn. Mach. Intell. 25(10) (2003)

  7. Darrell T., Gordon G., Harville M., Woodfill J. (2000) Integrated person tracking using stereo, color, and pattern detection. Int. J. Comput. Vis. 37(2): 175–185

    MATH  Article  Google Scholar 

  8. Beymer, D.: Person counting using stereo. In: Proceedings IEEE Workshop on Human Motion (2000)

  9. Harville, M., Li, D.: Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2004)

  10. Collins, R., Lipton, A., Fujiyoshi, H., Kanade, T.: Algorithms for cooperative multi-sensor surveillance. In: Proceedings of IEEE, 89, p. 1456 (2001)

  11. Khan, S., Shah, M.: Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. IEEE Trans. Pattern Recogn. Mach. Intell. 25(10) (2003)

  12. Cupillard, F., Bremond, F., Thonnat, M.: Group behavior recognition with multiple cameras. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (2002)

  13. Pasula, H., Russell, S., Ostland, M., Ritov, Y.: Tracking many objects with many sensors. In: Proceedings of the International Joint Conference on Artificial Intelligence (1999)

  14. Kettnaker, V., Zabih, R.: Bayesian multi-camera surveillance. In: Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (1999)

  15. Javed, O., Rasheed, Z., Shafique, K., Shah, M.: Tracking across multiple cameras with disjoint views. In: Proceedings of the International Conference on Computer Vision (2003)

  16. Haralick, R.M.: Propagating covariance in computer vision. In: Proceedings of the Workshop on Performance Characteristics of Vision Algorithms (1996)

  17. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Recogn. Mach. Intell. 24(5) (2002)

  18. Mittal, A., Davis, L.S.: M2tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int. J. Comput. Vis. 51(3) (2003)

  19. Isard M., Blake A. (1998) Condensation-conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1): 5–28

    Article  Google Scholar 

  20. Dempster A.P., Laird N.M., Rubin D.B. (1977) Maximum likelihood from incomplete data via the em algorithm. J R. Stat. Soc. B 39(1): 1–38

    MATH  MathSciNet  Google Scholar 

  21. Kalman R. (1960) A new approach to linear filtering and prediction problems. J. Basic Eng. 82: 35–45

    Google Scholar 

  22. Goodwin and Sin: Adaptive Filtering, Prediction, and Control. Prentice Hall, Englewood Cliffs (1984)

  23. Chris Stauffer.: Estimating tracking sources and sinks. In: Proceedings IEEE Workshop on Event Mining (2003)

  24. LG Electronics USA, Inc.: LG IrisAccess(TM) 3000 system

  25. Identix Inc.: FaceIt (R) ARGUS system

  26. Duda O., Hart P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2000)

    Google Scholar 

  27. Sawnhey, H.S., Arpa, A., Kumar, R., Samarasekera, S., Aggarwal, M., Hsu, S.C., Nister, D., Hanna, K.J.: Video flashlights: real time rendering of multiple videos for immersive model visualization. In: Rendering Techniques, Pisa, Italy, pp. 157–168 (2002)

  28. Multispectral Solutions, Inc.: Sapphire DART UWB-based Precision Asset Location System(TM)

  29. Hartley, R., Zisserman A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)

Download references

Author information

Authors and Affiliations

  1. Sarnoff Corporation, 201 Washington Rd, Princeton, NJ, 08543, USA

    Tao Zhao, Manoj Aggarwal, Thomas Germano, Ian Roth, Alexandar Knowles, Rakesh Kumar, Harpreet Sawhney & Supun Samarasekera

Authors
  1. Tao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manoj Aggarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Germano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ian Roth
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexandar Knowles
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rakesh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Harpreet Sawhney
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Supun Samarasekera
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Zhao.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Zhao, T., Aggarwal, M., Germano, T. et al. Toward a sentient environment: real-time wide area multiple human tracking with identities. Machine Vision and Applications 19, 301–314 (2008). https://doi.org/10.1007/s00138-008-0154-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-008-0154-y

Keywords

  • Video surveillance
  • Visual tracking
  • Stereo vision
  • Camera network
  • Human activity recognition
  • Surveillance system