Exploiting Pedestrian Interaction via Global Optimization and Social Behaviors

  • Laura Leal-Taixé
  • Gerard Pons-Moll
  • Bodo Rosenhahn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7474)


Multiple people tracking consists in detecting the subjects at each frame and matching these detections to obtain full trajectories. In semi-crowded environments, pedestrians often occlude each other, making tracking a challenging task. Tracking methods mostly work with the assumption that each pedestrian moves independently unaware of the objects or the other pedestrians around it. In the real world though, it is clear that when walking in a crowd, pedestrians try to avoid collisions, keep a close distance to a group of friends or avoid static obstacles in the scene.

In this paper, we present an approach which includes the interaction between pedestrians in two ways: first, including social and grouping behavior as a physical model within the tracking system, and second, using a global optimization scheme which takes into account all trajectories and all frames to solve the data association problem . Results are presented on three challenging publicly available datasets, showing our method outperforms state-of-the-art tracking systems. We also make a thorough analysis of the effect of the parameters of the proposed tracker as well as its robustness against noise, outliers and missing data.


Group Behavior Tracking Accuracy Multiple Object Tracking Track Precision Crowd Simulation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gall, J., Yao, A., Razavi, N., van Gool, L., Lempitsky, V.: Hough forests for object detection, tracking, and action recognition. TPAMI (2011)Google Scholar
  2. 2.
    Breitenstein, M., Reichlin, F., Leibe, B., Koller-Meier, E., van Gool, L.: Robust tracking-by-detection using a detector confidence particle filter. In: ICCV (2009)Google Scholar
  3. 3.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet part detectors. IJCV 75(2) (2007)Google Scholar
  4. 4.
    Leibe, B., Schindler, K., Cornelis, N., van Gool, L.: Coupled detection and tracking from static cameras and moving vehicles. TPAMI 30(10) (2008)Google Scholar
  5. 5.
    Kaucic, R., Perera, A., Brooksby, G., Kaufhold, J., Hoogs, A.: A unified framework for tracking through occlusions and across sensor gaps. In: CVPR (2005)Google Scholar
  6. 6.
    Ali, S., Shah, M.: Floor Fields for Tracking in High Density Crowd Scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 1–14. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Rodriguez, M., Sivic, J., Laptev, I., Audibert, J.: Data-driven crowd analysis in videos. In: ICCV (2011)Google Scholar
  8. 8.
    Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: CVPR (2008)Google Scholar
  9. 9.
    Berclaz, J., Fleuret, F., Türetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. TPAMI (2011)Google Scholar
  10. 10.
    Pellegrini, S., Ess, A., Schindler, K., van Gool, L.: You’ll never walk alone: modeling social behavior for multi-target tracking. In: ICCV (2009)Google Scholar
  11. 11.
    Yamaguchi, K., Berg, A., Ortiz, L., Berg, T.: Who are you with and where are you going? In: CVPR (2011)Google Scholar
  12. 12.
    Leal-Taixé, L., Pons-Moll, G., Rosenhahn, B.: Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker. In: ICCV Workshops, 1st Workshop on Modeling, Simulation and Visual Analysis of Large Crowds (2011)Google Scholar
  13. 13.
    Khan, Z., Balch, T., Dellaert, F.: Mcmc-based particle filtering for tracking a variable number of interacting targets. TPAMI (2005)Google Scholar
  14. 14.
    Leal-Taixé, L., Heydt, M., Rosenhahn, A., Rosenhahn, B.: Automatic tracking of swimming microorganisms in 4d digital in-line holography data. In: IEEE Workshop on Motion and Video Computing, WMVC (2009)Google Scholar
  15. 15.
    Nillius, P., Sullivan, J., Carlsson, S.: Multi-target tracking - linking identities using bayesian network inference. In: CVPR (2006)Google Scholar
  16. 16.
    Yang, M., Yu, T., Wu, Y.: Game-theoretic multiple target tracking. In: ICCV (2007)Google Scholar
  17. 17.
    Berclaz, J., Fleuret, F., Fua, P.: Robust people tracking with global trajectory optimization. In: CVPR (2006)Google Scholar
  18. 18.
    Jiang, H., Fels, S., Little, J.: A linear programming approach for multiple object tracking. In: CVPR (2007)Google Scholar
  19. 19.
    Andriyenko, A., Schindler, K.: Globally Optimal Multi-target Tracking on a Hexagonal Lattice. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 466–479. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. 20.
    Wu, Z., Kunz, T., Betke, M.: Efficient track linking methods for track graphs using network-flow and set-cover techniques. In: CVPR (2011)Google Scholar
  21. 21.
    Helbing, D., Molnár, P.: Social force model for pedestrian dynamics. Physical Review E 51, 4282 (1995)CrossRefGoogle Scholar
  22. 22.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: CVPR (2009)Google Scholar
  23. 23.
    Pelechano, N., Allbeck, J., Badler, N.: Controlling individual agents in high-density crowd simulation. In: Eurographics/ACM SIGGRAPH Symposium on Computer Animation (2007)Google Scholar
  24. 24.
    Scovanner, P., Tappen, M.: Learning pedestrian dynamics from the real world. In: ICCV (2009)Google Scholar
  25. 25.
    Luber, M., Stork, J., Tipaldi, G., Arras, K.: People tracking with human motion predictions from social forces. In: ICRA (2010)Google Scholar
  26. 26.
    Ge, W., Collins, R., Ruback, B.: Automatically detecting the small group structure of a crowd. In: WACV (2009)Google Scholar
  27. 27.
    Choi, W., Savarese, S.: Multiple Target Tracking in World Coordinate with Single, Minimally Calibrated Camera. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 553–567. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  28. 28.
    Pellegrini, S., Ess, A., Van Gool, L.: Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 452–465. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  29. 29.
    Bertsekas, D.: Nonlinear programming. Athena Scientific (1999)Google Scholar
  30. 30.
    Dantzig, G.: Linear programming and extensions. Princeton University Press, Princenton (1963)zbMATHGoogle Scholar
  31. 31.
    Makhorin, A.: Gnu linear programming kit (glpk) (2010),
  32. 32.
    Pirsiavash, H., Ramanan, D., Fowlkes, C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR (2011)Google Scholar
  33. 33.
    Ahuja, R., Magnanti, T., Orlin, J.: Network flows: Theory, algorithms and applications. Prentice Hall (1993)Google Scholar
  34. 34.
    Suurballe, J.: Disjoint paths in a network. Networks 4, 125–145 (1974)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Kasturi, R., Goldgof, D., Soundararajan, P., Manohar, V., Garofolo, J., Boonstra, M., Korzhova, V., Zhang, J.: Framework for performance evaluation for face, text and vehicle detection and tracking in video: data, metrics, and protocol. TPAMI 31(2) (2009)Google Scholar
  36. 36.
    Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: CVPR (2011)Google Scholar
  37. 37.
    Ferryman, J.: Pets 2009 dataset: Performance and evaluation of tracking and surveillance (2009)Google Scholar
  38. 38.
    Brostow, G., Cipolla, R.: Unsupervised detection of independent motion in crowds. In: CVPR (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Laura Leal-Taixé
    • 1
  • Gerard Pons-Moll
    • 1
  • Bodo Rosenhahn
    • 1
  1. 1.Leibniz Universität HannoverHannoverGermany

Personalised recommendations