International Journal of Computer Vision

, Volume 69, Issue 2, pp 159–180 | Cite as

Behavioral Priors for Detection and Tracking of Pedestrians in Video Sequences

  • Gianluca AntoniniEmail author
  • Santiago Venegas Martinez
  • Michel Bierlaire
  • Jean Philippe Thiran


In this paper we address the problem of detection and tracking of pedestrians in complex scenarios. The inclusion of prior knowledge is more and more crucial in scene analysis to guarantee flexibility and robustness, necessary to have reliability in complex scenes. We aim to combine image processing methods with behavioral models of pedestrian dynamics, calibrated on real data. We introduce Discrete Choice Models (DCM) for pedestrian behavior and we discuss their integration in a detection and tracking context. The obtained results show how it is possible to combine both methodologies to improve the performances of such systems in complex sequences.


Image Processing Artificial Intelligence Pattern Recognition Computer Vision Prior Knowledge 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. AlGadhi, S.A.H., Mahmassani, H., and Herman, R. 2002. A speed-concentration relation for bi-directional crowd movements. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 3–20.Google Scholar
  2. Antonini, G., Bierlaire, M., and Weber, M. 2004. Discrete choice models of pedestrian walking behavior. Accepted for publications in Transportation Research Part B.Google Scholar
  3. Antonini, G. and Thiran, J.P. 2004. Trajectories clustering in ica space: an application to automatic counting of pedestrians in video sequences. In Advanced Concepts for Intelligent Vision Systems (ACIVS), J. Blanc-Talon and D. Popescu (eds.), Brussels, Belgium.Google Scholar
  4. Antonini, G., Venegas, S., Thiran, J.P., and Bierlaire, M. 2004. A discrete choice pedestrian behavior model for pedestrian detection in visual tracking systems. In Advanced Concepts for Intelligent Vision Systems (ACIVS), J. Blanc-Talon and D. Popescu (eds.), Brussels, Belgium.Google Scholar
  5. Arulampalam, M.S., Maskell, S., Gordon, N., and Clapp, T. 2002. A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. on Signal Processing, 50(2):174–188.CrossRefGoogle Scholar
  6. Ben-Akiva, M. and Bierlaire, M. 1999. Discrete choice methods and their applications to short-term travel decisions. In Handbook of Transportation Science, R. Hall (ed.), Kluwer, pp. 5–34.Google Scholar
  7. Ben-Akiva, M.E., Bergman, M.J., Daly, A.J., and Ramaswamy, R. 1984. Modeling inter-urban route choice behaviour. In Proceedings from the Ninth International Symposium on Transportation and Traffic Theory, J. Volmuller and R. Hamerslag (eds.), VNU Science Press: Utrecht, Netherlands, pp. 299–330.Google Scholar
  8. Ben-Akiva, M.E. and Lerman, S.R. 1985. Discrete Choice Analysis: Theory and Application to Travel Demand. MIT Press: Cambridge, MA.Google Scholar
  9. Bierlaire, M. 2001. A theoretical analysis of the cross-nested logit model. Accepted for publications in Annals of Operations Research.Google Scholar
  10. Bierlaire, M. 2002. The network GEV model. In Proceedings of the 2nd Swiss Transportation Research Conference, Ascona, Switzerland,
  11. Bierlaire, M. 2003. An introduction to BIOGEME Version 0.6, February Scholar
  12. Bierlaire, M., Antonini, G., and Weber, M. 2003. Behavioral dynamics for pedestrians. In Moving Through Nets: The Physical and Social Dimensions of Travel, K. Axhausen (ed.), Elsevier, pp. 1–18.Google Scholar
  13. Biliotti, D., Antonini, G., and Thiran, J.P. 2005. Multi-layer trajectories clustering for automatic counting of pedestrians in video sequences. In IEEE Motion 2005, IEEE Computer Society.Google Scholar
  14. Blue, V.J. and Adler, J.L. 2001. Cellular automata microsimulation for modeling bi-directional pedestrian walkways. Transportation Research B, 35(3):293–312.CrossRefGoogle Scholar
  15. Borgers, A. and Timmermans, H. 1986a. A model of pedestrian route choice and demand for retail facilities within inner-city shopping areas. Geographical Analysis, 18(2):115–128.CrossRefGoogle Scholar
  16. Borgers, A. and Timmermans, H. 1986b. City centre entry points, store location patterns and pedestrian route choice behaviour: A micro-level simulation model. Socio-Economie Planning Sciences, 20(1):25–31.CrossRefGoogle Scholar
  17. Bregler, C. 1997. Learning and recognizing human dynamics in video sequences. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.Google Scholar
  18. Campbell, L.W. and Bobick, A.F. 1995. Recognition of human body motion using phase space constraints. In International Conference in Computer Vision (ICCV), pp. 624–630.Google Scholar
  19. Canny, J.F. 1986. A computational approach to edge detection. IEEE Trans. Patt. Anal. Mach. Intell., 8:679–698.CrossRefGoogle Scholar
  20. Cascetta, E., Nuzzolo, A., and Biggiero, L. 1992. Analysis and modeling of commuters’ departure time and route choice in urban networks. In Proceedings of the Second International Capri Seminar on Urban Traffic Networks.Google Scholar
  21. Cohen, L.D. and Cohen, I. 1900. A finite element method applied to new active contour models and 3d reconstruction from cross sections. In International Conference on Computer Vision (ICCV).Google Scholar
  22. Collins, R., Lipton, A., Fujiyoshi, H., and Kanade, T. 2001. Algorithms for cooperative multisensor surveillance. Proceedings of the IEEE, 89(10):1456–1477.CrossRefGoogle Scholar
  23. Conroy, R.A. 2001. Spatial navigation in immersive virtual environments. PhD thesis, University of London.Google Scholar
  24. Daly, A. 2001. Recursive Nested EV model. ITS Working Paper 559, Institute for Transport Studies, University of Leeds.Google Scholar
  25. DeCarlo, D. and Metaxas, D. 2000. Optical flow constraints on deformable models with applications to face tracking. Int. J. Comp. Vis., 38(2):99–127.CrossRefzbMATHGoogle Scholar
  26. Ferryman, J.M., Maybank, S.J., and Worrall, A.D. 2000. Visual surveillance for moving vehicles. Int. J. Comp. Vis., 37(2):187–197.CrossRefzbMATHGoogle Scholar
  27. Gavrila, D.M. 1999. The visual analysis of human movement: A survey. Computer Vision and Image Understanding: CVIU, 73(l):82–98.zbMATHGoogle Scholar
  28. Geman, S., Geman, D., and Dong, P. 1990. Boundary detection by constrained optimization. IEEE Pattern Analysis and Machine Intelligence (PAMI), 12:609–628.CrossRefGoogle Scholar
  29. Haklay, M., O’Sullivan, D., Thurstain-Goodwin, M., and Schelhorn, T. 2001. “So go down town”: Simulating pedestrian movement in town centres. Environment and Planning B, 28(3):343–359.Google Scholar
  30. Haritaoglu, I., Harwood, D., and Davis, L.S. 2000. W4: Real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell., 22:809–830.CrossRefGoogle Scholar
  31. Helbing, D., Farkas, I., and Vicsek, T. 2000. Simulating dynamical features of escape panic. Nature, 407(28):487–490.CrossRefGoogle Scholar
  32. Helbing, D., Farkas, I.J., Molnar, P., and Vicsek, T. 2002. Simulation of pedestrian crowds in normal and evacuation simulations. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 21–58.Google Scholar
  33. Helbing, D. and Molnar, P. 1995. Social force model for pedestrian dynamics. Physical review E, 51(5):4282–4286.CrossRefGoogle Scholar
  34. Hensher, D.A. and Johnson, L.W. 1981. Applied Discrete-Choice Modelling. Groom Helm: London.Google Scholar
  35. Hess, S., Bierlaire, M. and Polak, J.W. 2005. Capturing taste heterogeneity and correlation structure with mixed gev models. In 84th Annual Meeting of the Transportation Research Board, Washington B.C.Google Scholar
  36. Hoogendoorn, S.P. 2003. Pedestrian travel behavior modeling. In 10th International Conference on Travel Behavior Research, Lucerne.Google Scholar
  37. Hoogendoorn, S.P., Bovy, P.H.L., and Daamen, W. 2002. Microscopic pedestrian wayfinding and dynamics modelling. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 123–155.Google Scholar
  38. Isard, M. and Blake, A. 1996. Contour tracking by stochastic propagation of conditional density. European Conference on Computer Vision, 1:343–356.Google Scholar
  39. Isard, M. and Blake, A. 1998. Condensation—conditional density propagation for visual tracking. International Journal on Computer Vision, 1(29):5–28.CrossRefGoogle Scholar
  40. Johnson, N. and Hogg, D. 1995. Learning the distribution of object trajectories for event recognition. In BMVC ’95: Proceedings of the 6th British Conference on Machine Vision, (Vol. 2), BMVA Press. Surrey, UK, pp. 583–592.Google Scholar
  41. Jurie, F. and Dhome, M. 2001. Real time 3d template matching. In International Conference on Computer Vision and Pattern Recognition, Hawai, pp. I 791–797.Google Scholar
  42. Kakadiaris, I., Metaxas, D., and Bajcsy, R. 1994. Active part-decomposition, shape and motion estimation of articulated objects: A physics-based approach. In Computer Vision and Pattern Recognition, pp. 980–984.Google Scholar
  43. Kaneko, T. and Hori, O. 2003. Feature selection for reliable tracking using template matching. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 796–802.Google Scholar
  44. Kitagawa, G. 1996. Monte carlo filter and smoother for non-gaussian nonlinear state space models. Journal of Computational and Graphical Statistics, 5(1):1–25.MathSciNetCrossRefGoogle Scholar
  45. Klüpfel, H., Meyer-König, M., Wähle, J., and Schreckenberg, M. 2000. Microscopic simulation of evacuation processes on passenger ships. In Theoretical and Practical Issues on Cellular Automata, S. Bandini and Th. Worsch (eds.), London, pp. 63–71.Google Scholar
  46. Koning, R.H. 1991. Discrete choice and stochastic utility maximization. Research Memorandum 414, Department of Economics, Groningen University.Google Scholar
  47. Koning, R.H. and Ridder, G. 1994. On the compatibility of nested logit models with utility maximization. Journal of Econometrics, 63:389–396.MathSciNetCrossRefzbMATHGoogle Scholar
  48. Lawrence, C.T., Zhou, J.L., and Tits, A. 1997. User ’s guide for CFSQP version 2.5: AC code for solving (large scale) constrained nonlinear (minimax) optimization problems, generating iterates satisfying all inequality constraints. Technical Report TR-94-16r1, Institute for Systems Research, University of Maryland, College Park, MD 20742.Google Scholar
  49. Luce, R.D. 1959. Individual Choice Behavior: A theoretical analysis. John Wiley & Sons: New York.Google Scholar
  50. Manski, C.F. and McFadden, D. 1981. Econometric models of probabilistic choice. In Structural Analysis of Discrete Data with Econometric Applications, C.F. Manski and D. McFadden (eds.), MIT Press: Cambridge, pp. 198–272.Google Scholar
  51. McFadden, D. 1997. Modelling the choice of residential location. The Economics of Housing, 1:531–552, reprinted.Google Scholar
  52. Mendels, F., Vandergheynst, P., and Thiran, J.P. 2002. Rotation and scale invariant shape representation and recognition using matching pursuit. In Proc. of the International Conference on Pattern Recognition ICPR 2002, IEEE, vol. 4, pp. 326–329.Google Scholar
  53. Moeslund, T.B. and Granum, E. 2001. A survey of computer vision-based human motion capture. Computer Vision and Image Understanding: CVIU, 81(3):231–268.CrossRefzbMATHGoogle Scholar
  54. Nummiaro, K., Koller-Meier, E., Svoboda, T., Roth, D., and Van Gool, L. 2003. Color-based object tracking in multi-camera environments. In 25th Pattern Recognition Symposium, DAGM 2003, B. Michaelis and G. Krell (eds.), LNCS, Springer, pp. 591–599.Google Scholar
  55. Nummiaro, K., Koller-Meier, E., and Van Gool, L. 2002. Object tracking with an adaptive color-based particle filter. In Symposium for Pattern Recognition of the DAGM, L. Van Gool (ed.), Springer, pp. 353–360.Google Scholar
  56. Oliver, N.M., Rosario, B., and Pentland, A.P. 2000. A bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell., 22:831–843.CrossRefGoogle Scholar
  57. Penn, A. and Turner, A. 2002. Space syntax based agent simulation. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 99–114.Google Scholar
  58. Ramming, M.S. 2001. Network knowledge and route choice. PhD thesis, Massachusetts Institute of Technology.Google Scholar
  59. Rosales, R. and Sclaroff, S. 1999. 3d trajectory recovery for tracking multiple objects and trajectory-guided recognition of actions. Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 117–123.Google Scholar
  60. Schadschneider, A. 2002. Cellular automaton approach to pedestrian dynamics—Theory. In Pedestrian and Evacuation Dynamics. M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 75–86.Google Scholar
  61. Schreckenberg, M. and Sharma, S.D. (eds.) 2002. Pedestrian and Evacuation Dynamics. Springer Verlag.Google Scholar
  62. Senior, A.W. 2002. Tracking with probabilistic appearance models. In Proc. ECCV Workshop on Performance Evaluation of Tracking and Surveillance Systems, pp. 48–55.Google Scholar
  63. Small, K. 1987. A discrete choice model for ordered alternatives. Econometrica, 55(2):409–424.zbMATHMathSciNetGoogle Scholar
  64. Stauffer, C. and crimson, W.E.L. 2000. Learning patterns of activity using realtime tracking. IEEE Trans. Pattern Anal. Mach. Intell., 22:747–757.CrossRefGoogle Scholar
  65. Terzopoulos, D., Witkin, A., and Kass, M. 1988. Constraints on deformable models: Recovering 3d shape and nonrigid motion. Artificial Intelligence, 36(1):91–123.CrossRefzbMATHGoogle Scholar
  66. Thayananthan, A., Stenger, B., Torr, P.H.S., and Cipolla, R. 2003. Learning a kinematic prior for tree-based filtering. In Proc. British Machine Vision Conference, Norwich, UK, vol. 2, pp. 589–598.Google Scholar
  67. Toledo, T. 2003. Integrated driving behavior modeling. PhD thesis, Massachusetts Institute of Technology.Google Scholar
  68. Train, K. 2003. Discrete Choice Methods with Simulation. Cambridge University Press, University of California, Berkeley.Google Scholar
  69. Turner, A., Doxa, M., O’Sullivan, D., and Penn, A. 2001. From isovists to visibility graphs: a methodology for the analysis of architectural space. Environment and Planning B, 28(1):103–121.CrossRefGoogle Scholar
  70. Venegas, S., Knebel, S.F., and Thiran, J.P. 2004. Multi-object tracking using particle filter algorithm on the top-view plan. In European Signal Processing Conference (EUSIPCO).Google Scholar
  71. Vovsha, P. 1997. Cross-nested logit model: An application to mode choice in the Tel-Aviv metropolitan area. Transportation Research Board, 76th Annual Meeting, Washington DC, January 1997. Paper #970387.Google Scholar
  72. Walker, J.L. 2001. Extended discrete choice models: Integrated framework, flexible error structures, and latent variables. PhD thesis, Massachusetts Institute of Technology.Google Scholar
  73. Wang, J.J. and Singh, S. 2003. Video analysis of human dynamics: A survey. RealTimeImg, 9(5):320–345.Google Scholar
  74. Wen, C.-H. and Koppelman, F.S. 2001.The generalized nested logit model. Transportation Research B (TRB), 35(7):627–641.Google Scholar
  75. Wren, C.R. and Pentland, A.P. 1998. Dynamic models of human motion. In FG ’98: Proceedings of the 3rd. International Conference on Face and Gesture Recognition, Washington, DC, USA, IEEE Computer Society, pp. 22.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Gianluca Antonini
    • 1
    Email author
  • Santiago Venegas Martinez
    • 1
  • Michel Bierlaire
    • 2
  • Jean Philippe Thiran
    • 1
  1. 1.Ecole Polytechnique Federale de Lausanne (EPFL)Signal Processing Institute (STI/ITS/LTS5)LausanneSwitzerland
  2. 2.Ecole Polytechnique Federale de Lausanne (EPFL)Operation ResearchLausanneSwitzerland

Personalised recommendations