Skip to main content
SpringerLink
Log in

Behavioral Priors for Detection and Tracking of Pedestrians in Video Sequences

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper we address the problem of detection and tracking of pedestrians in complex scenarios. The inclusion of prior knowledge is more and more crucial in scene analysis to guarantee flexibility and robustness, necessary to have reliability in complex scenes. We aim to combine image processing methods with behavioral models of pedestrian dynamics, calibrated on real data. We introduce Discrete Choice Models (DCM) for pedestrian behavior and we discuss their integration in a detection and tracking context. The obtained results show how it is possible to combine both methodologies to improve the performances of such systems in complex sequences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • AlGadhi, S.A.H., Mahmassani, H., and Herman, R. 2002. A speed-concentration relation for bi-directional crowd movements. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 3–20.

  • Antonini, G., Bierlaire, M., and Weber, M. 2004. Discrete choice models of pedestrian walking behavior. Accepted for publications in Transportation Research Part B.

  • Antonini, G. and Thiran, J.P. 2004. Trajectories clustering in ica space: an application to automatic counting of pedestrians in video sequences. In Advanced Concepts for Intelligent Vision Systems (ACIVS), J. Blanc-Talon and D. Popescu (eds.), Brussels, Belgium.

  • Antonini, G., Venegas, S., Thiran, J.P., and Bierlaire, M. 2004. A discrete choice pedestrian behavior model for pedestrian detection in visual tracking systems. In Advanced Concepts for Intelligent Vision Systems (ACIVS), J. Blanc-Talon and D. Popescu (eds.), Brussels, Belgium.

  • Arulampalam, M.S., Maskell, S., Gordon, N., and Clapp, T. 2002. A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. on Signal Processing, 50(2):174–188.

    Article  Google Scholar 

  • Ben-Akiva, M. and Bierlaire, M. 1999. Discrete choice methods and their applications to short-term travel decisions. In Handbook of Transportation Science, R. Hall (ed.), Kluwer, pp. 5–34.

  • Ben-Akiva, M.E., Bergman, M.J., Daly, A.J., and Ramaswamy, R. 1984. Modeling inter-urban route choice behaviour. In Proceedings from the Ninth International Symposium on Transportation and Traffic Theory, J. Volmuller and R. Hamerslag (eds.), VNU Science Press: Utrecht, Netherlands, pp. 299–330.

    Google Scholar 

  • Ben-Akiva, M.E. and Lerman, S.R. 1985. Discrete Choice Analysis: Theory and Application to Travel Demand. MIT Press: Cambridge, MA.

    Google Scholar 

  • Bierlaire, M. 2001. A theoretical analysis of the cross-nested logit model. Accepted for publications in Annals of Operations Research.

  • Bierlaire, M. 2002. The network GEV model. In Proceedings of the 2nd Swiss Transportation Research Conference, Ascona, Switzerland, http://www.strc.ch/pdf_2002/bierlaire2.zip.

  • Bierlaire, M. 2003. An introduction to BIOGEME Version 0.6, February 2003.roso.epfl.ch/biogeme.

  • Bierlaire, M., Antonini, G., and Weber, M. 2003. Behavioral dynamics for pedestrians. In Moving Through Nets: The Physical and Social Dimensions of Travel, K. Axhausen (ed.), Elsevier, pp. 1–18.

  • Biliotti, D., Antonini, G., and Thiran, J.P. 2005. Multi-layer trajectories clustering for automatic counting of pedestrians in video sequences. In IEEE Motion 2005, IEEE Computer Society.

  • Blue, V.J. and Adler, J.L. 2001. Cellular automata microsimulation for modeling bi-directional pedestrian walkways. Transportation Research B, 35(3):293–312.

    Article  Google Scholar 

  • Borgers, A. and Timmermans, H. 1986a. A model of pedestrian route choice and demand for retail facilities within inner-city shopping areas. Geographical Analysis, 18(2):115–128.

    Article  Google Scholar 

  • Borgers, A. and Timmermans, H. 1986b. City centre entry points, store location patterns and pedestrian route choice behaviour: A micro-level simulation model. Socio-Economie Planning Sciences, 20(1):25–31.

    Article  Google Scholar 

  • Bregler, C. 1997. Learning and recognizing human dynamics in video sequences. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.

  • Campbell, L.W. and Bobick, A.F. 1995. Recognition of human body motion using phase space constraints. In International Conference in Computer Vision (ICCV), pp. 624–630.

  • Canny, J.F. 1986. A computational approach to edge detection. IEEE Trans. Patt. Anal. Mach. Intell., 8:679–698.

    Article  Google Scholar 

  • Cascetta, E., Nuzzolo, A., and Biggiero, L. 1992. Analysis and modeling of commuters’ departure time and route choice in urban networks. In Proceedings of the Second International Capri Seminar on Urban Traffic Networks.

  • Cohen, L.D. and Cohen, I. 1900. A finite element method applied to new active contour models and 3d reconstruction from cross sections. In International Conference on Computer Vision (ICCV).

  • Collins, R., Lipton, A., Fujiyoshi, H., and Kanade, T. 2001. Algorithms for cooperative multisensor surveillance. Proceedings of the IEEE, 89(10):1456–1477.

    Article  Google Scholar 

  • Conroy, R.A. 2001. Spatial navigation in immersive virtual environments. PhD thesis, University of London.

  • Daly, A. 2001. Recursive Nested EV model. ITS Working Paper 559, Institute for Transport Studies, University of Leeds.

  • DeCarlo, D. and Metaxas, D. 2000. Optical flow constraints on deformable models with applications to face tracking. Int. J. Comp. Vis., 38(2):99–127.

    Article  MATH  Google Scholar 

  • Ferryman, J.M., Maybank, S.J., and Worrall, A.D. 2000. Visual surveillance for moving vehicles. Int. J. Comp. Vis., 37(2):187–197.

    Article  MATH  Google Scholar 

  • Gavrila, D.M. 1999. The visual analysis of human movement: A survey. Computer Vision and Image Understanding: CVIU, 73(l):82–98.

    MATH  Google Scholar 

  • Geman, S., Geman, D., and Dong, P. 1990. Boundary detection by constrained optimization. IEEE Pattern Analysis and Machine Intelligence (PAMI), 12:609–628.

    Article  Google Scholar 

  • Haklay, M., O’Sullivan, D., Thurstain-Goodwin, M., and Schelhorn, T. 2001. “So go down town”: Simulating pedestrian movement in town centres. Environment and Planning B, 28(3):343–359.

    Google Scholar 

  • Haritaoglu, I., Harwood, D., and Davis, L.S. 2000. W4: Real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell., 22:809–830.

    Article  Google Scholar 

  • Helbing, D., Farkas, I., and Vicsek, T. 2000. Simulating dynamical features of escape panic. Nature, 407(28):487–490.

    Article  Google Scholar 

  • Helbing, D., Farkas, I.J., Molnar, P., and Vicsek, T. 2002. Simulation of pedestrian crowds in normal and evacuation simulations. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 21–58.

  • Helbing, D. and Molnar, P. 1995. Social force model for pedestrian dynamics. Physical review E, 51(5):4282–4286.

    Article  Google Scholar 

  • Hensher, D.A. and Johnson, L.W. 1981. Applied Discrete-Choice Modelling. Groom Helm: London.

  • Hess, S., Bierlaire, M. and Polak, J.W. 2005. Capturing taste heterogeneity and correlation structure with mixed gev models. In 84th Annual Meeting of the Transportation Research Board, Washington B.C.

  • Hoogendoorn, S.P. 2003. Pedestrian travel behavior modeling. In 10th International Conference on Travel Behavior Research, Lucerne.

  • Hoogendoorn, S.P., Bovy, P.H.L., and Daamen, W. 2002. Microscopic pedestrian wayfinding and dynamics modelling. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 123–155.

  • Isard, M. and Blake, A. 1996. Contour tracking by stochastic propagation of conditional density. European Conference on Computer Vision, 1:343–356.

    Google Scholar 

  • Isard, M. and Blake, A. 1998. Condensation—conditional density propagation for visual tracking. International Journal on Computer Vision, 1(29):5–28.

    Article  Google Scholar 

  • Johnson, N. and Hogg, D. 1995. Learning the distribution of object trajectories for event recognition. In BMVC ’95: Proceedings of the 6th British Conference on Machine Vision, (Vol. 2), BMVA Press. Surrey, UK, pp. 583–592.

  • Jurie, F. and Dhome, M. 2001. Real time 3d template matching. In International Conference on Computer Vision and Pattern Recognition, Hawai, pp. I 791–797.

  • Kakadiaris, I., Metaxas, D., and Bajcsy, R. 1994. Active part-decomposition, shape and motion estimation of articulated objects: A physics-based approach. In Computer Vision and Pattern Recognition, pp. 980–984.

  • Kaneko, T. and Hori, O. 2003. Feature selection for reliable tracking using template matching. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 796–802.

  • Kitagawa, G. 1996. Monte carlo filter and smoother for non-gaussian nonlinear state space models. Journal of Computational and Graphical Statistics, 5(1):1–25.

    Article  MathSciNet  Google Scholar 

  • Klüpfel, H., Meyer-König, M., Wähle, J., and Schreckenberg, M. 2000. Microscopic simulation of evacuation processes on passenger ships. In Theoretical and Practical Issues on Cellular Automata, S. Bandini and Th. Worsch (eds.), London, pp. 63–71.

  • Koning, R.H. 1991. Discrete choice and stochastic utility maximization. Research Memorandum 414, Department of Economics, Groningen University.

  • Koning, R.H. and Ridder, G. 1994. On the compatibility of nested logit models with utility maximization. Journal of Econometrics, 63:389–396.

    Article  MathSciNet  MATH  Google Scholar 

  • Lawrence, C.T., Zhou, J.L., and Tits, A. 1997. User ’s guide for CFSQP version 2.5: AC code for solving (large scale) constrained nonlinear (minimax) optimization problems, generating iterates satisfying all inequality constraints. Technical Report TR-94-16r1, Institute for Systems Research, University of Maryland, College Park, MD 20742.

  • Luce, R.D. 1959. Individual Choice Behavior: A theoretical analysis. John Wiley & Sons: New York.

    Google Scholar 

  • Manski, C.F. and McFadden, D. 1981. Econometric models of probabilistic choice. In Structural Analysis of Discrete Data with Econometric Applications, C.F. Manski and D. McFadden (eds.), MIT Press: Cambridge, pp. 198–272.

    Google Scholar 

  • McFadden, D. 1997. Modelling the choice of residential location. The Economics of Housing, 1:531–552, reprinted.

    Google Scholar 

  • Mendels, F., Vandergheynst, P., and Thiran, J.P. 2002. Rotation and scale invariant shape representation and recognition using matching pursuit. In Proc. of the International Conference on Pattern Recognition ICPR 2002, IEEE, vol. 4, pp. 326–329.

  • Moeslund, T.B. and Granum, E. 2001. A survey of computer vision-based human motion capture. Computer Vision and Image Understanding: CVIU, 81(3):231–268.

    Article  MATH  Google Scholar 

  • Nummiaro, K., Koller-Meier, E., Svoboda, T., Roth, D., and Van Gool, L. 2003. Color-based object tracking in multi-camera environments. In 25th Pattern Recognition Symposium, DAGM 2003, B. Michaelis and G. Krell (eds.), LNCS, Springer, pp. 591–599.

  • Nummiaro, K., Koller-Meier, E., and Van Gool, L. 2002. Object tracking with an adaptive color-based particle filter. In Symposium for Pattern Recognition of the DAGM, L. Van Gool (ed.), Springer, pp. 353–360.

  • Oliver, N.M., Rosario, B., and Pentland, A.P. 2000. A bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell., 22:831–843.

    Article  Google Scholar 

  • Penn, A. and Turner, A. 2002. Space syntax based agent simulation. In Pedestrian and Evacuation Dynamics, M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 99–114.

  • Ramming, M.S. 2001. Network knowledge and route choice. PhD thesis, Massachusetts Institute of Technology.

  • Rosales, R. and Sclaroff, S. 1999. 3d trajectory recovery for tracking multiple objects and trajectory-guided recognition of actions. Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 117–123.

  • Schadschneider, A. 2002. Cellular automaton approach to pedestrian dynamics—Theory. In Pedestrian and Evacuation Dynamics. M. Schreckenberg and S.D. Sharma (eds.), Springer, pp. 75–86.

  • Schreckenberg, M. and Sharma, S.D. (eds.) 2002. Pedestrian and Evacuation Dynamics. Springer Verlag.

  • Senior, A.W. 2002. Tracking with probabilistic appearance models. In Proc. ECCV Workshop on Performance Evaluation of Tracking and Surveillance Systems, pp. 48–55.

  • Small, K. 1987. A discrete choice model for ordered alternatives. Econometrica, 55(2):409–424.

    MATH  MathSciNet  Google Scholar 

  • Stauffer, C. and crimson, W.E.L. 2000. Learning patterns of activity using realtime tracking. IEEE Trans. Pattern Anal. Mach. Intell., 22:747–757.

    Article  Google Scholar 

  • Terzopoulos, D., Witkin, A., and Kass, M. 1988. Constraints on deformable models: Recovering 3d shape and nonrigid motion. Artificial Intelligence, 36(1):91–123.

    Article  MATH  Google Scholar 

  • Thayananthan, A., Stenger, B., Torr, P.H.S., and Cipolla, R. 2003. Learning a kinematic prior for tree-based filtering. In Proc. British Machine Vision Conference, Norwich, UK, vol. 2, pp. 589–598.

  • Toledo, T. 2003. Integrated driving behavior modeling. PhD thesis, Massachusetts Institute of Technology.

  • Train, K. 2003. Discrete Choice Methods with Simulation. Cambridge University Press, University of California, Berkeley.

    Google Scholar 

  • Turner, A., Doxa, M., O’Sullivan, D., and Penn, A. 2001. From isovists to visibility graphs: a methodology for the analysis of architectural space. Environment and Planning B, 28(1):103–121.

    Article  Google Scholar 

  • Venegas, S., Knebel, S.F., and Thiran, J.P. 2004. Multi-object tracking using particle filter algorithm on the top-view plan. In European Signal Processing Conference (EUSIPCO).

  • Vovsha, P. 1997. Cross-nested logit model: An application to mode choice in the Tel-Aviv metropolitan area. Transportation Research Board, 76th Annual Meeting, Washington DC, January 1997. Paper #970387.

  • Walker, J.L. 2001. Extended discrete choice models: Integrated framework, flexible error structures, and latent variables. PhD thesis, Massachusetts Institute of Technology.

  • Wang, J.J. and Singh, S. 2003. Video analysis of human dynamics: A survey. RealTimeImg, 9(5):320–345.

    Google Scholar 

  • Wen, C.-H. and Koppelman, F.S. 2001.The generalized nested logit model. Transportation Research B (TRB), 35(7):627–641.

    Google Scholar 

  • Wren, C.R. and Pentland, A.P. 1998. Dynamic models of human motion. In FG ’98: Proceedings of the 3rd. International Conference on Face and Gesture Recognition, Washington, DC, USA, IEEE Computer Society, pp. 22.

Download references

Author information

Authors and Affiliations

  1. Ecole Polytechnique Federale de Lausanne (EPFL), Signal Processing Institute (STI/ITS/LTS5), CH-1015, Lausanne, Switzerland

    Gianluca Antonini, Santiago Venegas Martinez & Jean Philippe Thiran

  2. Ecole Polytechnique Federale de Lausanne (EPFL), Operation Research, CH-1015, Lausanne, Switzerland

    Michel Bierlaire

Authors
  1. Gianluca Antonini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Santiago Venegas Martinez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michel Bierlaire
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jean Philippe Thiran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gianluca Antonini.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Antonini, G., Martinez, S.V., Bierlaire, M. et al. Behavioral Priors for Detection and Tracking of Pedestrians in Video Sequences. Int J Comput Vision 69, 159–180 (2006). https://doi.org/10.1007/s11263-005-4797-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-005-4797-0

Keywords

Search

Navigation