Real Time Vision Based Multi-person Tracking for Mobile Robotics and Intelligent Vehicles

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7102)


In this paper, we present a real-time vision-based multi-person tracking system working in crowded urban environments. Our approach combines stereo visual odometry estimation, HOG pedestrian detection, and multi-hypothesis tracking-by-detection to a robust tracking framework that runs on a single laptop with a CUDA-enabled graphics card. Through shifting the expensive computations to the GPU and making extensive use of scene geometry constraints we could build up a mobile system that runs with 10Hz. We experimentally demonstrate on several challenging sequences that our approach achieves competitive tracking performance.


Ground Plane Color Histogram Mobile Platform Mobile Robotic Pedestrian Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bajracharya, M., Moghaddam, B., Howard, A., Brennan, S., Matthies, L.: Results from a real-time stereo-based pedestrian detection system on a moving vehicle. In: ICRA 2009 (2009)Google Scholar
  2. 2.
    Bansal, M., Jung, S.H., Matei, B., Eledath, J., Sawhney, H.S.: A real-time pedestrian detection system based on structure and appearance classification. In: ICRA 2010 (2010)Google Scholar
  3. 3.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR 2005 (2005)Google Scholar
  4. 4.
    Enzweiler, M., Kanter, P., Gavrila, D.: Monocular Pedestrian Recognition Using Motion Parallax. In: IVS (2008)Google Scholar
  5. 5.
    Ess, A., Leibe, B., Schindler, K., Van Gool, L.: Robust Multi-Person Tracking from a Mobile Platform. PAMI 31(10), 1831–1846 (2009)CrossRefGoogle Scholar
  6. 6.
    Gavrila, D., Munder, S.: Multi-Cue Pedestrian Detection and Tracking from a Moving Vehicle. IJCV 73(1), 41–59 (2007)CrossRefGoogle Scholar
  7. 7.
    Geiger, A., Roser, M., Urtasun, R.: Efficient Large-Scale Stereo Matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 25–38. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Geronimo, D., Sappa, A., Ponsa, D., Lopez, A.: 2D-3D-based On-Board Pedestrian Detection System. CVIU 114(5), 583–595 (2010)Google Scholar
  9. 9.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, AVC 1988 (1988)Google Scholar
  10. 10.
    Labayrade, R., Aubert, D., Tarel, J.: Real time obstacle detection in stereovision on non flat road geometry through v-disparity representation. In: IVS (2002)Google Scholar
  11. 11.
    Leibe, B., Schindler, K., Van Gool, L.: Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles. PAMI 30(10), 1683–1698 (2008)CrossRefGoogle Scholar
  12. 12.
    Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. IJCV 60(1), 63–86 (2004)CrossRefGoogle Scholar
  13. 13.
    Nistér, D., Naroditsky, O., Bergen, J.: Visual odometry. In: CVPR 2004 (2004)Google Scholar
  14. 14.
    Nistér, D., Stewénius, H.: A minimal solution to the generalised 3-point pose problem. Journal of Mathematical Imaging and Vision 27(1), 67–79 (2007)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Shashua, A., Gdalyahu, Y., Hayon, G.: Pedestrian Detection for Driving Assistance Systems: Single-Frame Classification and System Level Performance. In: IVS (2004)Google Scholar
  16. 16.
    Sudowe, P., Leibe, B.: Efficient Use of Geometric Constraints for Sliding-Window Object Detection in Video ICVS (2011),
  17. 17.
    Wu, B., Nevatia, R.: Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet Part Detectors. IJCV 75(2) (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.UMIC Research CentreRWTH Aachen UniversityGermany

Personalised recommendations