Skip to main content
Book cover

European Conference on Computer Vision

ECCV 2012: Computer Vision – ECCV 2012 pp 566–579Cite as

Taking Mobile Multi-object Tracking to the Next Level: People, Unknown Objects, and Carried Items

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNIP,volume 7576)

Abstract

In this paper, we aim to take mobile multi-object tracking to the next level. Current approaches work in a tracking-by-detection framework, which limits them to object categories for which pre-trained detector models are available. In contrast, we propose a novel tracking-before-detection approach that can track both known and unknown object categories in very challenging street scenes. Our approach relies on noisy stereo depth data in order to segment and track objects in 3D. At its core is a novel, compact 3D representation that allows us to robustly track a large variety of objects, while building up models of their 3D shape online. In addition to improving tracking performance, this representation allows us to detect anomalous shapes, such as carried items on a person’s body. We evaluate our approach on several challenging video sequences of busy pedestrian zones and show that it outperforms state-of-the-art approaches.

Download conference paper PDF

References

  1. Leibe, B., Schindler, K., Van Gool, L.: Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles. PAMI 30(10), 1683–1698 (2008)

    CrossRef  Google Scholar 

  2. Ess, A., Leibe, B., Schindler, K., Van Gool, L.: Robust Multi-Person Tracking from a Mobile Platform. PAMI 31(10), 1831–1846 (2009)

    CrossRef  Google Scholar 

  3. Bajracharya, M., Moghaddam, B., Howard, A., Brennan, S., Matthies, L.: A Fast Stereo-based System for Detecting and Tracking Pedestrians from a Moving Vehicle. IJRS 28(11-12), 1466–1485 (2009)

    Google Scholar 

  4. Bansal, M., Jung, S.H., Matei, B., Eledath, J., Sawhney, H.S.: A real-time pedestrian detection system based on structure and appearance classification. In: ICRA (2010)

    Google Scholar 

  5. Kuo, C.H., Huang, C., Nevatia, R.: Multi-Target Tracking by On-Line Learned Discriminative Appearance Models. In: CVPR (2010)

    Google Scholar 

  6. Andriluka, M., Roth, S., Schiele, B.: Monocular 3D Pose Estimation and Tracking by Detection. In: CVPR (2010)

    Google Scholar 

  7. Wojek, C., Walk, S., Roth, S., Schiele, B.: Monocular 3D Scene Understanding with Explicit Occlusion Reasoning. In: CVPR (2011)

    Google Scholar 

  8. Felzenszwalb, P., Girshick, B., McAllester, D., Ramanan, D.: Object Detection with Discriminatively Trained Part-Based Models. PAMI 32(9), 1627–1645 (2010)

    CrossRef  Google Scholar 

  9. Mitzel, D., Horbert, E., Ess, A., Leibe, B.: Multi-person Tracking with Sparse Detection and Continuous Segmentation. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 397–410. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  10. Besl, P.J., Mckay, H.D.: A method for registration of 3-D shapes. PAMI 14(2) (1992)

    Google Scholar 

  11. Luber, M., Spinello, L., Arras, K.: People Tracking in RGB-D Data With Online-Boosted Target Models. In: IROS (2011)

    Google Scholar 

  12. Prengaman, R., Thurber, R., Bath, W.: Retrospective Detection Algorithm for Extraction of Weak Targets in Clutter and Interference Environments. In: IEEE Int. Radar Conf. (1982)

    Google Scholar 

  13. Salmond, D., Birch, H.: A Particle Filter for Track-Before-Detect. In: American Control Conf. (2001)

    Google Scholar 

  14. Danescu, R., Oniga, F., Nedevschi, S.: Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid. IEEE Trans. Intel. Transp. Syst. 12(4), 1331–1342 (2011)

    CrossRef  Google Scholar 

  15. Geronimo, D., Lopez, A., Sappa, A., Graf, T.: Survey of Pedestrian Detection for Advanced Driver Assistance Systems. PAMI 32(7), 1239–1258 (2010)

    CrossRef  Google Scholar 

  16. Petrovskaya, A., Thrun, S.: Model Based Vehicle Detection and Tracking for Autonomous Urban Driving. AR 26(2–3), 123–139 (2009)

    Google Scholar 

  17. Kaestner, R., Maye, J., Siegwart, R.: Generative Object Detection and Tracking in 3D Range Data. In: ICRA (2012)

    Google Scholar 

  18. Gavrila, D., Munder, S.: Multi-Cue Pedestrian Detection and Tracking from a Moving Vehicle. IJCV 73(1), 41–59 (2007)

    CrossRef  Google Scholar 

  19. Keller, C., Fernandez-Llorca, D., Gavrila, D.: Dense Stereo-Based ROI Generation for Pedestrian Detection. In: Denzler, J., Notni, G., Süße, H. (eds.) Pattern Recognition. LNCS, vol. 5748, pp. 81–90. Springer, Heidelberg (2009)

    CrossRef  Google Scholar 

  20. Mitzel, D., Leibe, B.: Real-Time Multi-Person Tracking with Detector Assisted Structure Propagation. In: ICCV CORP Workshop (2011)

    Google Scholar 

  21. Feldman, A., Hybinette, M., Balch, T., Cavallaro, R.: The Multi-ICP Tracker: An Online Algorithm for Tracking Multiple Interacting Targets. J. Field Robotics (2012)

    Google Scholar 

  22. Teichman, A., Thrun, S.: Tracking-Based Semi-Supervised Learning. In: RSS (2011)

    Google Scholar 

  23. Teichman, A., Levinson, J., Thrun, S.: Towards 3D Object Recognition via Classification of Arbitrary Object Tracks. In: ICRA (2011)

    Google Scholar 

  24. Bjorkman, M., Kragic, D.: Active 3D Segmentation and Detection of Unknown Objects. In: IROS (2010)

    Google Scholar 

  25. Damen, D., Hogg, D.: Detecting Carried Objects in Short Video Sequences. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 154–167. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  26. Nistér, D., Naroditsky, O., Bergen, J.: Visual odometry. In: CVPR (2004)

    Google Scholar 

  27. Vedaldi, A., Soatto, S.: Quick Shift and Kernel Methods for Mode Seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  28. Kaucic, R., Perera, A., Brooksby, G., Kaufhold, J., Hoogs, A.: A Unified Framework for Tracking through Occlusions and Across Sensor Gaps. In: CVPR (2005)

    Google Scholar 

  29. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive Foreground Extraction Using Iterated Graph Cuts. In: SIGGRAPH (2004)

    Google Scholar 

  30. Boykov, Y., Veksler, O., Zabih, R.: Fast Approximate Energy Minimization via Graph Cuts. PAMI 23(11), 1222–1239 (2001)

    CrossRef  Google Scholar 

  31. Geiger, A., Roser, M., Urtasun, R.: Efficient Large-Scale Stereo Matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 25–38. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision Group, RWTH Aachen University, Germany

    Dennis Mitzel & Bastian Leibe

Authors
  1. Dennis Mitzel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bastian Leibe
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Microsoft Research Ltd., CB3 0FB, Cambridge, UK

    Andrew Fitzgibbon

  2. Dept. of Computer Science, University of North Carolina, 27599, Chapel Hill, NC, USA

    Svetlana Lazebnik

  3. California Institute of Technology, 91125, Pasadena, CA, USA

    Pietro Perona

  4. Institute of Industrial Science, The University of Tokyo, 153-8505, Tokyo, Japan

    Yoichi Sato

  5. INRIA, 38330, Montbonnot, France

    Cordelia Schmid

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Mitzel, D., Leibe, B. (2012). Taking Mobile Multi-object Tracking to the Next Level: People, Unknown Objects, and Carried Items. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7576. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33715-4_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33715-4_41

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33714-7

  • Online ISBN: 978-3-642-33715-4

  • eBook Packages: Computer ScienceComputer Science (R0)