Integrating Recognition and Reconstruction for Cognitive Traffic Scene Analysis from a Moving Vehicle

  • Bastian Leibe
  • Nico Cornelis
  • Kurt Cornelis
  • Luc Van Gool
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4174)


This paper presents a practical system for vision-based traffic scene analysis from a moving vehicle based on a cognitive feedback loop which integrates real-time geometry estimation with appearance-based object detection. We demonstrate how those two components can benefit from each other’s continuous input and how the transferred knowledge can be used to improve scene analysis. Thus, scene interpretation is not left as a matter of logical reasoning, but is instead addressed by the repeated interaction and consistency checks between different levels and modes of visual processing. As our results show, the proposed tight integration significantly increases recognition performance, as well as overall system robustness. In addition, it enables the construction of novel capabilities such as the accurate 3D estimation of object locations and orientations and their temporal integration in a world coordinate frame. The system is evaluated on a challenging real-world car detection task in an urban scenario.


Object Detection Stereo Pair Scene Analysis Reconstruction Module Scene Geometry 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andreone, L., Antonello, P.C., Bertozzi, M., Broggi, A., Fascioli, A., Ranzato, D.: Vehicle detection and localization in infra-red images. In: Intel. Vehicles Symp. 2002 (2002)Google Scholar
  2. 2.
    Betke, M., Haritaoglu, E., Davis, L.S.: Real-time multiple vehicle tracking from a moving vehicle. MVA 12(2), 69–83 (2000)CrossRefGoogle Scholar
  3. 3.
    Cornelis, N., Cornelis, K., Van Gool, L.: Fast compact city modeling for navigation pre-visualization. In: CVPR 2006 (2006)Google Scholar
  4. 4.
    Fischler, M., Bolles, R.: Random sampling consensus: A paradigm for model fitting with application to image analysis and automated cartography. Comm. ACM 24, 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Gavrila, D., Philomin, V.: Real-time object detection for smart vehicles. In: ICCV 1999, pp. 87–93 (1999)Google Scholar
  6. 6.
    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, Cambridge (2000)MATHGoogle Scholar
  7. 7.
    Koller, D., Daniilidis, K., Nagel, H.-H.: Model-based object tracking in monocular image sequences of road traffic scenes. IJCV 10(3), 257–281 (1993)CrossRefGoogle Scholar
  8. 8.
    Leibe, B., Schiele, B.: Scale invariant object categorization using a scale-adaptive mean-shift search. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 145–153. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Leibe, B., Seemann, E., Schiele, B.: Pedestrian detection in crowded scenes. In: CVPR 2005 (2005)Google Scholar
  10. 10.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60(2) (2004)Google Scholar
  11. 11.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. PAMI 27(10) (2005)Google Scholar
  12. 12.
    Mittal, A., Davis, L.S.: M2 tracker: A multi-view approach to segmenting and tracking people in a cluttered scene. IJCV 51(3), 183–203 (2003)CrossRefGoogle Scholar
  13. 13.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: a database and web-based tool for image annotation. Technical Report AIM-2005-025, MIT AI Lab (2005)Google Scholar
  14. 14.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47, 7–42 (2002)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Bastian Leibe
    • 1
  • Nico Cornelis
    • 2
  • Kurt Cornelis
    • 2
  • Luc Van Gool
    • 1
    • 2
  1. 1.ETH ZurichSwitzerland
  2. 2.KU LeuvenBelgium

Personalised recommendations