Probabilistic Multi-class Scene Flow Segmentation for Traffic Scenes

  • Alexander Barth
  • Jan Siegemund
  • Annemarie Meißner
  • Uwe Franke
  • Wolfgang Förstner
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6376)


A multi-class traffic scene segmentation approach based on scene flow data is presented. Opposed to many other approaches using color or texture features, our approach is purely based on dense depth and 3D motion information. Using prior knowledge on tracked objects in the scene and the pixel-wise uncertainties of the scene flow data, each pixel is assigned to either a particular moving object class (tracked/unknown object), the ground surface, or static background. The global topological order of classes, such as objects are above ground, is locally integrated into a conditional random field by an ordering constraint. The proposed method yields very accurate segmentation results on challenging real world scenes, which we made publicly available for comparison.


Ground Surface Object Class Conditional Random Field Unary Potential Lead Vehicle 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wojek, C., Schiele, B.: A dynamic conditional random field model for joint labeling of object and scene classes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 733–747. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Ess, A., Müller, T., Grabner, H., van Gool, L.: Segmentation-based urban traffic scene understanding. In: BMVC (2009)Google Scholar
  3. 3.
    Brox, T., Rousson, M., Deriche, R., Weickert, J.: Colour, texture, and motion in level set based segmentation and tracking. Image & Vision Comp. 28 (2010)Google Scholar
  4. 4.
    Sturgess, P., Alahari, K., Ladicky, L., Torr, P.: Combining appearance and structure from motion features for road scene understanding. In: BMVC 2009 (2009)Google Scholar
  5. 5.
    Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Rabe, C., Müller, T., Wedel, A., Franke, U.: Dense, robust, and accurate 3D motion field estimation from stereo image sequences in real-time. In: Daniilidis, K. (ed.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 582–595. Springer, Heidelberg (2010)Google Scholar
  7. 7.
    Wedel, A., Meißner, A., Rabe, C., Franke, U., Cremers, D.: Detection and segmentation of independently moving objects from dense scene flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 14–27. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Bachmann, A.: Applying recursive EM to scene segmentation. In: Denzler, J., Notni, G., Süße, H. (eds.) Pattern Recognition. LNCS, vol. 5748, pp. 512–521. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. 9.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: ICML, pp. 282–289 (2001)Google Scholar
  10. 10.
    MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  11. 11.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M.F., Rother, C.: A comparative study of energy minimization methods for markov random fields. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part II. LNCS, vol. 3952, pp. 16–29. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Ramalingam, S., Kohli, P., Alahari, K., Torr, P.H.S.: Exact inference in multi-label CRFs with higher-order cliques. In: CVPR, pp. 1–8 (2008)Google Scholar
  13. 13.
    Barth, A., Franke, U.: Estimating the driving state of oncoming vehicles from a moving platform using stereo vision. IEEE Trans. on ITS 10(4), 560–571 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Alexander Barth
    • 1
  • Jan Siegemund
    • 1
  • Annemarie Meißner
    • 2
  • Uwe Franke
    • 2
  • Wolfgang Förstner
    • 1
  1. 1.Department of PhotogrammetryUniversity of BonnGermany
  2. 2.Daimler AG, Group ResearchSindelfingenGermany

Personalised recommendations