Pixels, Stixels, and Objects

  • David Pfeiffer
  • Friedrich Erbs
  • Uwe Franke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7585)


Dense stereo vision has evolved into a powerful foundation for the next generation of intelligent vehicles. The high spatial and temporal resolution allows for robust obstacle detection in complex inner city scenarios, including pedestrian recognition and detection of partially hidden moving objects. Aiming at a vision architecture for efficiently solving an increasing number of vision tasks, the medium-level representation named Stixel World has been developed. This paper shows how this representation forms the foundation for a very efficient, robust and comprehensive understanding of traffic scenes. A recently proposed Stixel computation scheme allows the extraction of multiple objects per image column and generates a segmentation of the input data. The motion of the Stixels is obtained by applying the 6D-Vision principle to track Stixels over time. Subsequently, this allows for an optimal Stixel grouping such that all dynamic objects can be detected easily. Pose and motion of moving Stixel groups are used to initialize more specific object trackers. Moreover, appearance-based object recognition highly benefits from the attention control offered by the Stixel World, both in performance and efficiency.


Motion State Intelligent Vehicle Occupancy Grid Dense Stereo IEEE Intelligent Vehicle Symposium 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Badino, H., Franke, U., Mester, R.: Free space computation using stochastic occupancy grids and dynamic programming. In: Workshop on Dynamical Vision, ICCV, Rio de Janeiro, Brazil (October 2007)Google Scholar
  2. 2.
    Badino, H., Franke, U., Pfeiffer, D.: The Stixel World - A Compact Medium Level Representation of the 3D-World. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 51–60. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Barth, A., Pfeiffer, D., Franke, U.: Vehicle tracking at urban intersections using dense stereo. In: 3rd Workshop on Behaviour Monitoring and Interpretation, BMI, Ghent, Belgium, pp. 47–58 (November 2009)Google Scholar
  4. 4.
    Bellman, R.: Dynamic Programming. Princeton University Press (1957)Google Scholar
  5. 5.
    Benenson, R., Timofte, R., Van Gool, L.: Stixels estimation without depth map computation. In: IEEE CVVT: E2M at ICCV (November 2011)Google Scholar
  6. 6.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. In: ICCV, Kerkyra, Corfu, Greece, pp. 377–384 (September 1999)Google Scholar
  7. 7.
    Elfes, A.E.: Sonar-based real-world mapping and navigation. Journal of Robotics and Automation 3(3), 249–265 (1987)CrossRefGoogle Scholar
  8. 8.
    Enzweiler, M., Hummel, M., Pfeiffer, D., Franke, U.: Efficient stixel-based object recognition. In: IEEE Intelligent Vehicles Symposium, Alcalá de Henares, Spain (June 2012)Google Scholar
  9. 9.
    Erbs, F., Franke, U.: Stixmentation - probabilistic stixel based traffic scene labeling. In: BMVC, Guildford, UK. BMVA Press (September 2012)Google Scholar
  10. 10.
    Felzenszwalb, P.F., Veksler, O.: Tiered scene labeling with dynamic programming. In: IEEE CVPR, San Francisco, CA, USA, pp. 3097–3104 (June 2010)Google Scholar
  11. 11.
    Franke, U., Rabe, C., Badino, H., Gehrig, S.: 6D-Vision: Fusion of Stereo and Motion for Robust Environment Perception. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 216–223. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Gallup, D., Pollefeys, M., Frahm, J.-M.: 3D Reconstruction Using an n-Layer Heightmap. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) DAGM 2010. LNCS, vol. 6376, pp. 1–10. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Gehrig, S.K., Eberli, F., Meyer, T.: A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 134–143. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE CVPR, San Diego, CA, USA, pp. 807–814 (June 2005)Google Scholar
  15. 15.
    Hoiem, D., Efros, A.A., Hebert, M.: Geometric context from a single image. In: ICCV, pp. 654–661 (2005)Google Scholar
  16. 16.
    Kolmogorov, V., Rother, C.: Minimizing nonsubmodular functions with graph cuts-a review. IEEE Trans. on PAMI 29(7), 1274–1279 (2007)CrossRefGoogle Scholar
  17. 17.
    Kweon, I.S., Kanade, T.: High-resolution terrain map from multiple sensor data. IEEE Trans. on PAMI 14, 278–292 (1992)CrossRefGoogle Scholar
  18. 18.
    Lacroix, S., Kyun Jung, I., Mallet, A.: Digital elevation map building from low altitude stereo imagery. In: Int. SIRS (2001)Google Scholar
  19. 19.
    Liu, X., Veksler, O., Samarabandu, J.: Order-preserving moves for graph-cut-based optimization. IEEE Trans. on PAMI 32(7), 1182–1196 (2010)CrossRefGoogle Scholar
  20. 20.
    Moravec, H.P.: Robot spatial perception by stereoscopic vision and 3D evidence grids. Technical Report CMU-RI-TR-96-34, Carnegie Mellon University (1996)Google Scholar
  21. 21.
    Muffert, M., Milbich, T., Pfeiffer, D., Franke, U.: May I enter the roundabout? a time-to-contact computation based on stereo-vision. In: IEEE IV, Alcalá de Henares, Spain (June 2012)Google Scholar
  22. 22.
    Oniga, F., Nedevschi, S.: Curb detection for driving assistance systems: A cubic spline-based approach. In: IEEE IV, Baden-Baden, Germany, pp. 945–950 (June 2011)Google Scholar
  23. 23.
    Oniga, F., Nedevschi, S., Meinecke, M.-M., To, T.B.: Road surface and obstacle detection based on elevation maps from dense stereo. In: IEEE ITSC, Seattle, WA, USA (September 2007)Google Scholar
  24. 24.
    Pfeiffer, D., Franke, U.: Efficient representation of traffic scenes by means of dynamic Stixels. In: IEEE IV, San Diego, CA, USA, pp. 217–224 (June 2010)Google Scholar
  25. 25.
    Pfeiffer, D., Franke, U.: Towards a global optimal multi-layer Stixel representation of dense 3D data. In: BMVC, Dundee, Scotland. BMVA Press (August 2011)Google Scholar
  26. 26.
    Scharstein, D., Szeliski, R.: Middlebury online stereo evaluation (2002),
  27. 27.
    Siegemund, J., Pfeiffer, D., Franke, U., Förstner, W.: Curb reconstruction using conditional random fields. In: IEEE IV, San Diego, CA, USA, pp. 203–210 (June 2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • David Pfeiffer
    • 1
  • Friedrich Erbs
    • 1
  • Uwe Franke
    • 1
  1. 1.Research & DevelopmentDaimler AGSindelfingenGermany

Personalised recommendations