Improved Stixel Estimation Based on Transitivity Analysis in Disparity Space

  • Noor Haitham Saleem
  • Hsiang-Jen Chien
  • Mahdi Rezaei
  • Reinhard Klette
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10424)


We present a novel method for stixel construction using a calibrated collinear trinocular vision system. Our method takes three conjugate stereo images at the same time to measure the consistency of disparity values by means of the transitivity error in disparity space. Unlike previous stixel estimation methods that are built based on a single disparity map, our proposed method introduces a multi-map fusion technique to obtain more robust stixel calculations. We also apply a polynomial curve fitting approach to detect an accurate road manifold, using the v-disparity space which is built based on a confidence map, which further supports accurate stixel calculation. Comparing the depth information from the extracted stixels (using stixel maps) with depth measurements obtained from a highly accurate LiDAR range sensor, we evaluate the accuracy of the proposed method. Experimental results indicate a significant improvement of 13.6% in the accuracy of stixel detection compared to conventional binocular vision.


  1. 1.
    Klette, R.: Vision-based driver assistance. In: Wiley Encyclopaedia Electrical Electronics Engineering, pp. 1–15. Wiley (2015)Google Scholar
  2. 2.
    Rezaei, M., Klette, R.: Computer Vision for Driver Assistance. CIV, vol. 45. Springer, Cham (2017)Google Scholar
  3. 3.
    Kaaniche, K., Demonceaux, C., Vasseur, P.: Analysis of low-altitude aerial sequences for road traffic diagnosis using graph partitioning and Markov hierarchical models. In: Proceeding International Multi-conference Systems Signals Devices, pp. 656–661 (2016)Google Scholar
  4. 4.
    Wu, J., Cui, Z., Sheng, V.S., Zhao, P., Su, D., Gong, S.: A comparative study of SIFT and its variants. Measur. Sci. Rev. 13(3), 122–131 (2013)Google Scholar
  5. 5.
    Farabet, C., Couprie, C., Najman, L., Lecun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)CrossRefGoogle Scholar
  6. 6.
    Anders, J., Mefenza, M., Bobda, C., Yonga, F., Aklah, Z., Gunn, K.: A hardware/software prototyping system for driving assistance investigations. J. Real-Time Image Process. 11(3), 559–569 (2016)CrossRefGoogle Scholar
  7. 7.
    Badino, H., Franke, U., Pfeiffer, D.: The stixel world - a compact medium level representation of the 3D-world. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 51–60. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-03798-6_6 CrossRefGoogle Scholar
  8. 8.
    Pfeiffer, D., Franke, U.: Efficient representation of traffic scenes by means of dynamic stixels. In: Processing Intelligent Vehicles Symposium, pp. 217–224 (2010)Google Scholar
  9. 9.
    Scharwächter, T., Enzweiler, M., Franke, U., Roth, S.: Stixmantics: a medium-level model for real-time semantic scene understanding. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 533–548. Springer, Cham (2014). doi: 10.1007/978-3-319-10602-1_35 Google Scholar
  10. 10.
    Schneider, L., Cordts, M., Rehfeld, T., Pfeiffer, D., Enzweiler, M., Franke, U., Pollefeys, M., Roth, S.: Semantic stixels: depth is not enough. In: Proceeding Intelligent Vehicles Symposium, pp. 110–117 (2016)Google Scholar
  11. 11.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceeding of Computer Vision Pattern Recognition, pp. 3354–3361 (2012)Google Scholar
  12. 12.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30, 328–341 (2008)CrossRefGoogle Scholar
  13. 13.
    Shin, B.-S., Xu, Z., Klette, R.: Visual lane analysis and higher-order tasks: a concise review. Mach. Vis. Appl. 25(6), 1519–1547 (2014)CrossRefGoogle Scholar
  14. 14.
    Onoguchi, K., Takeda, N., Watanabe, M.: Obstacle location estimation using planar projection stereopsis method. Syst. Comput. Japan 32(14), 67–76 (2001)CrossRefGoogle Scholar
  15. 15.
    Saleem, N.H., Klette, R.: Accuracy of free-space detection for stereo versus monocular vision. In: Proceeding of Image Vision Computing New Zealand, pp. 48–53 (2016)Google Scholar
  16. 16.
    Seo, J., Oh, C., Sohn, K.: Segment-based free space estimation using plane normal vector in disparity space. In: Proceeding of Connected Vehicles Expo, pp. 144–149 (2015)Google Scholar
  17. 17.
    Sanberg, W.P., Dubbelman, G., de With, P.H.N.: Color-based free-space segmentation using online disparity-supervised learning. In: Proceeding Intelligent Transportation Systems, pp. 906–912 (2015)Google Scholar
  18. 18.
    Spangenberg, R., Langner, T., Adfeldt, S., Rojas, R.: Large scale semi-global matching on the CPU. In: Proceeding IEEE Intelligent Vehicles Symposium, pp. 195–201 (2014)Google Scholar
  19. 19.
    Klette, R.: Concise Computer Vision. Springer, London (2014). doi: 10.1007/978-1-4471-6320-6 CrossRefzbMATHGoogle Scholar
  20. 20.
    Pfeiffer, D.: The stixel world. Doctoral Thesis, Humboldt Universität Berlin (2011)Google Scholar
  21. 21.
    Levi, D., Garnett, N., Fetaya, E.: StixelNet: a deep convolutional network for obstacle detection and road segmentation. In: Proceeding British Machine Vision Conference, vol. 1, p. 12 (2015)Google Scholar
  22. 22.
    Benenson, R., Timofte, R., Van Gool, L.: Stixels estimation without depth map computation. In: Proceeding International Conference Computer Vision Workshops, pp. 2010–2017 (2011)Google Scholar
  23. 23.
    Benenson, R., Mathias, M., Timofte, R., Van Gool, L.: Fast stixels estimation for fast pedestrian detection. In: Proceeding of European Conference Computer Vision, pp. 11–20 (2012)Google Scholar
  24. 24.
    Scharwächter, T., Franke, U.: Low-level fusion of color, texture and depth for robust road scene understanding. In: Proceeding of Intelligent Vehicles Symposium, pp. 599–604 (2015)Google Scholar
  25. 25.
    Cordts, M., Schneider, L., Enzweiler, M., Franke, U., Roth, S.: Object-level priors for stixel generation. In: Proceeding of German Conference on Pattern Recognition, pp. 172–183 (2014)Google Scholar
  26. 26.
    Pfeiffer, D., Gehrig, S., Schneider, N.: Exploiting the power of stereo confidences. In: Proceeding of Conference Computer Vision, Pattern Recognition, pp. 297–304 (2013)Google Scholar
  27. 27.
    Brandao, M., Ferreira, R., Hashimoto, K., Takanishi, A., Santos-Victor, J.: On stereo confidence measures for global methods, evaluation, new model and integration into occupancy grids. In: Proceeding of Pattern Analysis Machine Intelligence, pp. 116–128 (2016)Google Scholar
  28. 28.
    Haeusler, R., Nair, R., Köndermann, D.: Ensemble learning for confidence measures in stereo vision. In: Proceeding of Conference Computer Vision, Pattern Recognition, pp. 305–312 (2013)Google Scholar
  29. 29.
    Chien, H.-J., Geng, H., Klette, R.: Improved visual odometry based on transitivity error in disparity space: a third-eye approach. In: Proceeding Image Vision Computing, New Zealand, pp. 72–77 (2014)Google Scholar
  30. 30.
    Iloie, A., Giosan, I., Nedevschi, S.: UV disparity based obstacle detection and pedestrian classification in urban traffic scenarios. In: Proceeding of Intelligent Computer Communication Processing, pp. 119–125 (2014)Google Scholar
  31. 31.
    Labayrade, R., Aubert, D., Tarel, J.: Real time obstacle detection in stereovision on non flat road geometry through v-disparity representation. In: Proceeding of IEEE Intelligent Vehicle Symposium, pp. 646–651 (2002)Google Scholar
  32. 32.
    Keller, C., Dang, T., Fritz, H., Joos, A., Rabe, C., Gavrila, D.: Active pedestrian safety by automatic braking and evasive steering. In: Proceeding of Intelligent Transportation Systems, pp. 1292–1304 (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Noor Haitham Saleem
    • 1
  • Hsiang-Jen Chien
    • 1
  • Mahdi Rezaei
    • 2
  • Reinhard Klette
    • 1
  1. 1.EEE Department, School of Engineering, Computer, and Mathematical SciencesAuckland University of TechnologyAucklandNew Zealand
  2. 2.CE Department, Faculty of Computer and Information Technology EngineeringQazvin Islamic Azad UniversityQazvinIran

Personalised recommendations