Skip to main content
SpringerLink
Log in

Real-World Stereo-Analysis Evaluation

  • Conference paper
Book cover Outdoor and Large-Scale Real-World Scene Analysis

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 7474))

  • 1450 Accesses

Abstract

Evaluation of stereo-analysis algorithms is usually done by analysing the performance of stereo matchers on data sets with available ground truth. The trade-off between precise results, obtained with this sort of evaluation, and the limited amount (in both, quantity and diversity) of data sets, needs to be considered if the algorithms are required to analyse real-world environments. This chapter discusses a technique to objectively evaluate the performance of stereo-analysis algorithms using real-world image sequences. The lack of ground truth is tackled by incorporating an extra camera into a multi-view stereo camera system. The relatively simple hardware set-up of the proposed technique can easily be reproduced for specific applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Badino, H., Franke, U., Mester, R.: Free space computation using stochastic occupancy grids and dynamic programming. In: Proc. Dynamic Vision, ICCV Workshop, pp. 1–12 (2007)

    Google Scholar 

  2. Banks, J., Corke, P.: Quantitative evaluation of matching methods and validity measures for stereo vision. Int. J. Robotics Research 20, 512–532 (2001)

    Article  Google Scholar 

  3. Bolles, R., Baker, H., Hannah, M.: The JISCT stereo evaluation. In: ARPA Image Understanding Workshop, pp. 263–274 (1993)

    Google Scholar 

  4. CMU/VASC. Stereo image data base, http://vasc.ri.cmu.edu/idb/html/stereo/ (retrieved 2012)

  5. Computer Vision Group, University of Bonn. Stereo images with ground truth disparity and occlusion, http://www.uni-bonn.de/~uzs75l/MRTStereo/stereo_data/index.html (retrieved 2012)

  6. DAGM 2011, adverse vision condition challenge, http://www.dagm2011.org/adverse-vision-conditions-challenge.html (retrieved 2012)

  7. The .enpeda.. project, The University of Auckland. EISATS, Set 2, http://www.mi.auckland.ac.nz/EISATS (retrieved 2012)

  8. Faugeras, O., Fua, P., Hotz, B., Ma, R., Robert, L., Thonnat, M., Zhang, Z.: Quantitative and qualitative comparison of some area and feature-based stereo-analysis algorithms. In: Proc. Workshop Robust Computer Vision, pp. 1–26 (1992)

    Google Scholar 

  9. Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. Int. J. Computer Vision 70, 41–54 (2006)

    Article  Google Scholar 

  10. Georgescu, B., Meer, P.: Point matching under large image deformations and illumination changes. IEEE Trans. Pattern Anal. Mach. Intel. 26, 674–688 (2004)

    Article  Google Scholar 

  11. Gherardi, R.: Confidence-based cost modulation for stereo matching. In: Proc. ICPR (2008) 978-1-4244-2175-6

    Google Scholar 

  12. Gülch, E.: Results of test on image matching of ISPRS WG III/4. ISPRS J. Photogrammetry Remote Sensing 46, 1–18 (1991)

    Article  Google Scholar 

  13. Haeusler, R., Klette, R.: Benchmarking Stereo Data (Not the Matching Algorithms). In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) DAGM 2010. LNCS, vol. 6376, pp. 383–392. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  14. Hermann, S., Vaudrey, T.: The gradient - a powerful and robust cost function for stereo matching. In: Proc. IVCNZ (2010) 978-1-4244-9631-0

    Google Scholar 

  15. Hermann, S., Morales, S., Klette, R.: Half-resolution semi-global stereo matching. In: Proc. IEEE Symp. IV, pp. 201–206 (2011)

    Google Scholar 

  16. Hermann, S., Klette, R.: Evaluation of a New Coarse-to-Fine Strategy for Fast Semi-Global Stereo Matching. In: Ho, Y.-S. (ed.) PSIVT 2011, Part I. LNCS, vol. 7087, pp. 395–406. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  17. Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: Proc. CVPR, vol. 2, pp. 807–814 (2005)

    Google Scholar 

  18. JISCT Stereo Images, http://vasc.ri.cmu.edu/idb/html/jisct/index.html (retrieved 2012)

  19. Klappstein, J., Vaudrey, T., Rabe, C., Wedel, A., Klette, R.: Moving Object Segmentation using Optical Flow and Depth Information. In: Wada, T., Huang, F., Lin, S. (eds.) PSIVT 2009. LNCS, vol. 5414, pp. 611–623. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  20. Klaus, A., Sormann, M., Karner, K.: Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: Proc. ICPR (2006), 10.1109/ICPR.2006.1033

    Google Scholar 

  21. Klette, R., Schlüns, K., Koschan, A.: Computer vision: three-dimensional data from images. Springer, Singapore (1998)

    MATH  Google Scholar 

  22. Klette, R., Zamperoni, P.: Handbook of Image Processing Operators. John Wiley & Sons, Inc. (1996)

    Google Scholar 

  23. Klette, R., Krüger, N., Vaudrey, T., Pauwels, K., Hulle, M., Morales, S., Kandil, F., Haeusler, R., Pugeault, N., Rabe, C., Leppe, M.: Performance of correspondence algorithms in vision-based driver assistance using an online image sequence database. IEEE Trans. Vehicular Technology 60, 2012–2026 (2011)

    Article  Google Scholar 

  24. Kogler, J., Hemetsberger, H., Alefs, B., Kubinger, W., Travis, W.: Embedded stereo vision system for intelligent autonomous vehicles. In: Proc. IEEE Symp. IV, pp. 64–69 (2006)

    Google Scholar 

  25. Kolmogorov, V., Zabih, R.: Graph cut algorithms for binocular stereo with occlusions. In: Math. Models in Computer Vision: The Handbook, pp. 423–437. Springer (2005)

    Google Scholar 

  26. Kondermann, D., Meister, S., Lauer, P.: An outdoor stereo camera system for the generation of real-world benchmark datasets with ground truth. Universität Heidelberg HCI, Technical Rep. (2011)

    Google Scholar 

  27. Leclerc, Y.G., Luong, Q.-T., Fua, P.: Measuring the Self-Consistency of Stereo Algorithms. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 282–298. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  28. Liu, Z., Klette, R.: Approximate Ground Truth for Stereo and Motion Analysis on Real-World Sequences. In: Wada, T., Huang, F., Lin, S. (eds.) PSIVT 2009. LNCS, vol. 5414, pp. 874–885. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  29. Mohan, R., Medioni, G., Nevatia, R.: Stereo error detection, correction and evaluation. IEEE Trans. Pattern Analysis Machine Intelligence 11, 113–120 (1989)

    Article  Google Scholar 

  30. Morales, S., Klette, R.: A Third Eye for Performance Evaluation in Stereo Sequence Analysis. In: Jiang, X., Petkov, N. (eds.) CAIP 2009. LNCS, vol. 5702, pp. 1078–1086. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  31. Morales, S., Klette, R.: Ground Truth Evaluation of Stereo Algorithms for Real World Applications. In: Koch, R., Huang, F. (eds.) ACCV 2010 Workshops, Part II. LNCS, vol. 6469, pp. 152–162. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  32. Mordohai, P.: The self-aware matching measure for stereo. In: Proc. ICCV, pp. 1841–1848 (2009)

    Google Scholar 

  33. Satoh, Y., Sakaue, K.: An omnidirectional stereo vision-based smart wheelchair. EURASIP J. Image Video Processing, 1–12 (2007)

    Google Scholar 

  34. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Computer Vision 47, 7–42 (2001)

    Article  Google Scholar 

  35. Schauwecker, K., Morales, S., Hermann, S., Klette, R.: A comparative study of stereo-matching algorithms for road-modelling in the presence of windscreen wipers. In: Proc. IEEE Symp. IV, pp. 7–12 (2011)

    Google Scholar 

  36. Steingrube, P., Gehrig, S.K., Franke, U.: Performance Evaluation of Stereo Algorithms for Automotive Applications. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 285–294. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  37. Szeliski, R.: Prediction error as a quality metric for motion and stereo. In: Proc. ICCV, pp. 781–788 (1999)

    Google Scholar 

  38. van der Mark, W., Gavrila, M.: Real-time dense stereo for intelligent vehicles. IEEE Trans. Intelligent Transportation Systems 7, 38–50 (2006)

    Article  Google Scholar 

  39. Vaudrey, T., Rabe, C., Klette, R., Milburn, J.: Differences between stereo and motion behaviour on synthetic and real-world stereo sequences. In: Proc. IVCNZ, pp. 1–6 (2008)

    Google Scholar 

  40. Vaudrey, T., Morales, S., Wedel, A., Klette, R.: Generalized residual images effect on illumination artifact removal for correspondence algorithms. Pattern Recognition 44, 2034–2046 (2011)

    Article  Google Scholar 

  41. Zabih, R., Woodfill, J.: Non-Parametric Local Transforms for Computing Visual Correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994, Part II. LNCS, vol. 801, pp. 151–158. Springer, Heidelberg (1994)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The .enpeda.. Project, The University of Auckland, New Zealand

    Sandino Morales, Simon Hermann & Reinhard Klette

Authors
  1. Sandino Morales
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Hermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Reinhard Klette
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Computing Building, Georgia Institute of Technology, 801 Atlantic Drive, 30332-0280, Atlanta, GA, USA

    Frank Dellaert

  2. Department of Computer Science, University of North Carolina at Chapel Hill, Brooks Computer Science Building, CB#3175, 27599, Chapel Hill, NC, USA

    Jan-Michael Frahm

  3. CVG - Institute of Visual Computing, Department of Computer Science, ETH Zurich, CNB G105, Universitaetstrasse 6, 8092, Zurich, Switzerland

    Marc Pollefeys

  4. Institute for Information Processing (TNT), Leibniz Universität, Appelstr. 9A, 30167, Hannover, Germany

    Laura Leal-Taixé

  5. Institute for Information Processing (TNT), Leibniz Universität, Appelstr. 9A, 30167, Hannover, Germany

    Bodo Rosenhahn

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Morales, S., Hermann, S., Klette, R. (2012). Real-World Stereo-Analysis Evaluation. In: Dellaert, F., Frahm, JM., Pollefeys, M., Leal-Taixé, L., Rosenhahn, B. (eds) Outdoor and Large-Scale Real-World Scene Analysis. Lecture Notes in Computer Science, vol 7474. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34091-8_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34091-8_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34090-1

  • Online ISBN: 978-3-642-34091-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation