Benchmarking Stereo Data (Not the Matching Algorithms)

  • Ralf Haeusler
  • Reinhard Klette
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6376)


Current research in stereo image analysis focuses on improving matching algorithms in terms of accuracy, computational costs, and robustness towards real-time applicability for complex image data and 3D scenes. Interestingly, performance testing takes place for a huge number of algorithms, but, typically, on very small sets of image data only. Even worse, there is little reasoning whether data as commonly applied is actually suitable to prove robustness or even correctness of a particular algorithm. We argue for the need of testing stereo algorithms on a much broader variety of image data then done so far by proposing a simple measure for putting image stereo data of different quality into relation to each other. Potential applications include purpose-directed decisions for the selection of image stereo data for testing the applicability of matching techniques under particular situations, or for realtime estimation of stereo performance (without any need for providing ground truth) in cases where techniques should be selected depending on the given situation.


Ground Truth Stereo Image Stereo Match Stereo Pair Normalize Cross Correlation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aujol, J.F., Gilboa, G., Chan, T., Osher, S.: Structure-texture image decomposition - modeling, algorithms, and parameter selection. Int. J. Computer Vision 67, 111–136 (2006)CrossRefGoogle Scholar
  2. 2.
    EISATS: enpeda.. image sequence analysis test site,
  3. 3.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Computer Vision 70, 41–54 (2006)CrossRefGoogle Scholar
  4. 4.
    Förstner, W.: 10 pros and cons against performance characterization of vision algorithms. Machine Vision Applications 9, 215–218 (1997)CrossRefGoogle Scholar
  5. 5.
    Foo, J.J., Sinha, R.: Pruning SIFT for scalable near-duplicate image matching. In: Proc. Australasian Database Conf., pp. 63–71 (2007)Google Scholar
  6. 6.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Analysis Machine Intelligence 30, 328–341 (2007)CrossRefGoogle Scholar
  7. 7.
    Hirschmüller, H., Scharstein, D.: Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans. Pattern Analysis Machine Intelligence 31, 1582–1599 (2009)CrossRefGoogle Scholar
  8. 8.
    Hyvärinen, A., Hurri, J., Hoyer, P.: Natural Image Statistics. Springer, Amsterdam (2009)zbMATHCrossRefGoogle Scholar
  9. 9.
    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions via graph cuts. In: Proc. Int. Conf. Computer Vision, pp. 508–515 (2001)Google Scholar
  10. 10.
    Leclerc, Y.G., Luong, Q.-T., Fua, P.V.: Self-consistency, Stereo, MDL, and Change detection. In: Proc. IJCV (2002)Google Scholar
  11. 11.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Computer Vision 60, 91–110 (2004)CrossRefGoogle Scholar
  12. 12.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Analysis Machine Intelligence 10, 1615–1630 (2005)CrossRefGoogle Scholar
  13. 13.
    Morales, S., Klette, R.: A third eye for performance evaluation in stereo sequence analysis. In: Jiang, X., Petkov, N. (eds.) Computer Analysis of Images and Patterns. LNCS, vol. 5702, pp. 1078–1086. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: Proc. IEEE Conf. Computer Vision Pattern Recognition, vol. 2, pp. 2161–2168 (2006)Google Scholar
  15. 15.
    Ohta, Y., Kanade, T.: Stereo by two-level dynamic programming. In: Proc. Int. Joint Conf. Artificial Intelligence, pp. 1120–1126 (1985)Google Scholar
  16. 16.
    Pharr, M., Humphreys, G.: Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann, San Francisco (2004)Google Scholar
  17. 17.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Computer Vision 47, 7–42 (2002)zbMATHCrossRefGoogle Scholar
  18. 18.
    Thacker, N.A., Clark, A.F., Barron, J.L., Beveridge, J.R., Courtney, P., Crum, W.R., Ramesh, V., Clark, C.: Performance characterization in computer vision: A guide to best practices. Computer Vision Image Understanding 109, 305–334 (2008)CrossRefGoogle Scholar
  19. 19.
    Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms (2008),

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Ralf Haeusler
    • 1
  • Reinhard Klette
    • 1
  1. 1.The University of Auckland 

Personalised recommendations