Real-Time Spatiotemporal Stereo Matching Using the Dual-Cross-Bilateral Grid

  • Christian Richardt
  • Douglas Orr
  • Ian Davies
  • Antonio Criminisi
  • Neil A. Dodgson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6313)

Abstract

We introduce a real-time stereo matching technique based on a reformulation of Yoon and Kweon’s adaptive support weights algorithm [1]. Our implementation uses the bilateral grid to achieve a speedup of 200× compared to a straightforward full-kernel GPU implementation, making it the fastest technique on the Middlebury website. We introduce a colour component into our greyscale approach to recover precision and increase discriminability. Using our implementation, we speed up spatial-depth superresolution 100×. We further present a spatiotemporal stereo matching approach based on our technique that incorporates temporal evidence in real time (> 14 fps). Our technique visibly reduces flickering and outperforms per-frame approaches in the presence of image noise. We have created five synthetic stereo videos, with ground truth disparity maps, to quantitatively evaluate depth estimation from stereo video. Source code and datasets are available on our project website.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-642-15558-1_37_MOESM1_ESM.avi (5.9 mb)
Electronic Supplementary Material (6,085 KB)

References

  1. 1.
    Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. PAMI 28, 650–656 (2006)Google Scholar
  2. 2.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI 23, 1222–1239 (2001)Google Scholar
  3. 3.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Effcient belief propagation for early vision. IJCV 70, 41–54 (2006)Google Scholar
  4. 4.
    Gong, M., Yang, R., Wang, L., Gong, M.: A performance study on different cost aggregation approaches used in real-time stereo matching. IJCV 75, 283–296 (2007)Google Scholar
  5. 5.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 42, 7–42 (2002)Google Scholar
  6. 6.
    Yang, Q., Yang, R., Davis, J., Nistér, D.: Spatial-depth super resolution for range images. In: Proc. CVPR (2007)Google Scholar
  7. 7.
    Egnal, G., Wildes, R.P.: Detecting binocular half-occlusions: Empirical comparisons of five approaches. PAMI 24, 1127–1133 (2002)Google Scholar
  8. 8.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. ICCV, pp. 839–846 (1998)Google Scholar
  9. 9.
    Paris, S., Kornprobst, P., Tumblin, J., Durand, F.: A gentle introduction to bilateral filtering and its applications. In: SIGGRAPH Classes (2008) Course material available online at http://people.csail.mit.edu/sparis/bf_course
  10. 10.
    Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26, 103 (2007)CrossRefGoogle Scholar
  11. 11.
    Pham, T., van Vliet, L.: Separable bilateral filtering for fast video preprocessing. In: Proc. ICME (2005)Google Scholar
  12. 12.
    Weiss, B.: Fast median and bilateral filtering. ACM Trans. Graph. 25, 519–526 (2006)CrossRefGoogle Scholar
  13. 13.
    Yang, Q., Tan, K.H., Ahuja, N.: Real-time O(1) bilateral filtering. In: Proc. CVPR (2009)Google Scholar
  14. 14.
    Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. IJCV 81, 24–52 (2009)CrossRefGoogle Scholar
  15. 15.
    Wang, L., Liao, M., Gong, M., Yang, R., Nistér, D.: High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In: Proc. 3DPVT, pp. 798–805 (2006)Google Scholar
  16. 16.
    Gong, M., Yang, Y.H.: Near real-time reliable stereo matching using programmable graphics hardware. In: Proc. CVPR, pp. 924–931 (2005)Google Scholar
  17. 17.
    Yang, Q., Engels, C., Akbarzadeh, A.: Near real-time stereo for weakly-textured scenes. In: Proc. BMVC (2008)Google Scholar
  18. 18.
    Davis, J., Nehab, D., Ramamoorthi, R., Rusinkiewicz, S.: Spacetime stereo: a unifying framework for depth from triangulation. PAMI 27, 296–302 (2005)Google Scholar
  19. 19.
    Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. 23, 548–558 (2004)CrossRefGoogle Scholar
  20. 20.
    Paris, S.: Edge-preserving smoothing and mean-shift segmentation of video streams. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 460–473. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  21. 21.
    Yang, R., Pollefeys, M., Li, S.: Improved real-time stereo on commodity graphics hardware. In: Proc. CVPR Workshops, pp. 36–36 (2004)Google Scholar
  22. 22.
    Adams, A., Baek, J., Davis, A.: Fast high-dimensional filtering using the permu- tohedral lattice. Comp. Graph. Forum 29, 753–762 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Christian Richardt
    • 1
  • Douglas Orr
    • 1
  • Ian Davies
    • 1
  • Antonio Criminisi
    • 2
  • Neil A. Dodgson
    • 1
  1. 1.University of Cambridge, United Kingdom 
  2. 2.Microsoft Research CambridgeUnited Kingdom

Personalised recommendations