Journal of Mathematical Imaging and Vision

, Volume 38, Issue 1, pp 70–82 | Cite as

Multi-label Depth Estimation for Graph Cuts Stereo Problems

Article

Abstract

We describe here a method to compute the depth of a scene from a set of at least two images taken at known view-points. Our approach is based on an energy formulation of the 3D reconstruction problem which we minimize using a graph-cut approach that computes a local minimum whose energy is comparable (modulo a multiple constant) with the energy of the absolute minimum. As usually done, we treat the input images symmetrically, match pixels using photoconsistency, treat occlusion and visibility problems and consider a spatial regularization term which preserves discontinuities. The details of the graph construction as well as the proof of the correctness of the method are given. Moreover we introduce a multi-label refinement algorithm in order to increase the number of depth labels without significantly increasing the computational complexity. Finally we compared our algorithm with the results available in the Middlebury database.

Keywords

Depth estimation Disparity Graph cut Multi label refinement 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: Proceedings of the Second European Conference on Computer Vision, pp. 237–252. Springer, Berlin (1992) Google Scholar
  2. 2.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996) CrossRefGoogle Scholar
  3. 3.
    Bobick, A.F., Intille, S.S.: Large occlusion stereo. Int. J. Comput. Vis. 33(3), 181–200 (1999) CrossRefGoogle Scholar
  4. 4.
    Boykov, Y., Veksler, O., Zabih, R.: Markov random fields with efficient approximations. In: IEEE Conf. Comp. Vis. and Pat. Recog. (CVPR’98), pp. 648–655 (1998) Google Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001) CrossRefGoogle Scholar
  6. 6.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: European Conference on Comput. Vis. (ECCV’04). Springer, Berlin (2004) Google Scholar
  7. 7.
    Collins, R.: A space-sweep approach to true multi-image matching. In: Proc. Comp. Vis. and Pat. Recog. Conf. (CVPR’96), pp. 358–363 (1996) Google Scholar
  8. 8.
    Ford, L.R., Fulkerson, D.R.: Maximal flow through a network. Can. J. Math. 8, 399–404 (1956) MATHMathSciNetGoogle Scholar
  9. 9.
    Fua, P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Mach. Vis. Appl. 6(1), 35–49 (1993) CrossRefGoogle Scholar
  10. 10.
    Gu, Z., Su, X., Liu, Y., Zhang, Q.: Local stereo matching with adaptive support-weight, rank transform and disparity calibration. Pattern Recogn. Lett. Arch. 29(9), 1230–1235 (2008) CrossRefGoogle Scholar
  11. 11.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003) Google Scholar
  12. 12.
    Ishikawa, H.: Exact optimization for Markov random fields with convex priors. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1333–1336 (2003) CrossRefGoogle Scholar
  13. 13.
    Ishikawa, H., Geiger, D.: Occlusions, discontinuities, and epipolar lines in stereo. In: Europ. Conf. Comp. Vis. (ECCV’98), vol. 1, pp. 232–248 (1998) Google Scholar
  14. 14.
    Kanade, T., Okutomi, M.: A stereo matching algorithm with an adaptive window: Theory and experiment. IEEE Trans. Pattern Anal. Mach. Intell. 16(9), 920–932 (1994) CrossRefGoogle Scholar
  15. 15.
    Kang, S.B., Szeliski, R.: Extracting view-dependent depth maps from a collection of images. Int. J. Comput. Vis. 58, 139–163 (2004) CrossRefGoogle Scholar
  16. 16.
    Klaus, A., Sormann, M., Karner, K.: Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: Int. Conf. on Pat. Recogn. (ICPR ’06), pp. 15–18 (2006) Google Scholar
  17. 17.
    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions via graph cuts. In: IEEE Int. Conf. Comp. Vis. (ICCV’01), vol. 2, pp. 508–515 (2001) Google Scholar
  18. 18.
    Kolmogorov, V., Zabih, R.: Multi-camera scene reconstruction via graph cuts. In: Europ. Conf. Comp. Vis. (ECCV’02), pp. 82–96 (2002) Google Scholar
  19. 19.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004) CrossRefGoogle Scholar
  20. 20.
    Kolmogorov, V., Zabih, R., Gortler, S.: Generalized multi-camera scene reconstruction using graph cuts. In: Energy Minimization Methods in Comp. Vis. and Pat. Recog., pp. 501–516 (2003) Google Scholar
  21. 21.
    Mémin, E., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. Int. J. Comput. Vis. 46(2), 129–155 (2002) MATHCrossRefGoogle Scholar
  22. 22.
    Papadakis, N., Caselles, V.: Multi-label depth estimation for graph cuts stereo problems. http://www.dtic.upf.edu/~vcaselles/technicalreport2Web.pdf (2009)
  23. 23.
    Roy, S., Cox, I.J.: A maximum flow formulation of the n-camera stereo correspondence problem. In: IEEE Int. Conf. Comp. Vis. (ICCV’98), pp. 492–502 (1998) Google Scholar
  24. 24.
    Sarkis, M., Dörfler, N., Diepold, K.: Fast adaptive graph-cuts based stereo matching. In: Advanced Concepts for Intel. Vis. Sys. (ACIVS’07), pp. 818–827 (2007) Google Scholar
  25. 25.
    Scharstein, D.: High-accuracy stereo depth maps using structured light. In: Conf. on Comp. Vis. and Pat. Recog. (CVPR’03), pp. 195–202 (2003) Google Scholar
  26. 26.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002) MATHCrossRefGoogle Scholar
  27. 27.
    Sun, J., Li, Y., Kang, S.B., Shum, H.Y.: Symmetric stereo matching for occlusion handling. In: Conf. Comp. Vis. and Pat. Recogn. (CVPR’05), vol. 2, pp. 399–406 (2005) Google Scholar
  28. 28.
    Taguchi, Y., Wilburn, B., Zitnick, L.: Stereo reconstruction with mixed pixels using adaptive over-segmentation. In: Conf. Comp. Vis. and Pat. Recog. (CVPR’08) (2008) Google Scholar
  29. 29.
    Wang, Z.F., Zheng, Z.G.: A region based stereo matching algorithm using cooperative optimization. In: Conf. Comp. Vis. and Pat. Recog. (CVPR’08) (2008) Google Scholar
  30. 30.
    Xu, L., Jia, J.: Stereo matching: An outlier confidence approach. In: Europ. Conf. Comp. Vis. (ECCV’06) (2008) Google Scholar
  31. 31.
    Yáng, Q., Wang, L., Yang, R., Stewénius, H., Nistér, D.: Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. IEEE Trans. Pattern Recogn. Mach. Intell. 31(3), 492–504 (2009) CrossRefGoogle Scholar
  32. 32.
    Zitnick, C.L., Kang, S.B.: Stereo for image-based rendering using image over-segmentation. Int. J. Comput. Vis. 75(1), 49–65 (2007) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Barcelona MediaBarcelonaSpain
  2. 2.Universitat Pompeu FabraBarcelonaSpain

Personalised recommendations