Accelerating Cost Volume Filtering Using Salient Subvolumes and Robust Occlusion Handling

  • Mohamed A. HelalaEmail author
  • Faisal Z. Qureshi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9004)


Several fundamental computer vision problems, such as depth estimation from stereo, optical flow computation, etc., can be formulated as a discrete pixel labeling problem. Traditional Markov Random Fields (MRF) based solutions to these problems are computationally expensive. Cost Volume Filtering (CF) presents a compelling alternative. Still these methods must filter the entire cost volume to arrive at a solution. In this paper, we propose a new CF method for depth estimation by stereo. First, we propose the Accelerated Cost Volume Filtering (ACF) method which identifies salient subvolumes in the cost volume. Filtering is restricted to these subvolumes, resulting in significant performance gains. The proposed method does not consider the entire cost volume and results in a marginal increase in unlabeled (occluded) pixels. We address this by developing an Occlusion Handling (OH) technique, which uses superpixels and performs label propagation via a simulated annealing inspired method. We evaluate the proposed method (ACF+OH) on the Middlebury stereo benchmark and on high resolution images from Middlebury 2005/2006 stereo datasets, and our method achieves state-of-the-art results. Our occlusion handling method, when used as a post-processing step, also significantly improves the accuracy of two recent cost volume filtering methods.


  1. 1.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Lu, J., Shi, K., Min, D., Lin, L., Do, M.: Cross-based local multipoint filtering. In: Proceedings of the IEEE CVPR, pp. 430–437 (2012)Google Scholar
  3. 3.
    Hosni, A., Rhemann, C., Bleyer, M., Rother, C., Gelautz, M.: Fast cost-volume filtering for visual correspondence and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 25, 504–511 (2013)CrossRefGoogle Scholar
  4. 4.
    Ben-Ari, R., Sochen, N.: Stereo matching with mumford-shah regularization and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2071–2084 (2010)CrossRefGoogle Scholar
  5. 5.
    Delong, A., Osokin, A., Isack, H., Boykov, Y.: Fast approximate energy minimization with label costs. Int. J. Comput. Vis. 96, 1–27 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001)CrossRefGoogle Scholar
  7. 7.
    Weiss, Y., Freeman, W.: On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Trans. Inf. Theory 47, 723–735 (2001)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Sun, J., Zheng, N., Shum, H.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25, 787–800 (2003)CrossRefGoogle Scholar
  9. 9.
    Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70, 41–54 (2006)CrossRefGoogle Scholar
  10. 10.
    Lu, J., Yang, H., Min, D., Do, M.: Patch match filter: efficient edge-aware filtering meets randomized search for fast correspondence field estimation. In: Proceedings of the IEEE CVPR, pp. 1854–1861 (2013)Google Scholar
  11. 11.
    Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell. 28, 650–656 (2006)CrossRefGoogle Scholar
  12. 12.
    Richardt, C., Orr, D., Davies, I., Criminisi, A., Dodgson, N.A.: Real-time spatiotemporal stereo matchingusing the dual-cross-bilateral grid. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 510–523. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  13. 13.
    Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: Proceedings of the IEEE CVPR, pp. 1–8 (2007)Google Scholar
  14. 14.
    Schick, A., Bauml, M., Stiefelhagen, R.: Improving foreground segmentations with probabilistic superpixel markov random fields. In: Proceedings of the IEEE CVPRW, pp. 27–31 (2012)Google Scholar
  15. 15.
    Granville, V., Krivanek, M., Rasson, J.: Simulated annealing: a proof of convergence. IEEE Trans. Pattern Anal. Mach. Intell. 16, 652–656 (1994)CrossRefGoogle Scholar
  16. 16.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Brown, M., Hua, G., Winder, S.: Discriminative learning of local image descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 33, 43–57 (2011)CrossRefGoogle Scholar
  18. 18.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012)CrossRefGoogle Scholar
  19. 19.
    Min, D., Sohn, K.: Cost aggregation and occlusion handling with WLS in stereo matching. IEEE Trans. Image Process. 17, 1431–1442 (2008)CrossRefMathSciNetGoogle Scholar
  20. 20.
    He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1397–1409 (2013)CrossRefGoogle Scholar
  21. 21.
    Paris, S., Kornprobst, P., Tumblin, J., Durand, F.: Bilateral filtering: theory and applications. Found. Trends Comput. Graph. Vis. 4, 1–73 (2009)CrossRefGoogle Scholar
  22. 22.
    Min, D., Lu, J., Do, M.: A revisit to cost aggregation in stereo matching: how far can we reduce its computational redundancy? In: Proceedings of the IEEE ICCV, pp. 1567–1574 (2011)Google Scholar
  23. 23.
    Boufama, B., Jin, K.: Towards a fast and reliable dense matching algorithm. Soc. Manuf. Eng. J. (2003)Google Scholar
  24. 24.
    Sun, J., Li, Y., Kang, S., Shum, H.Y.: Symmetric stereo matching for occlusion handling. In: Proceedings of the IEEE CVPR, vol. 2, pp. 399–406 (2005)Google Scholar
  25. 25.
    Yang, Q., Wang, L., Yang, R., Stewenius, H., Nister, D.: Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 31, 492–504 (2009)CrossRefGoogle Scholar
  26. 26.
    Gallup, D., Frahm, J.M., Mordohai, P., Qingxiong, Y., Pollefeys, M.: Real-time plane-sweeping stereo with multiple sweeping directions. In: Proceedings of the IEEE CVPR, pp. 1–8 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Faculty of ScienceUniversity of Ontario Institute of TechnologyOshawaCanada

Personalised recommendations