Advertisement

SPMVP: Spatial PatchMatch Stereo with Virtual Pixel Aggregation

  • Peng Yao
  • Hua Zhang
  • Yanbing Xue
  • Shengyong Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10636)

Abstract

Stereo matching is one of the critical problems in the field of computer vision and it has been widely applied to 3D Reconstruction, Image Refocusing and etc. Recently proposed PatchMatch (PM) stereo algorithm effectively overcomes the limitation of integer-value within the support window but it is still inferior in twofold: (1) view propagation of PM stereo algorithm generally yields underwhelming particle propagation; (2) it still suffers from a coarse performance in textureless regions. To mitigate these weaknesses, a Spatial-PM stereo algorithm without view propagation is proposed for improving the original one at first. Then a virtual pixel based cost aggregation framework with two sped-up strategies is proposed for tackling the problem of textureless mismatching. Jointing the two incremental improvements, we name the novel one as Spatial PatchMatch Stereo with Virtual Pixel Aggregation (SPMVP). Experiments show that SPMVP achieves superior results than other four challenging PM based stereo algorithms both in integer & subpixel level accuracy on all 31 Middlebury stereo pairs; and also performs better on Microsoft i2i stereo videos.

Keywords

PatchMatch stereo Particle propagation Virtual pixel Sped-up strategy Subpixel accuracy 

Notes

Acknowledgements

This research has been supported by National Natural Science Foundation of China (U1509207, 61325019, 61472278, 61403281 and 61572357), Key project of Natural Science Foundation of Tianjin (14JCZDJC31700).

References

  1. 1.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002). SpringerCrossRefMATHGoogle Scholar
  2. 2.
    Gong, M., Yang, R., Wang, L., et al.: A performance study on different cost aggregation approaches used in real-time stereo matching. Int. J. Comput. Vis. 75, 283–296 (2007). SpringerCrossRefGoogle Scholar
  3. 3.
    Tombari, F., Mattoccia, S., Stefano, L., et al.: Classification and evaluation of cost aggregation methods for stereo correspondence. In: IEEE International Conference on Computer Vision Pattern Recognition, pp. 1–8 (2008)Google Scholar
  4. 4.
    Wang, L., Yang, R., Gong, M., et al.: Real-time stereo using approximated joint bilateral filtering and dynamic programming. J. Real-Time Image Proc. 9, 447–461 (2014). SpringerCrossRefGoogle Scholar
  5. 5.
    Taniai, T., Matsushita, Y., Naemura, T.: Graph cut based continuous stereo matching using locally shared labels. In: IEEE International Conference on Computer Vision Pattern Recognition, pp. 1613–1620 (2014)Google Scholar
  6. 6.
    Yang, Q., Wang, L., Ahuja, N.: A constant space belief propagation algorithm for stereo matching. In: IEEE International Conference on Computer Vision Pattern Recognition, pp. 1458–1465 (2010)Google Scholar
  7. 7.
    Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. 28, 650–656 (2006)CrossRefGoogle Scholar
  8. 8.
    Hosni, A., Bleyer, M., Gelautz, M., et al.: Local stereo matching using geodesic support weights. In: IEEE International Conference Image Processing, pp. 2093–2096 (2009)Google Scholar
  9. 9.
    Hu, W., Zhang, K., Sun, L., et al.: Virtual support window for adaptive-weight stereo matching. In: IEEE Visual Communications and Image Processing, pp. 1–4 (2011)Google Scholar
  10. 10.
    Min, D., Lu, J., Do, M.N.: A revisit to cost aggregation in stereo matching: how far can we reduce its computational redundancy? In: IEEE International Conference on Computer Vision, pp. 1567–1574 (2011)Google Scholar
  11. 11.
    Min, D., Lu, J., Do, M.N.: Joint histogram based cost aggregation for stereo matching. IEEE Trans. Pattern Anal. 35, 2539–2545 (2013). IEEECrossRefGoogle Scholar
  12. 12.
    Bleyer, M., Breiteneder, C.: Stereo matching—state-of-the-art and research challenges. In: Farinella, G., Battiato, S., Cipolla, R. (eds.) Advanced Topics in Computer Vision. Advances in Computer Vision and Pattern Recognition. Springer, London (2013). doi: 10.1007/978-1-4471-5520-1_6 Google Scholar
  13. 13.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 24:1–24:11 (2009)CrossRefGoogle Scholar
  14. 14.
    Bleyer, M., Rhemann, C., Rother, C.: PatchMatch stereo - stereo matching with slanted support windows. In: British Machine Vision Conference (2011)Google Scholar
  15. 15.
    Besse, F., Rother, C., Fitzgibbon A., et al.: PMBP: PatchMatch belief propagation for correspondence field estimation. In: British Machine Vision Conference (2012)Google Scholar
  16. 16.
    Heise, P., Klose, S., Jensen, B., et al.: PM-Huber: PatchMatch with huber regularization for stereo matching. In: IEEE International Conference on Computer Vision, pp. 2360–2367 (2013)Google Scholar
  17. 17.
    Achanta, R., Shaji, A., Smith, K., et al.: SLIC Superpixels Compared to State-of-The-Art Superpixels Methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012). IEEECrossRefGoogle Scholar
  18. 18.
    Zhang, C., Li, Z., Cai, R., et al.: As-rigid-as-possible stereo under second order smoothness priors. In: European Conference on Computer Vision, pp. 112–126 (2014)Google Scholar
  19. 19.
    Zhang, K., Fang, Y., Min, D., et al.: Cross-scale cost aggregation for stereo matching. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1590–1597 (2014)Google Scholar
  20. 20.
    Zhang, K., Fang, Y., Min, D., et al.: Cross-scale cost aggregation for stereo matching. In: IEEE Transactions on Circuits and Systems for Video Technology (2016)Google Scholar
  21. 21.
    Lu, J., Yang, H., Min D., et al.: PatchMatch filter: efficient edge aware filtering meets randomized search for fast correspondence field estimation. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1854–1861 (2013)Google Scholar
  22. 22.
    Li, Y., Min, D., Brown, M.S., et al.: SPM-BP: sped-up PatchMatch belief propagation for continuous MRFs. In: IEEE International Conference on Computer Vision, pp. 4006–4014 (2015)Google Scholar
  23. 23.
    Lu, L., Shi, K., Min, D., et al.: Cross-based local multipoint filtering. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 430–437 (2012)Google Scholar
  24. 24.
    Christo, R., Hosni, A., Bleyer, M., et al.: Fast cost-volume filtering for visual correspondence and beyond. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3017–3024 (2011)Google Scholar
  25. 25.
    Hosni, A., Christo, R., Bleyer, M., et al.: Fast cost-volume filtering for visual correspondence and beyond. IEEE Trans. Pattern Anal. 32, 504–511 (2013)CrossRefGoogle Scholar
  26. 26.
    Middlebury Online Stereo Evaluation (2016). http://vision.middlebury.edu/stereo/data/
  27. 27.

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Peng Yao
    • 1
    • 2
  • Hua Zhang
    • 1
    • 2
  • Yanbing Xue
    • 1
    • 2
  • Shengyong Chen
    • 1
    • 2
  1. 1.Key Laboratory of Computer Vision and System (Ministry of Education)Tianjin University of TechnologyTianjinChina
  2. 2.Tianjin Key Laboratory of Intelligence Computing and Novel Software TechnologyTianjin University of TechnologyTianjinChina

Personalised recommendations