Advertisement

Real-Time Edge-Sensitive Local Stereo Matching with Iterative Disparity Refinement

  • Maarten Dumont
  • Patrik GoortsEmail author
  • Steven Maesen
  • Gauthier Lafruit
  • Philippe Bekaert
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 554)

Abstract

First, we present a novel cost aggregation method for stereo matching that uses two edge-sensitive shape-adaptive support windows per pixel region; one following the horizontal edges in the image, the other the vertical edges. Their combination defines the final aggregation window shape that closely follows all object edges and thereby achieves increased hypothesis confidence. Second, we present a novel iterative disparity refinement process and apply it to the initially estimated disparity map. The process consists of four rigorously defined and lightweight modules that can be iterated multiple times: a disparity cross check, bitwise fast voting, invalid disparity handling, and median filtering. We demonstrate that our iterative refinement has a large effect on the overall quality, resulting in smooth disparity maps with sharp object edges, especially around occluded areas. It can be applied to any stereo matching algorithm and tends to converge to a final solution. Finally, we perform a quantitative evaluation on various Middlebury datasets, showing an increase in quality of over several dB PSNR compared with their ground truth. Our whole disparity estimation algorithm supports efficient GPU implementation to facilitate scalability and real-time performance.

Keywords

Local stereo matching Disparity estimation Iterative disparity refinement Edge-sensitive aggregation windows Real-time Bi-directional cross-based 

References

  1. 1.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47, 7–42 (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Dumont, M., Goorts, P., Maesen, S., Degraen, D., Bekaert, P., Lafruit, G.: Iterative refinement for real-time local stereo matching. In: Proceedings of 3D Stereo Media, Liege, Belgium, pp. 1–8 (2014)Google Scholar
  3. 3.
    Zhang, K., Lu, J., Lafruit, G.: Cross-based local stereo matching using orthogonal integral images. IEEE Trans. Circuits Syst. Video Technol. 19, 1073–1079 (2009)CrossRefGoogle Scholar
  4. 4.
    Dumont, M., Goorts, P., Maesen, S., Bekaert, P., Lafruit, G.: Real-time local stereo matching using edge sensitive adaptive windows. In: Proceedings of the 11th International Conference on Signal Processing and Multimedia Applications (SIGMAP 2014), pp. 117–126 (2014)Google Scholar
  5. 5.
    Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2003), vol. 1, IEEE, pp. 195–202 (2003)Google Scholar
  6. 6.
    Ryoo, S., Rodrigues, C., Baghsorkhi, S., Stone, S., Kirk, D., Hwu, W. M.: Optimization principles and application performance evaluation of a multithreaded GPU using CUDA. In: Proceedings of ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 73–82 (2008)Google Scholar
  7. 7.
    Goorts, P., Rogmans, S., Eynde, S. V., Bekaert, P.: Practical examples of gpu computing optimization principles. In: Proceedings of International Conference on Signal Processing and Multimedia Applications, Athens, Greece, pp. 46–49 (2010)Google Scholar
  8. 8.
    Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 211–220. IEEE Computer Society, Madison (2003)Google Scholar
  9. 9.
    Veksler, O.: Fast variable window for stereo correspondence using integral images. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, vol. 1, pp. 556–561 (2003)Google Scholar
  10. 10.
    Hirschmüller, H., Innocent, P.R., Garibaldi, J.: Real-time correlation-based stereo vision with reduced border errors. Int. J. Comput. Vision 47, 229–246 (2002)CrossRefzbMATHGoogle Scholar
  11. 11.
    Yang, R., Pollefeys, M., Li, S.: Improved real-time stereo on commodity graphics hardware. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition Workshop, pp. 36–36 (2004)Google Scholar
  12. 12.
    Lu, J., Rogmans, S., Lafruit, G., Catthoor, F.: Stream-centric stereo matching and view synthesis: a high-speed image-based rendering paradigm on gpus, signal processing: image communication. Trans. Circuit Syst. Video Technol. 19(11), 1598–1611 (2009)CrossRefGoogle Scholar
  13. 13.
    Gerrits, M., Bekaert, P.: Local stereo matching with segmentation-based outlier rejection. In: The 3rd Canadian Conference on Computer and Robot Vision, 2006, IEEE, pp. 66–66 (2006)Google Scholar
  14. 14.
    Wang, Z.F., Zheng, Z.G.: A region based stereo matching algorithm using cooperative optimization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 1–8 (2008)Google Scholar
  15. 15.
    Gong, M., Yang, R.: Image-gradient-guided real-time stereo on graphics hardware. In: Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, 3DIM 2005, pp. 548–555 (2005)Google Scholar
  16. 16.
    Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell. 28, 650–656 (2006)CrossRefGoogle Scholar
  17. 17.
    Rhemann, C., Hosni, A., Bleyer, M., Rother, C., Gelautz, M.: Fast cost-volume filtering for visual correspondence and beyond. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 3017–3024 (2011)Google Scholar
  18. 18.
    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: Proceedings, of the Eighth IEEE International Conference on Computer Vision, 2001, ICCV 2001, vol. 2, IEEE 508–515 (2001)Google Scholar
  19. 19.
    Sun, J., Zheng, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25, 787–800 (2003)CrossRefGoogle Scholar
  20. 20.
    Yang, Q., Wang, L., Yang, R., Stewenius, H., Nister, D.: Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 31, 492–504 (2009)CrossRefGoogle Scholar
  21. 21.
    Wang, L., Liao, M., Gong, M., Yang, R., Nister, D.: High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In: Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’06), 3DPVT 2006, pp. 798–805. IEEE Computer Society, Washington (2006)Google Scholar
  22. 22.
    Zitnick, C., Kang, S.: Stereo for image-based rendering using image over-segmentation. Int. J. Comput. Vis. 75, 49–65 (2007)CrossRefGoogle Scholar
  23. 23.
    Davis, J., Nehab, D., Ramamoorthi, R., Rusinkiewicz, S.: Spacetime stereo: a unifying framework for depth from triangulation. IEEE Trans. Pattern Anal. Mach. Intell. 27, 296–302 (2005)CrossRefGoogle Scholar
  24. 24.
    Zhang, K., Lu, J., Lafruit, G., Lauwereins, R., Van Gool, L.: Real-time accurate stereo with bitwise fast voting on cuda. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), IEEE, pp. 794–800 (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Maarten Dumont
    • 1
  • Patrik Goorts
    • 1
    Email author
  • Steven Maesen
    • 1
  • Gauthier Lafruit
    • 2
  • Philippe Bekaert
    • 1
  1. 1.Hasselt University - tUL - iMinds, Expertise Centre for Digital MediaDiepenbeekBelgium
  2. 2.LISA DepartmentUniversité Libre de Bruxelles/Brussels UniversityBrusselsBelgium

Personalised recommendations