Advertisement

Beyond Local Reasoning for Stereo Confidence Estimation with Deep Learning

  • Fabio Tosi
  • Matteo Poggi
  • Antonio Benincasa
  • Stefano Mattoccia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11210)

Abstract

Confidence measures for stereo gained popularity in recent years due to their improved capability to detect outliers and the increasing number of applications exploiting these cues. In this field, convolutional neural networks achieved top-performance compared to other known techniques in the literature by processing local information to tell disparity assignments from outliers. Despite this outstanding achievements, all approaches rely on clues extracted with small receptive fields thus ignoring most of the overall image content. Therefore, in this paper, we propose to exploit nearby and farther clues available from image and disparity domains to obtain a more accurate confidence estimation. While local information is very effective for detecting high frequency patterns, it lacks insights from farther regions in the scene. On the other hand, enlarging the receptive field allows to include clues from farther regions but produces smoother uncertainty estimation, not particularly accurate when dealing with high frequency patterns. For these reasons, we propose in this paper a multi-stage cascaded network to combine the best of the two worlds. Extensive experiments on three datasets using three popular stereo algorithms prove that the proposed framework outperforms state-of-the-art confidence estimation techniques.

Keywords

Confidence measures Stereo matching Deep learning 

Notes

Acknowledgement

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research. We also thank Alessandro Fusco for his preliminar experiments on the ConfNet architecture.

References

  1. 1.
    Poggi, M., Tosi, F., Mattoccia, S.: Quantitative evaluation of confidence measures in a machine learning world. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar
  2. 2.
    Poggi, M., Mattoccia, S.: Learning from scratch a confidence measure. In: Proceedings of the 27th British Conference on Machine Vision, BMVC (2016)Google Scholar
  3. 3.
    Seki, A., Pollefeys, M.: Patch based confidence prediction for dense disparity map. In: British Machine Vision Conference (BMVC) (2016)Google Scholar
  4. 4.
    Poggi, M., Mattoccia, S.: Learning a general-purpose confidence measure based on o(1) features and a smarter aggregation strategy for semi global matching. In: Proceedings of the 4th International Conference on 3D Vision, 3DV (2016)Google Scholar
  5. 5.
    Haeusler, R., Nair, R., Kondermann, D.: Ensemble learning for confidence measures in stereo vision. In: CVPR Proceedings, pp. 305–312 (2013)Google Scholar
  6. 6.
    Fu, Z., Ardabilian, M.: Stereo matching confidence learning based on multi-modal convolution neural networks. In: Representation, Analysis and Recognition of Shape and Motion from Image Data (RFMI) (2017)Google Scholar
  7. 7.
    Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  8. 8.
    Zbontar, J., LeCun, Y.: Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res. 17(1–32), 2 (2016)zbMATHGoogle Scholar
  9. 9.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3354–3361. IEEE (2012)Google Scholar
  10. 10.
    Scharstein, D., et al.: High-resolution stereo datasets with subpixel-accurate ground truth. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 31–42. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-11752-2_3CrossRefGoogle Scholar
  11. 11.
    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 151–158. Springer, Heidelberg (1994).  https://doi.org/10.1007/BFb0028345CrossRefGoogle Scholar
  12. 12.
    Hirschmuller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 807–814. IEEE (2005)Google Scholar
  13. 13.
    Hu, X., Mordohai, P.: A quantitative evaluation of confidence measures for stereo vision. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 34, 2121–2133 (2012)CrossRefGoogle Scholar
  14. 14.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47(1–3), 7–42 (2002)CrossRefGoogle Scholar
  15. 15.
    Spyropoulos, A., Komodakis, N., Mordohai, P.: Learning to detect ground control points for improving the accuracy of stereo matching. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1621–1628. IEEE (2014)Google Scholar
  16. 16.
    Park, M.G., Yoon, K.J.: Leveraging stereo matching with learning-based confidence measures. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015Google Scholar
  17. 17.
    Poggi, M., Mattoccia, S.: Learning to predict stereo reliability enforcing local consistency of confidence maps. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  18. 18.
    Poggi, M., Tosi, F., Mattoccia, S.: Even more confident predictions with deep machine-learning. In: 12th IEEE Embedded Vision Workshop (EVW2017) Held in Conjunction with IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  19. 19.
    Kim, S., Min, D., Kim, S., Sohn, K.: Feature augmentation for learning confidence measure in stereo matching. IEEE Trans. Image Process. 26(12), 6019–6033 (2017)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Mostegel, C., Rumpler, M., Fraundorfer, F., Bischof, H.: Using self-contradiction to learn confidence measures in stereo vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp. 4067–4076 (2016)Google Scholar
  21. 21.
    Tosi, F., Poggi, M., Tonioni, A., Di Stefano, L., Mattoccia, S.: Learning confidence measures in the wild. In: 28th British Machine Vision Conference, BMVC 2017, September 2017Google Scholar
  22. 22.
    Poggi, M., Tosi, F., Mattoccia, S.: Efficient confidence measures for embedded stereo. In: Battiato, S., Gallo, G., Schettini, R., Stanco, F. (eds.) ICIAP 2017. LNCS, vol. 10484, pp. 483–494. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-68560-1_43CrossRefGoogle Scholar
  23. 23.
    Marin, G., Zanuttigh, P., Mattoccia, S.: Reliable fusion of TOF and stereo depth driven by confidence measures. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 386–401. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46478-7_24CrossRefGoogle Scholar
  24. 24.
    Poggi, M., Mattoccia, S.: Deep stereo fusion: combining multiple disparity hypotheses with deep-learning. In: Proceedings of the 4th International Conference on 3D Vision, 3DV (2016)Google Scholar
  25. 25.
    Shaked, A., Wolf, L.: Improved stereo matching with constant highway networks and reflective confidence learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  26. 26.
    Tonioni, A., Poggi, M., Mattoccia, S., Di Stefano, L.: Unsupervised adaptation for deep stereo. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar
  27. 27.
    Zbontar, J., LeCun, Y.: Computing the stereo matching cost with a convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1592–1599 (2015)Google Scholar
  28. 28.
    Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  29. 29.
    Kendall, A., et al.: End-to-end learning of geometry and context for deep stereo regression. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar
  30. 30.
    Pang, J., Sun, W., Ren, J.S., Yang, C., Yan, Q.: Cascade residual learning: a two-stage convolutional neural network for stereo matching. In: The IEEE International Conference on Computer Vision (ICCV), October 2017Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Fabio Tosi
    • 1
  • Matteo Poggi
    • 1
  • Antonio Benincasa
    • 1
  • Stefano Mattoccia
    • 1
  1. 1.University of BolognaBolognaItaly

Personalised recommendations