Skip to main content
SpringerLink
Log in

Occlusion Aware Stereo Matching via Cooperative Unsupervised Learning

  • Conference paper
  • First Online:
Computer Vision – ACCV 2018 (ACCV 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11366))

Included in the following conference series:

Abstract

Occlusion as a core challenge for stereo computation has attracted extensive research efforts in the past decades. Apart from its adverse impact, occlusion itself is a crucial clue which has not been exploited in the field of CNN based stereo. In this paper, we argue that a deep stereo framework benefits from reasoning occlusion in advance. We present an occlusion aware stereo network comprising a prior occlusion inferring module and a subsequent disparity computation module. The occlusion inferring module is a sub-network that directly starts from images, which averts the sophisticated procedure to iteratively estimate occlusion with disparity. We additionally propose cooperative unsupervised learning of occlusion and disparity, based on a different hybrid loss enforcing them to be consensus and trained alternatively to reach convergence. The comprehensive experimental analyses show that our method achieves state-of-the-art results among unsupervised learning frameworks, and is even comparable to several supervised methods.

Supported by the National Key R&D Program of China (No. 2016YFB1001001) and the National Natural Science Foundation of China (No. 61573280, No. 91648121).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The matching with regularization architecture is exactly the one we used in our DC module.

References

  1. Ahmadi, A., Patras, I.: Unsupervised convolutional neural networks for motion estimation. In: Proceedings of IEEE International Conference on Image Processing, ICIP (2016)

    Google Scholar 

  2. Anderson, B.L., Nakayama, K.: Toward a general theory of stereopsis: binocular matching, occluding contours, and fusion. Psychol. Rev. 101(3), 414 (1994)

    Article  Google Scholar 

  3. Chang, J.R., Chen, Y.S.: Pyramid stereo matching network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2018)

    Google Scholar 

  4. Chen, Z., Sun, X., Wang, L., Yu, Y., Huang, C.: A deep visual correspondence embedding model for stereo matching costs. In: Proceedings of IEEE International Conference on Computer Vision, ICCV (2015)

    Google Scholar 

  5. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2012)

    Google Scholar 

  6. Godard, C., Mac Aodha, O., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2017)

    Google Scholar 

  7. Heise, P., Klose, S., Jensen, B., Knoll, A.: PM-Huber: PatchMatch with Huber regularization for stereo matching. In: Proceedings of IEEE International Conference on Computer Vision, ICCV (2013)

    Google Scholar 

  8. Heitz, F., Bouthemy, P.: Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Trans. Pattern Anal. Mach. Intell. PAMI 15(12), 1217–1232 (1993)

    Article  Google Scholar 

  9. Hosni, A., Bleyer, M., Gelautz, M., Rhemann, C.: Local stereo matching using geodesic support weights. In: Proceedings of the IEEE International Conference on Image Processing, ICIP (2009)

    Google Scholar 

  10. Intille, S.S., Bobick, A.F.: Disparity-space images and large occlusion stereo. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 179–186. Springer, Heidelberg (1994). https://doi.org/10.1007/BFb0028349

    Chapter  Google Scholar 

  11. Yu, J.J., Harley, A.W., Derpanis, K.G.: Back to basics: unsupervised learning of optical flow via brightness constancy and motion smoothness. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 3–10. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49409-8_1

    Chapter  Google Scholar 

  12. Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P.: End-to-end learning of geometry and context for deep stereo regression. In: Proceedings of the IEEE International Conference on Computer Vision, ICCV (2017)

    Google Scholar 

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  14. LeCun, Y.A., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient backprop. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 9–48. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35289-8_3

    Chapter  Google Scholar 

  15. Li, A., Yuan, Z.: SymmNet: a symmetric convolutional neural network for occlusion detection. In: Proceedings of the British Machine Vision Conference, BMVC (2018)

    Google Scholar 

  16. Liang, Z., et al.: Learning for disparity estimation through feature constancy. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2018)

    Google Scholar 

  17. Liu, M., Salzmann, M., He, X.: Discrete-continuous depth estimation from a single image. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2014)

    Google Scholar 

  18. Luo, W., Schwing, A.G., Urtasun, R.: Efficient deep learning for stereo matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2016)

    Google Scholar 

  19. Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2016)

    Google Scholar 

  20. Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2015)

    Google Scholar 

  21. Nakayama, K., Shimojo, S.: Da vinci stereopsis: depth and subjective occluding contours from unpaired image points. Vis. Res. 30(11), 1811–1825 (1990)

    Article  Google Scholar 

  22. Shaked, A., Wolf, L.: Improved stereo matching with constant highway networks and reflective confidence learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2017)

    Google Scholar 

  23. Silva, C., Santos-Victor, J.: Intrinsic images for dense stereo matching with occlusions. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 100–114. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45054-8_7

    Chapter  Google Scholar 

  24. Sun, J., Li, Y., Kang, S.B.: Symmetric stereo matching for occlusion handling. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)

    Google Scholar 

  25. Tonioni, A., Poggi, M., Mattoccia, S., Di Stefano, L.: Unsupervised adaptation for deep stereo. In: The IEEE International Conference on Computer Vision, ICCV (2017)

    Google Scholar 

  26. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. TIP 13(4), 600–612 (2004)

    Article  Google Scholar 

  27. Yamaguchi, K., Hazan, T., McAllester, D., Urtasun, R.: Continuous Markov random fields for robust stereo estimation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 45–58. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_4

    Chapter  Google Scholar 

  28. Yamaguchi, K., McAllester, D., Urtasun, R.: Efficient joint segmentation, occlusion labeling, stereo and flow estimation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 756–771. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_49

    Chapter  Google Scholar 

  29. Wang, Y., Yang, Y., Yang, Z., Zhao, L., Wang, P., Xu, W.: Occlusion aware unsupervised learning of optical flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2018)

    Google Scholar 

  30. Zbontar, J., LeCun, Y.: Computing the stereo matching cost with a convolutional neural network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2015)

    Google Scholar 

  31. Zbontar, J., LeCun, Y.: Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res. 17, 65:1–65:32 (2016)

    MATH  Google Scholar 

  32. Zhang, C., Li, Z., Cheng, Y., Cai, R., Chao, H., Rui, Y.: MeshStereo: a global stereo model with mesh alignment regularization for view interpolation. In: Proceedings of IEEE International Conference on Computer Vision, ICCV (2015)

    Google Scholar 

  33. Zhou, C., Zhang, H., Shen, X., Jia, J.: Unsupervised learning of stereo matching. In: Proceedings of the IEEE International Conference on Computer Vision, ICCV (2017)

    Google Scholar 

  34. Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, People’s Republic of China

    Ang Li & Zejian Yuan

Authors
  1. Ang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zejian Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zejian Yuan .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C.V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, A., Yuan, Z. (2019). Occlusion Aware Stereo Matching via Cooperative Unsupervised Learning. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11366. Springer, Cham. https://doi.org/10.1007/978-3-030-20876-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20876-9_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20875-2

  • Online ISBN: 978-3-030-20876-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation