Skip to main content
SpringerLink
Log in

Multi-view Stereo by Fusing Monocular and a Combination of Depth Representation Methods

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14450))

Included in the following conference series:

  • 574 Accesses

Abstract

The design of plane-sweep deep MVS primarily relies on patch-similarity based matching. However, this approach becomes impractical when dealing with low-textured, similar-textured and reflective regions in the scene, resulting in inaccurate matching results. One of the methods to avoid this kind of error is incorporating semantic information in matching process. In this paper, we propose an end-to-end method that uses monocular depth estimation to add semantic information to deep MVS. Additionally, we analyze the advantages and disadvantages of two main depth representations and propose a collaborative method to alleviate their drawbacks. Finally, we introduce a novel filtering criterion named Distribution Consistency, which can effectively filter out outliers with poor probability distribution, such as uniform distribution, to further enhance the reconstruction quality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Peng, R., Wang, R., Wang, Z., Lai, Y., Wang, R.: Rethinking depth estimation for multi-view stereo: a unified representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8645–8654 (2022)

    Google Scholar 

  2. Bae, G., Budvytis, I., Cipolla, R.: Multi-view depth estimation by fusing single-view depth probability with multi-view geometry. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2842–2851 (2022)

    Google Scholar 

  3. Wang, X., et al.: MVSTER: epipolar transformer for efficient multi-view stereo. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, pp. 573–591. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19821-2_33

    Chapter  Google Scholar 

  4. Zhang, C., Meng, G., Bing, S., Xiang, S., Pan, C.: Monocular contextual constraint for stereo matching with adaptive weights assignment. Image Vis. Comput. 121, 104424 (2022)

    Article  Google Scholar 

  5. Yao, Y., Luo, Z., Li, S., Fang, T., Quan, L.: Mvsnet: depth inference for unstructured multi-view stereo. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 767–783 (2018)

    Google Scholar 

  6. Yang, J., Mao, W., Alvarez, J.M., Liu, M.: Cost volume pyramid based depth inference for multi-view stereo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4877–4886 (2020)

    Google Scholar 

  7. Gu, X., Fan, Z., Zhu, S., Dai, Z., Tan, F., Tan, P.: Cascade cost volume for high-resolution multi-view stereo and stereo matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2495–2504 (2020)

    Google Scholar 

  8. Zhang, Y., et al.: Adaptive unimodal cost volume filtering for deep stereo matching. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12926–12934 (2020)

    Google Scholar 

  9. Wei, Z., Zhu, Q., Min, C., Chen, Y., Wang, G.: Aa-rmvsnet: adaptive aggregation recurrent multi-view stereo network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6187–6196 (2021)

    Google Scholar 

  10. Yao, Y., Luo, Z., Li, S., Shen, T., Fang, T., Quan, L.: Recurrent mvsnet for high-resolution multi-view stereo depth inference. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5525–5534 (2019)

    Google Scholar 

  11. Yan, J., et al.: Dense hybrid recurrent multi-view stereo net with dynamic consistency checking. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 674–689. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_39

    Chapter  Google Scholar 

  12. Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)

    Google Scholar 

  13. Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

  14. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  15. Knapitsch, A., Park, J., Zhou, Q.-Y., Koltun, V.: Tanks and temples: benchmarking large-scale scene reconstruction. ACM Trans. Graph. (ToG) 36(4), 1–13 (2017)

    Article  Google Scholar 

  16. Yao, Y., et al.: Blendedmvs: a large-scale dataset for generalized multi-view stereo networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1790–1799 (2020)

    Google Scholar 

  17. Yu, F., Pang, J., Wang, R.: Sub-pixel convolution and edge detection for multi-view stereo. In: 2022 IEEE 8th International Conference on Computer and Communications (ICCC), pp. 1864–1868. IEEE (2022)

    Google Scholar 

  18. Merrell, P., et al.: Real-time visibility-based fusion of depth maps. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE (2007)

    Google Scholar 

  19. Zhang, J., Yao, Y., Li, S., Luo, Z., Fang, T.: Visibility-aware multi-view stereo network. arXiv preprint arXiv:2008.07928 (2020)

  20. Cheng, S., et al.: Deep stereo using adaptive thin volume representation with uncertainty awareness. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2524–2534 (2020)

    Google Scholar 

  21. Xu, Q., Tao, W.: Planar prior assisted patchmatch multi-view stereo. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12516–12523 (2020)

    Google Scholar 

  22. Luo, K., Guan, T., Ju, L., Wang, Y., Chen, Z., Luo, Y.: Attention-aware multi-view stereo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1590–1599 (2020)

    Google Scholar 

  23. Wang, F., Galliani, S., Vogel, C., Speciale, P., Pollefeys, M.: Patchmatchnet: learned multi-view patchmatch stereo. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14194–14203 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Computer Engineering, Peking University, Beijing, China

    Fanqi Yu

  2. School of Software and Microelectronics, Peking University, Beijing, China

    Xinyang Sun

Authors
  1. Fanqi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinyang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fanqi Yu .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Biao Luo

  2. Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yu, F., Sun, X. (2024). Multi-view Stereo by Fusing Monocular and a Combination of Depth Representation Methods. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14450. Springer, Singapore. https://doi.org/10.1007/978-981-99-8070-3_23

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8070-3_23

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8069-7

  • Online ISBN: 978-981-99-8070-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation