Abstract
With the development of technology, 3D reconstruction has been widely used in many fields. In this paper, we propose a learnable 3D reconstruction method using a cascaded Patchmatch approach to form a new network. By introducing a dual-channel attention module, point clouds reconstruction has been improved in accuracy and completion This network has high computational speed and low memory requirements, which allows it to handle higher-resolution images. The network is more suitable for running on resource-constrained devices than competitors that employ 3D cost volume regularization. We introduce the feature fusion module to an end-to-end trainable framework for the first time. The weight parameters of the multi-scale network output can be adaptively learned in each calculation, which can reduce the feature dispersion caused by the multi-scale output. This method has good performance on DTU.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arno, K., Jaesik, P., Qianyi, Z., Vladlen, K.: Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Trans. Graph. (2017)
Thoms, S., Johannes, L.S., Silvano, G., Torsten, S., Konrad, S., Marc, P., Andreas, G.: A multi-view stereo benchmark with highresolution images and multi-camera videos. In Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Rui, C., Songfang, H., Jing, X., Hao, S.: Point-based multi-view stereo network. In International Conference on Computer Vision (ICCV) (2019)
Keyang, L., Tao, G., Lili, J., Haipeng, H., Yawei, L.: P-MVSNet: Learning patch-wise matching confidence aggregation for multi-view stereo. In: International Conference on Computer Vision (ICCV) (2019)
Silvano, G., Katrin, L., Konrad, S.: Massively parallel multiview stereopsis by surface normal diffusion. In: International Conference on Computer Vision (ICCV) (2015)
Qingshan, X., Wenbing, T.: Learning inverse depth regression for multi-view stereo with correlation cost volume. In: AAAI (2020)
Yao, Y., Zixin, L., Shiwei, L., Tian, F., Long, Q.: MVSNet: depth inference for unstructured multiview stereo. In: European Conference on Computer Vision (ECCV) (2018)
Connelly, B., Eli, S., Adam, F., Dan, B.G.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (SIGGRAPH) (2009)
Alex, K., Hayk, M., Saumitro, D., Peter, H., Ryan, K., Abraham, B., Adam, B.: End-to-end learning of geometry and context for deep stereo regression. In: International Conference on Computer Vision (ICCV) (2017)
Jia-Ren, C., Yong-Sheng, C.: Pyramid stereo matching network. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Haofei, X., Juyong, Z.: AANet: adaptive aggregation network for efficient stereo matching. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Abhishek, K., Christian, H., Jitendra, M.: Learning a multi-view stereo machine. In: Advances in Neural Information Processing Systems (2017)
Robert, C.: A space-sweep approach to true multi-image matching. In: Conference on Computer Vision and Pattern Recognition (CVPR) (1996)
Xiaodong, G., Zhiwen, F., Siyu, Z., Zuozhuo, D., Feitong, T., Ping, T.: Cascade cost volume for high-resolution multi-view stereo and stereo matching. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Shuo, C., et al.: Deep stereo using adaptive thin volume representation with uncertainty awareness. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Jiayu, Y., Wei, M., Jose, M.A., Miaomiao, L.: Cost volume pyramid based depth inference for multi-view stereo. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Zehao, Y., Shenghua, G.: Fast-MVSNet: sparse-todense multi-view stereo with learned propagation and gaussnewton refinement. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Hui, T.-W., Loy, C.C., Tang, X.: Depth map super-resolution by deep multi-scale guidance. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 353–369. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_22
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Luo, X., Xie, Y. (2023). FFP-MVSNet: Feature Fusion Based Patchmatch for Multi-view Stereo. In: Liang, Q., Wang, W., Liu, X., Na, Z., Zhang, B. (eds) Communications, Signal Processing, and Systems. CSPS 2022. Lecture Notes in Electrical Engineering, vol 873. Springer, Singapore. https://doi.org/10.1007/978-981-99-1260-5_21
Download citation
DOI: https://doi.org/10.1007/978-981-99-1260-5_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1259-9
Online ISBN: 978-981-99-1260-5
eBook Packages: EngineeringEngineering (R0)