Skip to main content
SpringerLink
Log in

GTGMM: geometry transformer and Gaussian Mixture Models for robust point cloud registration

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Due to different acquisition time, viewpoint, and sensor noise during the process of point cloud data acquisition, the captured point clouds typically exhibit partial overlapped and contain large amounts of noise and outliers. However, this circumstance tends to diminish the accuracy of point-to-point correspondence searches. Existing point-level methods rely on idealized point-to-point correspondences, which cannot be guaranteed in practical applications. To address above limitations, a noval network based on a geometry transformer and a Gaussian Mixture Model (GMM) is proposed, called GTGMM. Specifically, we formulate the registration problem as the problem of aligning the two Gaussian mixtures, leveraging the advantages of the statistic model and learned robust features to overcome the noise and outliers variants. We utilize a Local Feature Extractor (LFE) to extract structural features of point clouds, while the Transformer encoders establish global relations among the point clouds. Additionally, a geometry transformer network is introduced to capture geometric relations within the point cloud, and overlap scores are learned to reject non-overlapping regions. Utilizing overlap scores, point cloud features, and 3D point cloud coordinates, the matching parameters of GMM to calculate to guide the alignment of two point clouds. Experimental results on synthetic datasets and the real Terracotta Warriors data demonstrate that our method achieves high accuracy and robustness under various registration conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

Data will be made available on reasonable request.

References

  1. Shamsfakhr F, SadeghiBigham B (2020) Gsr: geometrical scan registration algorithm for robust and fast robot pose estimation. Assem Autom 40(6):801–817

    Article  Google Scholar 

  2. Shao J, Zhang W, Mellado N, Grussenmeyer P, Li R, Chen Y, Wan P, Zhang X, Cai S (2019) Automated markerless registration of point clouds from tls and structured light scanner for heritage documentation. J Cult Herit 35:16–24

    Article  Google Scholar 

  3. Zhang Z, Dai Y, Sun J (2020) Deep learning based point cloud registration: an overview. Virtual Reality Intell Hardw 2(3):222–246

    Article  Google Scholar 

  4. Zhou L, Sun G, Li Y, Li W, Su Z (2022) Point cloud denoising review: from classical to deep learning-based approaches. Graph Models 121:101140

    Article  Google Scholar 

  5. Yuan W, Eckart B, Kim K, Jampani V, Fox D, Kautz J (2020) Deepgmr: Learning latent gaussian mixture models for registration. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16:733–750. Springer

  6. Aoki Y, Goforth H, Srivatsan RA, Lucey S (2019) Pointnetlk: robust & efficient point cloud registration using pointnet. In: Proceedings of the IEEE/CVF Conf Comput Vis Pattern Recognition 7163–7172. https://doi.org/10.1109/CVPR.2019.00733

  7. Qi CR, Su H, Mo K, Guibas LJ (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conf Comput Vis Pattern Recognition 652–660. https://doi.org/10.1109/CVPR.2017.16

  8. Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: IJCAI’81: 7th International Joint Conference on Artificial Intelligence 2:674–679

  9. Sarode V, Li X, Goforth H, Aoki Y, Srivatsan RA, Lucey S, Choset H (2019) Pcrnet: Point cloud registration network using pointnet encoding. arXiv preprint arXiv:1908.07906

  10. Huang X, Mei G, Zhang J (2020) Feature-metric registration: a fast semi-supervised approach for robust point cloud registration without correspondences. In: Proceedings of the IEEE/CVF Conf Comput Vis Pattern Recognition 11366–11374. https://doi.org/10.1109/CVPR42600.2020.01138

  11. Sarode V, Dhagat A, Srivatsan RA, Zevallos N, Lucey S, Choset H (2020) Masknet: a fully-convolutional network to estimate inlier points. In: 2020 Int Conf 3D Vision (3DV) 1029–1038. IEEE. https://doi.org/10.1109/3DV50981.2020.00113

  12. Xu H, Liu S, Wang G, Liu G, Zeng B (2021) Omnet: Learning overlapping mask for partial-to-partial point cloud registration. In: Proceedings of the IEEE/CVF Int Conf Comput Vis 3132–3141. https://doi.org/10.1109/ICCV48922.2021.00312

  13. Besl PJ, McKay ND (1992) Method for registration of 3-d shapes. In: sensor fusion IV: Control Paradigms and Data Struct 1611:586–606. Spie. https://doi.org/10.1117/12.57955

  14. Wang Y, Solomon JM (2019) Deep closest point: Learning representations for point cloud registration. In: Proceedings of the IEEE/CVF Int Conf Comput Vis 3523–3532. https://doi.org/10.1109/ICCV.2019.00362

  15. Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM (2019) Dynamic graph cnn for learning on point clouds. Acm Trans Graph (tog) 38(5):1–12. https://doi.org/10.1145/3326362

  16. Choy C, Dong W, Koltun V (2020) Deep global registration. In: Proceedings of the IEEE/CVF Conf Comput Vis Pattern Recognition 2514–2523. https://doi.org/10.1109/CVPR42600.2020.00259

  17. Yew ZJ, Lee GH (2020) Rpm-net: Robust point matching using learned features. In: Proceedings of the IEEE/CVF Conf Comput Vis and Pattern Recognition 11824–11833. https://doi.org/10.1109/CVPR42600.2020.01184

  18. Li J, Zhang C, Xu Z, Zhou H, Zhang C (2020) Iterative distance-aware similarity matrix convolution with mutual-supervised point elimination for efficient point cloud registration. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020. Proceedings, Part XXIV 16, Springer, pp 378–394. https://doi.org/10.1007/978-3-030-58586-0_23

  19. Wang H, Liu X, Kang W, Yan Z, Wang B, Ning Q (2022) Multi-features guidance network for partial-to-partial point cloud registration. Neural Comput Appl 34(2):1623–1634

    Article  CAS  Google Scholar 

  20. Li X, Sun J, Own C-M, Tao W (2020) Gaussian mixture model-based registration network for point clouds with partial overlap. In: Artificial Neural Networks and Machine Learning–ICANN 2022: 31st International Conference on Artificial Neural Networks, Bristol, UK, September 6–9, 2022. Proceedings, Part III, Springer, pp 405–416. https://doi.org/10.1007/978-3-031-15934-3_34

  21. Huang X, Li S, Zuo Y, Fang Y, Zhang J, Zhao X (2022) Unsupervised point cloud registration by learning unified gaussian mixture models. IEEE Robot Autom Lett 7(3):7028–7035

    Article  Google Scholar 

  22. Chen G, Wang M, Zhang Q, Yuan L, Yue Y (2023) Full transformer framework for robust point cloud registration with deep information interaction. IEEE Trans Neural Networks and Learn Syst. https://doi.org/10.1109/TNNLS.2023.3267333

  23. Fu K, Liu S, Luo X, Wang M (2021) Robust point cloud registration framework based on deep graph matching. In: Proceedings of the IEEE/CVF Conf Comput Vis and Pattern Recognition 8893–8902. https://doi.org/10.1109/TPAMI.2022.3204713

  24. Shi C, Chen X, Huang K, Xiao J, Lu H, Stachniss C (2021) Keypoint matching for point cloud registration using multiplex dynamic graph attention networks. IEEE Robot Autom Lett 6(4):8221–8228

    Article  Google Scholar 

  25. Wang Y, Yan C, Feng Y, Du S, Dai Q, Gao Y (2022) Storm: Structure-based overlap matching for partial point cloud registration. IEEE Trans Pattern Anal Mach Intell 45(1):1135–1149

    Article  PubMed  Google Scholar 

  26. Huang S, Gojcic Z, Usvyatsov M, Wieser A, Schindler K (2021) Predator: Registration of 3d point clouds with low overlap. In: Proceedings of the IEEE/CVF Conf Comput Vis and Pattern Recognition 4267–4276. https://doi.org/10.1109/CVPR46437.2021.00425

  27. Yu H, Li F, Saleh M, Busam B, Ilic S (2021) Cofinet: Reliable coarse-to-fine correspondences for robust pointcloud registration. Adv Neural Inf Process Syst 34:23872–23884

    Google Scholar 

  28. Yew ZJ, Lee GH (2022) Regtr: End-to-end point cloud correspondences with transformers. In: Proceedings of the IEEE/CVF Conf Comput Vis and Pattern Recognition 6677–6686. https://doi.org/10.1109/CVPR52688.2022.00656

  29. Yuan L, Chen Y, Wang T, Yu W, Shi Y, Jiang Z-H, Tay FE, Feng J, Yan S (2021) Tokens-to-token vit: Training vision transformers from scratch on imagenet. In: Proceedings of the IEEE/CVF Int Conf Comput Vis 558–567. https://doi.org/10.1109/ICCV48922.2021.00060

  30. Shotton J, Glocker B, Zach C, Izadi S, Criminisi A, Fitzgibbon A (2013) Scene coordinate regression forests for camera relocalization in rgb-d images. In: Proceedings of the IEEE Conf Comput Vis and Pattern Recognition 2930–2937. https://doi.org/10.1109/CVPR.2013.377

  31. Choi S, Zhou Q-Y, Koltun V (2015) Robust reconstruction of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5556–5565. https://doi.org/10.1109/CVPR.2015.7299195

  32. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  33. Rusu RB, Blodow N, Beetz M (2009) Fast point feature histograms (fpfh) for 3d registration. In: 2009 IEEE Int Conf Robot Autom 3212–3217. IEEE. https://doi.org/10.1109/ROBOT.2009.5152473

  34. Zhou Q-Y, Park J, Koltun V (2016) Fast global registration. In: Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14. Springer, p 766–782. https://doi.org/10.1007/978-3-319-46475-6_47

  35. Jian B, Vemuri BC (2010) Robust point set registration using gaussian mixture models. IEEE Trans Pattern Anal Mach Intell 33(8):1633–1645

    Article  PubMed  Google Scholar 

  36. Evangelidis GD, Horaud R (2017) Joint alignment of multiple point sets with batch and incremental expectation-maximization. IEEE Trans Pattern Anal Mach Intell 40(6):1397–1410

    Article  PubMed  Google Scholar 

  37. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conf Comput Vis and Pattern Recognition 1912–1920. https://doi.org/10.1109/CVPR.2015.7298801

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation of China under Grant (No.61902317), the Science and Technology Plan Program in Shaanxi Province of China under Grant (No.2019JQ-166).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Northwest University, Xi’an, 710127, Shaanxi, China

    Haibo Zhang, Linqi Hai, Haoran Sun, Xu Wang, Ruoxue Li, Guohua Geng & Mingquan Zhou

Authors
  1. Haibo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linqi Hai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoran Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruoxue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guohua Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mingquan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haibo Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Hai, L., Sun, H. et al. GTGMM: geometry transformer and Gaussian Mixture Models for robust point cloud registration. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18660-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18660-8

Keywords

Search

Navigation