Abstract
Point cloud registration is a fundamental problem in 3D computer vision, graphics and robotics. For the last few decades, existing registration algorithms have struggled in situations with large transformations, noise, and time constraints. In this paper, we introduce Deep Gaussian Mixture Registration (DeepGMR), the first learning-based registration method that explicitly leverages a probabilistic registration paradigm by formulating registration as the minimization of KL-divergence between two probability distributions modeled as mixtures of Gaussians. We design a neural network that extracts pose-invariant correspondences between raw point clouds and Gaussian Mixture Model (GMM) parameters and two differentiable compute blocks that recover the optimal transformation from matched GMM parameters. This construction allows the network learn an SE(3)-invariant feature space, producing a global registration method that is real-time, generalizable, and robust to noise. Across synthetic and real-world data, our proposed method shows favorable performance when compared with state-of-the-art geometry-based and learning-based registration methods.
W. Yuan—Work partially done during an internship at NVIDIA.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We performed a parameter search over the voxel size v, which is crucial for FGR’s performance. We used the best results with \(v=0.08\).
References
Aoki, Y., Goforth, H., Srivatsan, R.A., Lucey, S.: Pointnetlk: robust & efficient point cloud registration using pointnet. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7163–7172 (2019)
Beraldin, J.A., Blais, F., Cournoyer, L., Godin, G., Rioux, M.: Active 3d sensing. Modelli e metodi per lo studio e la conservazione dell’architettura storica, University: Scola Normale Superiore, Pisa 10, January 2000
Besl, P., McKay, H.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992). https://doi.org/10.1109/34.121791
Bishop, C.M.: Pattern recognition and machine learning. Springer, New York (2006)
Campbell, D., Petersson, L.: Gogma: globally-optimal gaussian mixture alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5685–5694 (2016)
Campbell, D., Petersson, L., Kneip, L., Li, H., Gould, S.: The alignment of the spheres: Globally-optimal spherical mixture alignment for camera pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11796–11806 (2019)
Chen, C., Li, G., Xu, R., Chen, T., Wang, M., Lin, L.: Clusternet: deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4994–5002 (2019)
Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image Vis. Comput. 10(3), 145–155 (1992). Range Image Understanding
Chetverikov, D., Stepanov, D., Krsek, P.: Robust euclidean alignment of 3d point sets: the trimmed iterative closest point algorithm. Image Vis. Comput. 23(3), 299–309 (2005)
Choi, S., Zhou, Q.Y., Koltun, V.: Robust reconstruction of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5556–5565 (2015)
Choy, C., Gwak, J., Savarese, S.: 4d spatio-temporal convnets: Minkowski convolutional neural networks. arXiv preprint arXiv:1904.08755 (2019)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. 1–38 (1977)
Dhawale, A., Shaurya Shankar, K., Michael, N.: Fast monte-carlo localization on aerial vehicles using approximate continuous belief representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5851–5859 (2018)
Eckart, B., Kim, K., Troccoli, A., Kelly, A., Kautz, J.: Mlmd: maximum likelihood mixture decoupling for fast and accurate point cloud registration. In: 2015 International Conference on 3D Vision (3DV), pp. 241–249. IEEE (2015)
Eckart, B., Kim, K., Kautz, J.: HGMR: Hierarchical gaussian mixtures for adaptive 3d registration. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 705–721 (2018)
Evangelidis, G.D., Horaud, R.: Joint alignment of multiple point sets with batch and incremental expectation-maximization. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1397–1410 (2017)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Fitzgibbon, A.W.: Robust registration of 2d and 3d point sets. Image Vis. Comput. 21(13), 1145–1153 (2003)
Gao, W., Tedrake, R.: Filterreg: robust and efficient probabilistic point-set registration using gaussian filter and twist parameterization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11095–11104 (2019)
Granger, S., Pennec, X.: Multi-scale EM-ICP: a fast and robust approach for surface registration. ECCV 2002, 69–73 (2002)
Handa, A., Whelan, T., McDonald, J., Davison, A.: A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In: IEEE International Conference on Robotics and Automation (ICRA). Hong Kong, China, May 2014
Horaud, R., Forbes, F., Yguel, M., Dewaele, G., Zhang, J.: Rigid and articulated point registration with expectation conditional maximization. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 587–602 (2011). http://doi.ieeecomputersociety.org/10.1109/TPAMI.2010.94
Järemo Lawin, F., Danelljan, M., Shahbaz Khan, F., Forssén, P.E., Felsberg, M.: Density adaptive point set registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3829–3837 (2018)
Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–1645 (2011). http://gmmreg.googlecode.com
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Low, K.L.: Linear least-squares optimization for point-to-plane ICP surface registration. Chapel Hill, University of North Carolina 4(10) (2004)
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI, pp. 674–679 (1981)
Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010)
Paszke, A., et al.: Automatic differentiation in pytorch (2017)
Pomerleau, F., Colas, F., Siegwart, R.: A review of point cloud registration algorithms for mobile robotics. Found. Trends Robot 4(1), 1–104 (2015)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings Computer Vision and Pattern Recognition (CVPR), IEEE, vol. 1, no. 2, p. 4 (2017)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems, pp. 5099–5108 (2017)
Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001)
Rusinkiewicz, S.: A symmetric objective function for ICP. ACM Trans. Graph. (Proceedings SIGGRAPH) 38(4) (2019)
Segal, A., Haehnel, D., Thrun, S.: Generalized ICP. Robot. Sci. Syst. 2, 4 (2009)
Stoyanov, T.D., Magnusson, M., Andreasson, H., Lilienthal, A.: Fast and accurate scan registration through minimization of the distance between compact 3D NDT representations. Int. J. Robot. Res. (2012)
Su, H., et al.: Splatnet: sparse lattice networks for point cloud processing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2530–2539 (2018)
Tabib, W., O’Meadhra, C., Michael, N.: On-manifold GMM registration. IEEE Robot. Automat. Lett. 3(4), 3805–3812 (2018)
Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: flexible and deformable convolution for point clouds. arXiv preprint arXiv:1904.08889 (2019)
Umeyama, S.: Least-squares estimation of transformation parameters between two point patterns. IEEE Trans. Pattern Anal. Mach. Intell. 13(4), 376–380 (1991)
Unnikrishnan, R., Lalonde, J., Vandapel, N., Hebert, M.: Scale selection for the analysis of Point-Sampled curves. In: International Symposium on 3D Data Processing Visualization and Transmission, vol. 0, pp. 1026–1033. IEEE Computer Society, Los Alamitos, CA, USA (2006). http://doi.ieeecomputersociety.org/10.1109/3DPVT.2006.123
Wang, Y., Solomon, J.M.: Deep closest point: learning representations for point cloud registration. In: The IEEE International Conference on Computer Vision (ICCV), October 2019
Wang, Y., Solomon, J.M.: PRNET: Self-supervised learning for partial-to-partial registration. In: Advances in Neural Information Processing Systems, pp. 8812–8824 (2019)
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. (TOG) 38(5), 146 (2019)
Wu, Z., et al.: 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)
Yang, H., Shi, J., Carlone, L.: Teaser: fast and certifiable point cloud registration. arXiv preprint arXiv:2001.07715 (2020)
Yang, J., Li, H., Campbell, D., Jia, Y.: Go-ICP: a globally optimal solution to 3d ICP point-set registration. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2241–2254 (2015)
Zhou, Q.-Y., Park, J., Koltun, V.: Fast global registration. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 766–782. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_47
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Yuan, W., Eckart, B., Kim, K., Jampani, V., Fox, D., Kautz, J. (2020). DeepGMR: Learning Latent Gaussian Mixture Models for Registration. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12350. Springer, Cham. https://doi.org/10.1007/978-3-030-58558-7_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-58558-7_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58557-0
Online ISBN: 978-3-030-58558-7
eBook Packages: Computer ScienceComputer Science (R0)