DeepGMR: Learning Latent Gaussian Mixture Models for Registration

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12350)


Point cloud registration is a fundamental problem in 3D computer vision, graphics and robotics. For the last few decades, existing registration algorithms have struggled in situations with large transformations, noise, and time constraints. In this paper, we introduce Deep Gaussian Mixture Registration (DeepGMR), the first learning-based registration method that explicitly leverages a probabilistic registration paradigm by formulating registration as the minimization of KL-divergence between two probability distributions modeled as mixtures of Gaussians. We design a neural network that extracts pose-invariant correspondences between raw point clouds and Gaussian Mixture Model (GMM) parameters and two differentiable compute blocks that recover the optimal transformation from matched GMM parameters. This construction allows the network learn an SE(3)-invariant feature space, producing a global registration method that is real-time, generalizable, and robust to noise. Across synthetic and real-world data, our proposed method shows favorable performance when compared with state-of-the-art geometry-based and learning-based registration methods.


Point cloud registration Gaussian Mixture Model 

Supplementary material

504441_1_En_43_MOESM1_ESM.pdf (3.8 mb)
Supplementary material 1 (pdf 3906 KB)


  1. 1.
    Aoki, Y., Goforth, H., Srivatsan, R.A., Lucey, S.: Pointnetlk: robust & efficient point cloud registration using pointnet. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7163–7172 (2019)Google Scholar
  2. 2.
    Beraldin, J.A., Blais, F., Cournoyer, L., Godin, G., Rioux, M.: Active 3d sensing. Modelli e metodi per lo studio e la conservazione dell’architettura storica, University: Scola Normale Superiore, Pisa 10, January 2000Google Scholar
  3. 3.
    Besl, P., McKay, H.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992). Scholar
  4. 4.
    Bishop, C.M.: Pattern recognition and machine learning. Springer, New York (2006)Google Scholar
  5. 5.
    Campbell, D., Petersson, L.: Gogma: globally-optimal gaussian mixture alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5685–5694 (2016)Google Scholar
  6. 6.
    Campbell, D., Petersson, L., Kneip, L., Li, H., Gould, S.: The alignment of the spheres: Globally-optimal spherical mixture alignment for camera pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11796–11806 (2019)Google Scholar
  7. 7.
    Chen, C., Li, G., Xu, R., Chen, T., Wang, M., Lin, L.: Clusternet: deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4994–5002 (2019)Google Scholar
  8. 8.
    Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image Vis. Comput. 10(3), 145–155 (1992). Range Image UnderstandingCrossRefGoogle Scholar
  9. 9.
    Chetverikov, D., Stepanov, D., Krsek, P.: Robust euclidean alignment of 3d point sets: the trimmed iterative closest point algorithm. Image Vis. Comput. 23(3), 299–309 (2005)CrossRefGoogle Scholar
  10. 10.
    Choi, S., Zhou, Q.Y., Koltun, V.: Robust reconstruction of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5556–5565 (2015)Google Scholar
  11. 11.
    Choy, C., Gwak, J., Savarese, S.: 4d spatio-temporal convnets: Minkowski convolutional neural networks. arXiv preprint arXiv:1904.08755 (2019)
  12. 12.
    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. 1–38 (1977)Google Scholar
  13. 13.
    Dhawale, A., Shaurya Shankar, K., Michael, N.: Fast monte-carlo localization on aerial vehicles using approximate continuous belief representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5851–5859 (2018)Google Scholar
  14. 14.
    Eckart, B., Kim, K., Troccoli, A., Kelly, A., Kautz, J.: Mlmd: maximum likelihood mixture decoupling for fast and accurate point cloud registration. In: 2015 International Conference on 3D Vision (3DV), pp. 241–249. IEEE (2015)Google Scholar
  15. 15.
    Eckart, B., Kim, K., Kautz, J.: HGMR: Hierarchical gaussian mixtures for adaptive 3d registration. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 705–721 (2018)Google Scholar
  16. 16.
    Evangelidis, G.D., Horaud, R.: Joint alignment of multiple point sets with batch and incremental expectation-maximization. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1397–1410 (2017)CrossRefGoogle Scholar
  17. 17.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Fitzgibbon, A.W.: Robust registration of 2d and 3d point sets. Image Vis. Comput. 21(13), 1145–1153 (2003)CrossRefGoogle Scholar
  19. 19.
    Gao, W., Tedrake, R.: Filterreg: robust and efficient probabilistic point-set registration using gaussian filter and twist parameterization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11095–11104 (2019)Google Scholar
  20. 20.
    Granger, S., Pennec, X.: Multi-scale EM-ICP: a fast and robust approach for surface registration. ECCV 2002, 69–73 (2002)zbMATHGoogle Scholar
  21. 21.
    Handa, A., Whelan, T., McDonald, J., Davison, A.: A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In: IEEE International Conference on Robotics and Automation (ICRA). Hong Kong, China, May 2014Google Scholar
  22. 22.
    Horaud, R., Forbes, F., Yguel, M., Dewaele, G., Zhang, J.: Rigid and articulated point registration with expectation conditional maximization. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 587–602 (2011).
  23. 23.
    Järemo Lawin, F., Danelljan, M., Shahbaz Khan, F., Forssén, P.E., Felsberg, M.: Density adaptive point set registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3829–3837 (2018)Google Scholar
  24. 24.
    Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–1645 (2011).
  25. 25.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  26. 26.
    Low, K.L.: Linear least-squares optimization for point-to-plane ICP surface registration. Chapel Hill, University of North Carolina 4(10) (2004)Google Scholar
  27. 27.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI, pp. 674–679 (1981)Google Scholar
  28. 28.
    Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010)CrossRefGoogle Scholar
  29. 29.
    Paszke, A., et al.: Automatic differentiation in pytorch (2017)Google Scholar
  30. 30.
    Pomerleau, F., Colas, F., Siegwart, R.: A review of point cloud registration algorithms for mobile robotics. Found. Trends Robot 4(1), 1–104 (2015)Google Scholar
  31. 31.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings Computer Vision and Pattern Recognition (CVPR), IEEE, vol. 1, no. 2, p. 4 (2017)Google Scholar
  32. 32.
    Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems, pp. 5099–5108 (2017)Google Scholar
  33. 33.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001) Google Scholar
  34. 34.
    Rusinkiewicz, S.: A symmetric objective function for ICP. ACM Trans. Graph. (Proceedings SIGGRAPH) 38(4) (2019)Google Scholar
  35. 35.
    Segal, A., Haehnel, D., Thrun, S.: Generalized ICP. Robot. Sci. Syst. 2, 4 (2009) Google Scholar
  36. 36.
    Stoyanov, T.D., Magnusson, M., Andreasson, H., Lilienthal, A.: Fast and accurate scan registration through minimization of the distance between compact 3D NDT representations. Int. J. Robot. Res. (2012)Google Scholar
  37. 37.
    Su, H., et al.: Splatnet: sparse lattice networks for point cloud processing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2530–2539 (2018)Google Scholar
  38. 38.
    Tabib, W., O’Meadhra, C., Michael, N.: On-manifold GMM registration. IEEE Robot. Automat. Lett. 3(4), 3805–3812 (2018)CrossRefGoogle Scholar
  39. 39.
    Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: flexible and deformable convolution for point clouds. arXiv preprint arXiv:1904.08889 (2019)
  40. 40.
    Umeyama, S.: Least-squares estimation of transformation parameters between two point patterns. IEEE Trans. Pattern Anal. Mach. Intell. 13(4), 376–380 (1991)CrossRefGoogle Scholar
  41. 41.
    Unnikrishnan, R., Lalonde, J., Vandapel, N., Hebert, M.: Scale selection for the analysis of Point-Sampled curves. In: International Symposium on 3D Data Processing Visualization and Transmission, vol. 0, pp. 1026–1033. IEEE Computer Society, Los Alamitos, CA, USA (2006).
  42. 42.
    Wang, Y., Solomon, J.M.: Deep closest point: learning representations for point cloud registration. In: The IEEE International Conference on Computer Vision (ICCV), October 2019Google Scholar
  43. 43.
    Wang, Y., Solomon, J.M.: PRNET: Self-supervised learning for partial-to-partial registration. In: Advances in Neural Information Processing Systems, pp. 8812–8824 (2019)Google Scholar
  44. 44.
    Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. (TOG) 38(5), 146 (2019)CrossRefGoogle Scholar
  45. 45.
    Wu, Z., et al.: 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)Google Scholar
  46. 46.
    Yang, H., Shi, J., Carlone, L.: Teaser: fast and certifiable point cloud registration. arXiv preprint arXiv:2001.07715 (2020)
  47. 47.
    Yang, J., Li, H., Campbell, D., Jia, Y.: Go-ICP: a globally optimal solution to 3d ICP point-set registration. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2241–2254 (2015)CrossRefGoogle Scholar
  48. 48.
    Zhou, Q.-Y., Park, J., Koltun, V.: Fast global registration. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 766–782. Springer, Cham (2016). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of WashingtonSeattleUSA
  2. 2.NVIDIASanta ClaraUSA

Personalised recommendations