Skip to main content
SpringerLink
Log in

Cy-CNN: cylinder convolution based rotation-invariant neural network for point cloud registration

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Point cloud registration is a challenging problem in the condition of large initial misalignments and noises. A major problem encountered in the registration algorithms is the definition of correspondence between two point clouds. Point clouds contain rich geometric information and the same geometric structure implies the same feature even if they are in different poses, which motivates us to seek a rotation-invariant feature representation for calculating the correspondence. This work proposes a rotation-invariant neural network for point cloud registration. To acquire rotation-invariant features, we firstly propose a rotation-invariant point cloud representation (RIPR) at the input level. Instead of using the original coordinates, we propose to use point pair features (PPF) and the transformed coordinates in the local reference frame (LRF) to represent a point. Then, we design a new convolution operator named Cylinder-Conv which utilizes the symmetry of cylinder-shaped voxels and the hierarchical geometry information of the surface of 3D shapes. By specifying the cylinder-shaped structures and directions, Cylinder-Conv can better capture the local neighborhood geometry of each point and maintain rotation-invariance. Finally, we combine RIPR and Cylinder-Conv to extract normalized rotation-invariant features to generate the correspondence and perform a differentiable singular value decomposition (SVD) step to estimate the rigid transformation. The proposed network presents state-of-the-art performance on point cloud registration. Experiments show that our method is robust to initial misalignments and noises.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Liu B S, Chen X M, Han Y H, et al. Accelerating DNN-based 3D point cloud processing for mobile computing. Sci China Inf Sci, 2019, 62: 212102

    Article  MathSciNet  Google Scholar 

  2. Peng H T, Zhou B, Yin L Y, et al. Semantic part segmentation of single-view point cloud. Sci China Inf Sci, 2020, 63: 224101

    Article  Google Scholar 

  3. Zhao H, Zhu C Y, Xu X, et al. Learning practically feasible policies for online 3D bin packing. Sci China Inf Sci, 2022, 65: 112105

    Article  MathSciNet  Google Scholar 

  4. Han L, Gu S, Zhong D, et al. Real-time globally consistent dense 3D reconstruction with online texturing. IEEE Trans Pattern Anal Mach Intell, 2022, 44: 1519–1533

    Article  Google Scholar 

  5. Fang S, Li H, Yang M. LiDAR SLAM based multivehicle cooperative localization using iterated split CIF. IEEE Trans Intell Transp Syst, 2022. doi: https://doi.org/10.1109/TITS.2022.3174479

  6. Holz D, Ichim A E, Tombari F, et al. Registration with the point cloud library: a modular framework for aligning in 3-D. IEEE Robot Automat Mag, 2015, 22: 110–124

    Article  Google Scholar 

  7. Chen Y, Medioni G. Object modelling by registration of multiple range images. Image Vision Computing, 1992, 10: 145–155

    Article  Google Scholar 

  8. Besl P J, McKay N D. Method for registration of 3-D shapes. In: Proceedings of Sensor Fusion IV: Control Paradigms and Data Structures, 1992. 586–606

  9. Segal A, Haehnel D, Thrun S. Generalized-ICP. In: Proceedings of Robotics: Science and Systems, Seattle, 2009. 435

  10. Agamennoni G, Fontana S, Siegwart R Y, et al. Point clouds registration with probabilistic data association. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016. 4092–4098

  11. Magnusson M, Lilienthal A, Duckett T. Scan registration for autonomous mining vehicles using 3D-NDT. J Field Robotics, 2007, 24: 803–827

    Article  Google Scholar 

  12. Zhou Q Y, Park J, Koltun V. Fast global registration. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2016. 766–782

    Google Scholar 

  13. Ma J, Jiang X, Fan A, et al. Image matching from handcrafted to deep features: a survey. Int J Comput Vis, 2021, 129: 23–79

    Article  MathSciNet  MATH  Google Scholar 

  14. Rusu R B, Blodow N, Beetz M. Fast point feature histograms (FPFH) for 3D registration. In: Proceedings of IEEE International Conference on Robotics and Automation, 2009. 3212–3217

  15. Aoki Y, Goforth H, Srivatsan R A, et al. PointNetLK: robust & efficient point cloud registration using pointnet. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019. 7163–7172

  16. Wang Y, Solomon J M. Deep closest point: learning representations for point cloud registration. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. 3523–3532

  17. Yew Z J, Lee G H. RPM-Net: robust point matching using learned features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. 11824–11833

  18. van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learning Res, 2008, 9: 2579–2605

    MATH  Google Scholar 

  19. Censi A. An ICP variant using a point-to-line metric. In: Proceedings of IEEE International Conference on Robotics and Automation, 2008. 19–25

  20. Pomerleau F, Colas F, Siegwart R, et al. Comparing ICP variants on real-world data sets. Auton Robot, 2013, 34: 133–148

    Article  Google Scholar 

  21. Yang J, Li H, Campbell D, et al. Go-ICP: a globally optimal solution to 3D ICP point-set registration. IEEE Trans Pattern Anal Mach Intell, 2015, 38: 2241–2254

    Article  Google Scholar 

  22. Rusu R B, Blodow N, Marton Z C, et al. Aligning point cloud views using persistent feature histograms. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. 3384–3391

  23. Yang J, Quan S, Wang P, et al. Evaluating local geometric feature representations for 3D rigid data matching. IEEE Trans Image Process, 2019, 29: 2522–2535

    Article  MATH  Google Scholar 

  24. Ma J, Wu J, Zhao J, et al. Nonrigid point set registration with robust transformation learning under manifold regularization. IEEE Trans Neural Netw Learn Syst, 2018, 30: 3584–3597

    Article  MathSciNet  Google Scholar 

  25. Ma J, Zhao J, Jiang J, et al. Locality preserving matching. Int J Comput Vis, 2019, 127: 512–531

    Article  MathSciNet  MATH  Google Scholar 

  26. Yang H, Shi J, Carlone L. TEASER: fast and certifiable point cloud registration. IEEE Trans Robot, 2020, 37: 314–333

    Article  Google Scholar 

  27. Myronenko A, Song X B. Point set registration: coherent point drift. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 2262–2275

    Article  Google Scholar 

  28. Jian B, Vemuri B C. Robust point set registration using Gaussian mixture models. IEEE Trans Pattern Anal Mach Intell, 2011, 33: 1633–1645

    Article  Google Scholar 

  29. Li L, Yang M, Wang C, et al. Robust point set registration using signature quadratic form distance. IEEE Trans Cybern, 2018, 50: 2097–2109

    Article  Google Scholar 

  30. Su H, Maji S, Kalogerakis E, et al. Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE International Conference on Computer Vision, 2015. 945–953

  31. Wu B, Wan A, Yue X, et al. SqueezeSeg: convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D lidar point cloud. In: Proceedings of IEEE International Conference on Robotics and Automation, 2018. 1887–1893

  32. Tchapmi L, Choy C, Armeni I, et al. SEGCloud: semantic segmentation of 3D point clouds. In: Proceedings of International Conference on 3D Vision, 2017. 537–547

  33. Maturana D, Scherer S. VoxNet: a 3D convolutional neural network for real-time object recognition. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2015. 922–928

  34. Qi C R, Su H, Mo K, et al. PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 652–660

  35. Qi C R, Yi L, Su H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Proceedings of Advances in Neural Information Processing Systems, 2017. 5099–5108

  36. Hua B S, Tran M K, Yeung S K. Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018. 984–993

  37. Lin L Q, Huang P D, Xue F Y, et al. Hausdorff point convolution with geometric priors. Sci China Inf Sci, 2021, 64: 210105

    Article  Google Scholar 

  38. Wang Y, Sun Y, Liu Z, et al. Dynamic graph CNN for learning on point clouds. ACM Trans Graph, 2019, 38: 1–12

    Google Scholar 

  39. Liang Z, Yang M, Deng L, et al. Hierarchical depthwise graph convolutional neural network for 3D semantic segmentation of point clouds. In: Proceedings of International Conference on Robotics and Automation, 2019. 8152–8158

  40. Valsesia D, Fracastoro G, Magli E. Learning localized representations of point clouds with graph-convolutional generative adversarial networks. IEEE Trans Multimedia, 2020, 23: 402–414

    Article  MATH  Google Scholar 

  41. Cao W M, Zheng C T, Yan Z Y, et al. Geometric deep learning: progress, applications and challenges. Sci China Inf Sci, 2022, 65: 126101

    Article  Google Scholar 

  42. Poulenard A, Rakotosaona M J, Ponty Y, et al. Effective rotation-invariant point CNN with spherical harmonics kernels. In: Proceedings of International Conference on 3D Vision, 2019. 47–56

  43. Li X, Li R, Chen G, et al. A rotation-invariant framework for deep point cloud analysis. IEEE Trans Visual Comput Graph, 2021. doi: https://doi.org/10.1109/TVCG.2021.3092570

  44. Chen C, Li G, Xu R, et al. ClusterNet: deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 4994–5002

  45. You Y, Lou Y, Liu Q, et al. Pointwise rotation-invariant network with adaptive sampling and 3D spherical voxel convolution. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2020. 12717–12724

  46. Deng H, Birdal T, Ilic S. PPFNet: global context aware local features for robust 3D point matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018. 195–205

  47. Deng H, Birdal T, Ilic S. PPF-FoldNet: unsupervised learning of rotation invariant 3D local descriptors. In: Proceedings of the European Conference on Computer Vision (ECCV), 2018. 602–618

  48. Ao S, Hu Q, Yang B, et al. SpinNet: learning a general surface descriptor for 3D point cloud registration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. 11753–11762

  49. Lu W, Wan G, Zhou Y, et al. DeepVCP: an end-to-end deep neural network for point cloud registration. In: Proceedings of the IEEE International Conference on Computer Vision, 2019. 12–21

  50. Khazari A E, Que Y, Sung T L, et al. Deep global features for point cloud alignment. Sensors, 2020, 20: 4032

    Article  Google Scholar 

  51. Gojcic Z, Zhou C, Wegner J D, et al. Learning multiview 3D point cloud registration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. 1759–1769

  52. Drost B, Ulrich M, Navab N, et al. Model globally, match locally: efficient and robust 3D object recognition. In: Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010. 998–1005

  53. Huang H, Li D, Zhang H, et al. Consolidation of unorganized point clouds for surface reconstruction. ACM Trans Graph, 2009, 28: 1–7

    Article  Google Scholar 

  54. Wu Z, Song S, Khosla A, et al. 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 1912–1920

  55. Turk G, Levoy M. The Stanford 3D Scanning Repository. Stanford University Computer Graphics Laboratory, 2005. http://graphics.stanford.edu/data/3Dscanrep

  56. Zeng A, Song S, Nießner M, et al. 3DMatch: learning local geometric descriptors from RGB-D reconstructions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 1802–1811

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 62173228, 61873165).

Author information

Author notes

  1. Zhao H W and Liang Z D have the same contribution to this work.

Authors and Affiliations

  1. Department of Automation, Shanghai Jiao Tong University, Shanghai, 200240, China

    Hengwang Zhao, Yuesheng He, Chunxiang Wang & Ming Yang

  2. Research Institute of Robotics, Shanghai Jiao Tong University, Shanghai, 200240, China

    Zhidong Liang

Authors
  1. Hengwang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhidong Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuesheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunxiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ming Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, H., Liang, Z., He, Y. et al. Cy-CNN: cylinder convolution based rotation-invariant neural network for point cloud registration. Sci. China Inf. Sci. 66, 152102 (2023). https://doi.org/10.1007/s11432-021-3570-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3570-5

Keywords

Search

Navigation