Advertisement

3D-Rotation-Equivariant Quaternion Neural Networks

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12365)

Abstract

This paper proposes a set of rules to revise various neural networks for 3D point cloud processing to rotation-equivariant quaternion neural networks (REQNNs). We find that when a neural network uses quaternion features, the network feature naturally has the rotation-equivariance property. Rotation equivariance means that applying a specific rotation transformation to the input point cloud is equivalent to applying the same rotation transformation to all intermediate-layer quaternion features. Besides, the REQNN also ensures that the intermediate-layer features are invariant to the permutation of input points. Compared with the original neural network, the REQNN exhibits higher rotation robustness.

Keywords

Rotation equivariance Permutation invariance 3D point cloud processing Quaternion 

Notes

Acknowledgments

The work is partially supported by the National Key Research and Development Project (No. 213), the National Nature Science Foundation of China (No. 61976160, U19B2043, and 61906120), the Special Project of the Ministry of Public Security (No. 20170004), and the Key Lab of Information Network Security, Ministry of Public Security (No.C18608).

Supplementary material

504476_1_En_32_MOESM1_ESM.pdf (237 kb)
Supplementary material 1 (pdf 236 KB)

References

  1. 1.
  2. 2.
    Arjovsky, M., Shah, A., Bengio, Y.: Unitary evolution recurrent neural networks. In: International Conference on Machine Learning, pp. 1120–1128 (2016)Google Scholar
  3. 3.
    Chang, A.X., et al.: ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012 (2015)
  4. 4.
    Chen, C., Li, G., Xu, R., Chen, T., Wang, M., Lin, L.: ClusterNet: deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4994–5002 (2019)Google Scholar
  5. 5.
    Cohen, T.S., Geiger, M., Köhler, J., Welling, M.: Spherical CNNs. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=Hkbd5xZRb
  6. 6.
    Cohen, T.S., Welling, M.: Steerable CNNs. arXiv preprint arXiv:1612.08498 (2016)
  7. 7.
    Danihelka, I., Wayne, G., Uria, B., Kalchbrenner, N., Graves, A.: Associative long short-term memory. arXiv preprint arXiv:1602.03032 (2016)
  8. 8.
    Deng, H., Birdal, T., Ilic, S.: PPF-FoldNet: unsupervised learning of rotation invariant 3D local descriptors. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11209, pp. 620–638. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01228-1_37CrossRefGoogle Scholar
  9. 9.
    Gaudet, C.J., Maida, A.S.: Deep quaternion networks. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2018)Google Scholar
  10. 10.
    Guberman, N.: On complex valued convolutional neural networks. arXiv preprint arXiv:1602.09046 (2016)
  11. 11.
    Hamilton, W.R.: On quaternions; or on a new system of imaginaries in algebra. Lond. Edinb. Dublin Philos. Mag. J. Sci. 33(219), 58–60 (1848)Google Scholar
  12. 12.
    Hua, B.S., Tran, M.K., Yeung, S.K.: Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 984–993 (2018)Google Scholar
  13. 13.
    Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 (2015)Google Scholar
  14. 14.
    Jiang, M., Wu, Y., Zhao, T., Zhao, Z., Lu, C.: PointSIFT: a SIFT-like network module for 3D point cloud semantic segmentation. arXiv preprint arXiv:1807.00652 (2018)
  15. 15.
    Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)Google Scholar
  16. 16.
    Klokov, R., Lempitsky, V.: Escape from cells: deep kd-networks for the recognition of 3D point cloud models. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 863–872 (2017)Google Scholar
  17. 17.
    Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on x-transformed points. In: Advances in Neural Information Processing Systems, pp. 820–830 (2018)Google Scholar
  18. 18.
    Liu, X., Han, Z., Liu, Y.S., Zwicker, M.: Point2Sequence: learning the shape representation of 3D point clouds with an attention-based sequence to sequence network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8778–8785 (2019)Google Scholar
  19. 19.
    Liu, Y., Fan, B., Xiang, S., Pan, C.: Relation-shape convolutional neural network for point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8895–8904 (2019)Google Scholar
  20. 20.
    Parcollet, T., et al.: Quaternion convolutional neural networks for end-to-end automatic speech recognition. In: Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, pp. 22–26 (2018)Google Scholar
  21. 21.
    Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)Google Scholar
  22. 22.
    Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)Google Scholar
  23. 23.
    Rao, Y., Lu, J., Zhou, J.: Spherical fractal convolutional neural networks for point cloud recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 452–460 (2019)Google Scholar
  24. 24.
    Shen, Y., Feng, C., Yang, Y., Tian, D.: Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4548–4557 (2018)Google Scholar
  25. 25.
    Shi, S., Wang, X., Li, H.: PointRCNN: 3D object proposal generation and detection from point cloud. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–779 (2019)Google Scholar
  26. 26.
    Shuster, M.D., et al.: A survey of attitude representations. Navigation 8(9), 439–517 (1993)MathSciNetGoogle Scholar
  27. 27.
    Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3693–3702 (2017)Google Scholar
  28. 28.
    Su, H., et al.: SPLATNet: sparse lattice networks for point cloud processing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2530–2539 (2018)Google Scholar
  29. 29.
    Thomas, N., et al.: Tensor field networks: rotation-and translation-equivariant neural networks for 3D point clouds. arXiv preprint arXiv:1802.08219 (2018)
  30. 30.
    Trabelsi, C., et al.: Deep complex networks. arXiv preprint arXiv:1705.09792 (2017)
  31. 31.
    Van Dyk, D.A., Meng, X.L.: The art of data augmentation. J. Comput. Graph. Stat. 10(1), 1–50 (2001)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Wang, W., Yu, R., Huang, Q., Neumann, U.: SGPN: similarity group proposal network for 3D point cloud instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2569–2578 (2018)Google Scholar
  33. 33.
    Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. arXiv preprint arXiv:1801.07829 (2018)
  34. 34.
    Weisstein, E.W.: Euler angles (2009)Google Scholar
  35. 35.
    Wisdom, S., Powers, T., Hershey, J., Le Roux, J., Atlas, L.: Full-capacity unitary recurrent neural networks. In: Advances in Neural Information Processing Systems, pp. 4880–4888 (2016)Google Scholar
  36. 36.
    Wolter, M., Yao, A.: Complex gated recurrent neural networks. In: Advances in Neural Information Processing Systems, pp. 10536–10546 (2018)Google Scholar
  37. 37.
    Wu, W., Qi, Z., Fuxin, L.: PointConv: deep convolutional networks on 3D point clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9621–9630 (2019)Google Scholar
  38. 38.
    Wu, Z., et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)Google Scholar
  39. 39.
    Xiang, L., Ma, H., Zhang, H., Zhang, Y., Zhang, Q.: Complex-valued neural networks for privacy protection. arXiv preprint arXiv:1901.09546 (2019)
  40. 40.
    You, Y., et al.: PRIN: pointwise rotation-invariant network. arXiv preprint arXiv:1811.09361 (2018)
  41. 41.
    Yu, L., Li, X., Fu, C.W., Cohen-Or, D., Heng, P.A.: PU-Net: point cloud upsampling network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2790–2799 (2018)Google Scholar
  42. 42.
    Zhang, Y., Lu, Z., Xue, J.H., Liao, Q.: A new rotation-invariant deep network for 3D object recognition. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), pp. 1606–1611. IEEE (2019)Google Scholar
  43. 43.
    Zhang, Z., Hua, B.S., Rosen, D.W., Yeung, S.K.: Rotation invariant convolutions for 3D point clouds deep learning. In: 2019 International Conference on 3D Vision (3DV), pp. 204–213. IEEE (2019)Google Scholar
  44. 44.
    Zhao, H., Jiang, L., Fu, C.W., Jia, J.: PointWeb: enhancing local neighborhood features for point cloud processing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5565–5573 (2019)Google Scholar
  45. 45.
    Zhao, Y., Birdal, T., Lenssen, J.E., Menegatti, E., Guibas, L., Tombari, F.: Quaternion equivariant capsule networks for 3D point clouds. arXiv preprint arXiv:1912.12098 (2019)
  46. 46.
    Zhu, X., Xu, Y., Xu, H., Chen, C.: Quaternion convolutional neural networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 645–661. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01237-3_39CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Shanghai Jiao Tong UniversityShanghaiChina
  2. 2.Tongji UniversityShanghaiChina

Personalised recommendations