Abstract
In recent years, convolutional neural networks on point clouds have greatly improved the performance of point cloud classification and segmentation. However, the irregularity and disorder of point clouds make the convolution operation ill-suited to preserve the spatial-local structure and make the existing convolution networks very shallow. In order to solve the problems, we propose a novel pre-processor network named KeypointNet which ranks the point cloud according to the contribution to the final task, such as classification and segmentation. With the ordered point cloud, a convolution neural network on point clouds goes as deeper as possible similar to that on images. Two scoring mechanisms: directly scoring (DS) and gradually scoring (GS), are designed based on a pre-trained PointNet which is easily and fast trained. Both scoring mechanisms score a point depending on its contribution to the output of PointNet. The former scores the point cloud only one time and the later scores the point cloud on grade level. KeypointNet can be unified with any existing convolution neural networks on point clouds. Extensive experiments demonstrate that our method is effective and efficient and achieves SOTA classification results and comparable segmentation results, while greatly reducing space consumption.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ben-Shabat, Y., Lindenbaum, M., Fischer, A.: 3dmfv: 3d point cloud classification in real-time using convolutional neural network. IEEE Robot. Autom. Lett. (99), 1 (2018)
Cao, Z., Huang, Q., Karthik, R.: 3d object classification via spherical projections. In: 2017 International Conference on 3D Vision (3DV), pp. 566–574. IEEE (2017)
Chen, S., Zheng, L., Zhang, Y., Sun, Z., Xu, K.: Veram: view-enhanced recurrent attention model for 3d shape classification. IEEE Trans. Visual. Comput. Graph. 25(12), 3244–3257 (2018)
Cheraghian, A., Petersson, L.: 3dcapsule: Extending the capsule architecture to classify 3d point clouds. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1194–1202. IEEE (2019)
Gao, H., Ji, S.: Graph u-nets. arXiv preprint arXiv:1905.05178 (2019)
Hua, B.S., Tran, M.K., Yeung, S.K.: Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 984–993 (2018)
Klokov, R., Lempitsky, V.: Escape from cells: deep kd-networks for the recognition of 3d point cloud models. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 863–872 (2017)
Li, J., Chen, B.M., Hee Lee, G.: So-net: Self-organizing network for point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9397–9406 (2018)
Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: Pointcnn: convolution on x-transformed points. In: Advances in Neural Information Processing Systems, pp. 820–830 (2018)
Liu, X., Han, Z., Liu, Y.S., Zwicker, M.: Point2sequence: Learning the shape representation of 3d point clouds with an attention-based sequence to sequence network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8778–8785 (2019)
Mao, J., Wang, X., Li, H.: Interpolated convolutional networks for 3d point cloud understanding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1578–1587 (2019)
Nezhadarya, E., Taghavi, E., Liu, B., Luo, J.: Adaptive hierarchical down-sampling for point cloud classification. arXiv preprint arXiv:1904.08506 (2019)
Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum pointnets for 3d object detection from rgb-d data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 918–927 (2018)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)
Riegler, G., Osman Ulusoy, A., Geiger, A.: Octnet: learning deep 3d representations at high resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3577–3586 (2017)
Shen, Y., Feng, C., Yang, Y., Tian, D.: Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4548–4557 (2018)
Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: Flexible and deformable convolution for point clouds. arXiv preprint arXiv:1904.08889 (2019)
Uy, M.A., Pham, Q.H., Hua, B.S., Nguyen, T., Yeung, S.K.: Revisiting point cloud classification: a new benchmark dataset and classification model on real-world data. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1588–1597 (2019)
Wang, P.S., Liu, Y., Guo, Y.X., Sun, C.Y., Tong, X.: O-cnn: octree-based convolutional neural networks for 3d shape analysis. Acm Trans. Graph. 36(4), 72 (2017)
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph cnn for learning on point clouds. ACM Trans. Graph. (TOG) 38(5), 146 (2019)
Wu, W., Qi, Z., Fuxin, L.: Pointconv: deep convolutional networks on 3d point clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9621–9630 (2019)
Wu, Z., et al.: 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)
Xie, S., Liu, S., Chen, Z., Tu, Z.: Attentional shapecontextnet for point cloud recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4606–4615 (2018)
Xu, Y., Fan, T., Xu, M., Zeng, L., Qiao, Y.: Spidercnn: deep learning on point sets with parameterized convolutional filters. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 87–102 (2018)
Yavartanoo, M., Kim, E.Y., Lee, K.M.: SPNet: deep 3D object classification and retrieval using stereographic projection. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11365, pp. 691–706. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20873-8_44
Zhang, K., et al.: Environmental features recognition for lower limb prostheses toward predictive walking. IEEE Trans. Neural Syst. Rehabil. Eng. 27(3), 465–476 (2019)
Zhang, Z., Hua, B.S., Yeung, S.K.: Shellnet: Efficient point cloud convolutional neural networks using concentric shells statistics. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1607–1616 (2019)
Zhu, Y., et al.: Target-driven visual navigation in indoor scenes using deep reinforcement learning. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3357–3364. IEEE (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, Z., Qu, Y., Li, C. (2021). KeypointNet: Ranking Point Cloud for Convolution Neural Network. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-87361-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87360-8
Online ISBN: 978-3-030-87361-5
eBook Packages: Computer ScienceComputer Science (R0)