Skip to main content
Log in

3D point cloud descriptors: state-of-the-art

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

The development of inexpensive 3D data acquisition devices has promisingly facilitated the wide availability and popularity of point clouds, which attracts increasing attention to the effective extraction of 3D point cloud descriptors for accuracy of the efficiency of 3D computer vision tasks in recent years. However, how to develop discriminative and robust feature representations from 3D point clouds remains a challenging task due to their intrinsic characteristics. In this paper, we give a comprehensively insightful investigation of the existing 3D point cloud descriptors. These methods can be principally divided into two categories according to their advancement: hand-crafted and deep learning-based approaches, which will be further discussed from the perspective of elaborate classification, their advantages, and limitations. Finally, we present the future research directions of the extraction of 3D point cloud descriptors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Aldoma A, Marton ZC, Tombari F, Wohlkinger W, Potthast C, Zeisl B, Rusu RB, Gedikli S, Vincze M (2012) Tutorial: point cloud library: three-dimensional object recognition and 6 dof pose estimation. IEEE Robotics Autom Mag 19(3):80–91

    Google Scholar 

  • Aldoma, A., Tombari, F., Rusu, R.B., Vincze, M.: Our-cvfh - oriented, unique and repeatable clustered viewpoint feature histogram for object recognition and 6dof pose estimation. In: Pattern Recognition, pp. 113–122 (2012)

  • Aldoma, A., Vincze, M., Blodow, N., Gossow, D., Gedikli, S., Rusu, R.B., Bradski, G.: Cad-model recognition and 6dof pose estimation using 3d cues. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 585–592 (2012)

  • Alexander Patterson, I.V., Mordohai, P., Daniilidis, K.: Object detection from large-scale 3d datasets using bottom-up and top-down descriptors. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 553–566. Springer Berlin Heidelberg (2008)

  • Alexandre, L.A.: 3d descriptors for object and category recognition: a comparative evaluation. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) workshop, vol. 1, p. 7 (2012)

  • Ali H, Shafait F, Giannakidou E, Vakali A, Figueroa N, Varvadoukas T, Mavridis N (2014) Contextual object category recognition for rgb-d scene labeling. Robotics Autonom Syst 62(2):241–256

    Google Scholar 

  • Anand A, Koppula HS, Joachims T, Saxena A (2013) Contextually guided semantic labeling and search for three-dimensional point clouds. Int J Robotics Res 32(1):19–34

    Google Scholar 

  • Anguelov, D., Taskarf, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., Ng, A.: Discriminative learning of markov random fields for segmentation of 3d scan data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 169–176 (2005)

  • Armeni I, Sax S, Zamir AR, Savarese S (2017) Joint 2d–3d-semantic data for indoor scene understanding. arXiv preprint arXiv 1702:01105

    Google Scholar 

  • Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S.: 3d semantic parsing of large-scale indoor spaces. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1534–1543 (2016)

  • Atzmon, M., Maron, H., Lipman, Y.: Point convolutional neural networks by extension operators. ACM Transactions on Graphics 37(4) (2018)

  • Barnes, D., Gadd, M., Murcutt, P., Newman, P., Posner, I.: The oxford radar robotcar dataset: A radar extension to the oxford robotcar dataset. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 6433–6438 (2020)

  • Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C., Gall, J.: Semantickitti: A dataset for semantic scene understanding of lidar sequences. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9296–9306 (2019)

  • Behley, J., Steinhage, V., Cremers, A.B.: Performance of histogram descriptors for the classification of 3d laser range data in urban environments. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 4391–4398 (2012)

  • Beksi, W.J., Papanikolopoulos, N.: Object classification using dictionary learning and rgb-d covariance descriptors. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1880–1885 (2015)

  • Ben-Shabat Y, Lindenbaum M, Fischer A (2018) 3dmfv: 3d point cloud classification in real-time using convolutional neural network. IEEE Robotics Autom Lett 99:1–1

    Google Scholar 

  • Beserra Gomes, R., Marques Ferreira Da Silva, B., Rocha, L.K.D.M., Aroca, R.V., Gon, Alves, L.M.G.: Efficient 3d object recognition using foveated point clouds. Computers and Graphics 37(5), 496–508 (2013)

  • Bo, L., Ren, X., Fox, D.: Depth kernel descriptors for object recognition. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), vol. 32, pp. 821–826 (2011)

  • Chan KC, Koh CK, Lee CSG (2014) A 3-d-point-cloud system for human-pose estimation. IEEE Trans Syst Man and Cybern Syst 44(11):1486–1497

    Google Scholar 

  • Chang AX, Funkhouser T, Guibas L, Hanrahan P, Huang Q, Li Z, Savarese S, Savva M, Song S, Su H (2015) Shapenet: An information-rich 3d model repository. Comput Sci 1512:03012

    Google Scholar 

  • Charles, R.Q., Hao, S., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  • Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. pp. 1–12. British Machine Vision Association (2014)

  • Chen, S., Niu, S., Lan, T., Liu, B.: Pct: Large-scale 3d point cloud representations via graph inception networks with applications to autonomous driving. In: IEEE International Conference on Image Processing (ICIP), pp. 4395–4399. IEEE (2019)

  • Chen, T., Dai, B., Liu, D., Song, J.: Performance of global descriptors for velodyne-based urban object recognition. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, pp. 667–673. IEEE (2014)

  • Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3d object detection network for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1907–1915 (2017)

  • Cheng, J., Xiang, Z., Cao, T., Liu, J.: Robust vehicle detection using 3d lidar under complex urban environment. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 691–696 (2014)

  • Cheng S, Chen X, He X, Liu Z, Bai X (2021) Pra-net: point relation-aware network for 3d point cloud analysis. IEEE Trans Image Proc 30:4436–4448

    Google Scholar 

  • Choe, J., Park, C., Rameau, F., Park, J., Kweon, I.S.: Pointmixer: Mlp-mixer for point cloud understanding. In: European Conference on Computer Vision, pp. 620–640. Springer (2022)

  • Cirujeda, P., Mateo, X., Dicente, Y., Binefa, X.: Mcov: A covariance descriptor for fusion of texture and shape features in 3d point clouds. In: Proceedings of the International Conference on 3d Vision, pp. 551–558 (2014)

  • Cui Y, Chen R, Chu W, Chen L, Tian D, Li Y, Cao D (2021) Deep learning for image and point cloud fusion in autonomous driving: a review. IEEE Trans on Intell Transp Syst 23(2):722–739

    Google Scholar 

  • Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: Scannet: Richly-annotated 3d reconstructions of indoor scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5828–5839 (2017)

  • Dai, A., Nießner, M.: 3dmv: Joint 3d-multi-view prediction for 3d semantic scene segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 452–468 (2018)

  • De Deuge, M., Quadros, A., Hung, C., Douillard, B.: Unsupervised feature learning for classification of outdoor 3d scans. In: Australasian Conference on Robitics and Automation, vol. 2, p. 1 (2013)

  • Deng, H., Birdal, T., Ilic, S.: Ppf-foldnet: Unsupervised learning of rotation invariant 3d local descriptors. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 602–618 (2018)

  • Deng, H., Birdal, T., Ilic, S.: Ppfnet: Global context aware local features for robust 3d point matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 195–205 (2018)

  • Deng, S., Xu, X., Wu, C., Chen, K., Jia, K.: 3d affordancenet: A benchmark for visual object affordance understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1778–1787 (2021)

  • Diniz, R., Freitas, P.G., Farias, M.C.: Local luminance patterns for point cloud quality assessment. In: 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6. IEEE (2020)

  • Diniz, R., Freitas, P.G., Farias, M.C.: Towards a point cloud quality assessment model using local binary patterns. In: 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–6. IEEE (2020)

  • Diniz R, Freitas PG, Farias MC (2021) Color and geometry texture descriptors for point-cloud quality assessment. IEEE Signal Proc Lett 28:1150–1154

    Google Scholar 

  • Diniz R, Freitas PG, Farias MC (2022) Point cloud quality assessment based on geometry-aware texture descriptors. Comput Graph 103:31–44

    Google Scholar 

  • Dong Z, Yang B, Liang F, Huang R, Scherer S (2018) Hierarchical registration of unordered tls point clouds based on binary shape context descriptor. ISPRS J Photogramm Remote Sens 144:61–79

    Google Scholar 

  • Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally: Efficient and robust 3d object recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 998–1005 (2010)

  • Duan, Y., Zheng, Y., Lu, J., Zhou, J., Tian, Q.: Structural relational reasoning of point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 949–958 (2019)

  • Engelcke, M., Rao, D., Wang, D.Z., Tong, C.H., Posner, I.: Vote3deep: Fast object detection in 3d point clouds using efficient convolutional neural networks. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1355–1361. IEEE (2017)

  • Fehr, D., Beksi, W.J., Zermas, D., Papanikolopoulos, N.: Rgb-d object classification using covariance descriptors. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 5467–5472 (2014)

  • Fehr, D., Cherian, A., Sivalingam, R., Nickolay, S.: Compact covariance descriptors in 3d point clouds for object recognition. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1793–1798 (2012)

  • Feng, Y., Zhang, Z., Zhao, X., Ji, R., Gao, Y.: Gvcnn: Group-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 264–272 (2018)

  • Fiolka, T., Stuckler, J., Klein, D.A., Schulz, D., Behnke, S.: Sure: Surface entropy for distinctive 3d features. In: Proceedings of the International Conference on Spatial Cognition, pp. 74–93 (2012)

  • Flint, A., Dick, A., Hengel, A.V.D.: Thrift: Local 3d structure recognition. In: Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA), pp. 182–188 (2007)

  • Frome A, Huber D, Kolluri R, Bülow T, Malik J (2004) Recognizing objects in range data using regional point descriptors. Springer, Berlin, pp 224–237

    MATH  Google Scholar 

  • Gadelha, M., Wang, R., Maji, S.: Multiresolution tree networks for 3d point cloud processing. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 103–118 (2018)

  • Ge X (2016) Non-rigid registration of 3d point clouds under isometric deformation. ISPRS J Photogramm Remote Sens 121:192–202

    Google Scholar 

  • Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3354–3361. IEEE (2012)

  • Ghorbani F, Ebadi H, Sedaghat A, Pfeifer N (2022) A novel 3-d local daisy-style descriptor to reduce the effect of point displacement error in point cloud registration. IEEE J Selected Topics Appl Earth Obs Remote Sens 15:2254–2273

    Google Scholar 

  • Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1440–1448 (2015)

  • Golovinskiy, A., Kim, V.G., Funkhouser, T.: Shape-based recognition of 3d point clouds in urban environments. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2154–2161 (2009)

  • Graham, B., Engelcke, M., van der Maaten, L.: 3d semantic segmentation with submanifold sparse convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9224–9232 (2018)

  • Guinard, S., Landrieu, L.: Weakly supervised segmentation-aided classification of urban scenes from 3d lidar point clouds. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences 42 (2017)

  • Guo MH, Cai JX, Liu ZN, Mu TJ, Martin RR, Hu SM (2021) Pct: point cloud transformer. Comput Visual Media 7(2):187–199

    Google Scholar 

  • Guo Y, Bennamoun M, Sohel F, Lu M, Wan J, Kwok NM (2016) A comprehensive performance evaluation of 3d local feature descriptors. Int J Comput Vision 116(1):66–89

    MathSciNet  Google Scholar 

  • Guo Y, Sohel F, Bennamoun M, Lu M, Wan J (2013) Rotational projection statistics for 3d local surface description and object recognition. Int J Comput Vision 105(1):63–86

    MathSciNet  MATH  Google Scholar 

  • Guo Y, Wang H, Hu Q, Liu H, Bennamoun M (2020) Deep learning for 3d point clouds: a survey. IEEE Trans Pattern Anal Mach Intell 12:4338–4364

    Google Scholar 

  • Hackel, T., Savinov, N., Ladicky, L., Wegner, J., Schindler, K., Pollefeys, M.: Semantic3d. net: A new large-scale point cloud classification benchmark. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 4, 91 (2017)

  • Hadji, I., Desouza, G.N.: Local-to-global signature descriptor for 3d object recognition. In: Proceedings of the Asian Conference on Computer Vision (ACCV), pp. 570–584 (2014)

  • Han, W., Wen, C., Wang, C., Li, X., Li, Q.: Point2node: Correlation learning of dynamic-node for point cloud feature modeling. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 10925–10932 (2020)

  • Han, X., Leung, T., Jia, Y., Sukthankar, R., Berg, A.C.: Matchnet: Unifying feature and metric learning for patch-based matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3279–3286 (2015)

  • Han XF, He ZY, Chen J, Xiao GQ (2022) 3crossnet: cross-level cross-scale cross-attention network for point cloud representation. IEEE Robotics Autom Lett 7(2):3718–3725

    Google Scholar 

  • Han XF, Jin JS, Wang MJ, Jiang W, Gao L, Xiao L (2017) A review of algorithms for filtering the 3d point cloud. Signal Proc Image Commun 57:103–112

    Google Scholar 

  • Han XF, Jin YF, Cheng HX, Xiao GQ (2022) Dual transformer for point cloud analysis. IEEE Trans Multimedia 10:55–82

    Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Tran Pattern Anal Mach Intell 37(9):1904–16

    Google Scholar 

  • Himmelsbach, M., Luettel, T., Wuensche, H.J.: Real-time object classification in 3d point clouds using point feature histograms. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 994–1000 (2009)

  • Hu, Q., Yang, B., Khalid, S., Xiao, W., Trigoni, N., Markham, A.: Towards semantic segmentation of urban-scale 3d point clouds: A dataset, benchmarks and challenges. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4977–4987 (2021)

  • Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., Markham, A.: Randla-net: Efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11108–11117 (2020)

  • Huang, J., You, S.: Point cloud matching based on 3d self-similarity. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 41–48 (2012)

  • Huang, J., You, S.: Detecting objects in scene point cloud: A combinational approach. In: Proceedings of the International Conference on 3D Vision, pp. 175–182 (2013)

  • Huang, Q., Wang, W., Neumann, U.: Recurrent slice networks for 3d segmentation of point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2626–2635 (2018)

  • Huang, Z., Yu, Y., Xu, J., Ni, F., Le, X.: Pf-net: Point fractal network for 3d point cloud completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7662–7670 (2020)

  • Huang, Z., Zhao, Z., Zhou, H., Zhao, X., Gao, Y.: Deepccfv: Camera constraint-free multi-view convolutional neural network for 3d object retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8505–8512 (2019)

  • Hwang, H., Hyung, S., Yoon, S., Roh, K.: Robust descriptors for 3d point clouds using geometric and photometric local feature. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4027–4033 (2012)

  • Ip, C.Y., Lapadat, D., Sieger, L., Regli, W.C.: Using shape distributions to compare solid models. In: Proceedings of the ACM Symposium on Solid Modeling and Applications, pp. 273–280 (2002)

  • Jaritz, M., Gu, J., Su, H.: Multi-view pointnet for 3d scene understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 0–0 (2019)

  • Jaritz, M., Vu, T.H., Charette, R.d., Wirbel, E., Pérez, P.: xmuda: Cross-modal unsupervised domain adaptation for 3d semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12605–12614 (2020)

  • Jiang, L., Zhao, H., Liu, S., Shen, X., Jia, J.: Hierarchical point-edge interaction network for point cloud semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10433–10441 (2019)

  • Johnson AE, Hebert M (1998) Surface matching for object recognition in complex 3-d scenes. Image Vision Comput 16(9–10):635–651

    Google Scholar 

  • Johnson AE, Hebert M (1999) Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans Pattern Anal Mach Intell 21(5):433–449

    Google Scholar 

  • Kahler, O., Reid, I.: Efficient 3d scene labeling using fields of trees. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3064–3071 (2013)

  • Kasaei SH, Tome AM, Lopes LS, Oliveira M (2016) Good: a global orthographic object descriptor for 3d object recognition and manipulation. Pattern Recognit Lett 83:312–320

    Google Scholar 

  • Khan S, Naseer M, Hayat M, Zamir SW, Khan FS, Shah M (2022) Transformers in vision: a survey. ACM Comput Surv (CSUR) 54(10s):1–41

    Google Scholar 

  • Khoury, M., Zhou, Q.Y., Koltun, V.: Learning compact geometric features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 153–161 (2017)

  • Klokov, R., Lempitsky, V.: Escape from cells: Deep kd-networks for the recognition of 3d point cloud models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 863–872 (2017)

  • Lalonde JF, Vandapel N, Huber DF, Hebert M (2006) Natural terrain classification using three-dimensional ladar data for ground robot mobility. J Field Robotics 23(10):839–861

    Google Scholar 

  • Landrieu, L., Simonovsky, M.: Large-scale point cloud semantic segmentation with superpoint graphs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4558–4567 (2018)

  • Lehtomaki, M., Jaakkola, A., Hyyppa, J., Lampinen, J., Kaartinen, H., Kukko, A., Puttonen, E., Hyyppä, H.: Object classification and recognition from mobile laser scanning point clouds in a road environment. IEEE Transactions on Geoscience and Remote Sensing 54(2), 1226–1239 (2016)

  • Lei, H., Akhtar, N., Mian, A.: Octree guided cnn with spherical kernels for 3d point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9631–9640 (2019)

  • Li, J., Chen, B.M., Hee Lee, G.: So-net: Self-organizing network for point cloud analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9397–9406 (2018)

  • Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: Pointcnn: Convolution on x-transformed points. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 820–830 (2018)

  • Lin B, Wang F, Zhao F, Sun Y (2018) Scale invariant point feature (sipf) for 3d point clouds and 3d multi-scale object detection. Neural Comput Appl 29(5):1209–1224

    Google Scholar 

  • Liu, X., Han, Z., Liu, Y.S., Zwicker, M.: Point2sequence: Learning the shape representation of 3d point clouds with an attention-based sequence to sequence network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8778–8785 (2019)

  • Liu, Y., Fan, B., Xiang, S., Pan, C.: Relation-shape convolutional neural network for point cloud analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8895–8904 (2019)

  • Logoglu KB, Kalkan S, Temizel A (2016) Cospair: colored histograms of spatial concentric surflet-pairs for 3d object recognition. Robotics Auton Syst 75:558–570

    Google Scholar 

  • Lu, D., Xie, Q., Wei, M., Xu, L., Li, J.: Transformers in 3d point clouds: A survey. arXiv preprint arXiv:2205.07417 (2022)

  • do, M., Lima, J.P.S., Teichrieb, V.: An efficient global point cloud descriptor for object recognition and pose estimation. In: Proceedings of the SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 56–63 (2016)

  • Ma, X., Qin, C., You, H., Ran, H., Fu, Y.: Rethinking network design and local geometry in point cloud: A simple residual mlp framework. In: International Conference on Learning Representations (2021)

  • Madry, M., Ek, C.H., Detry, R., Hang, K.: Improving generalization for 3d object categorization with global structure histograms. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1379–1386 (2012)

  • Madry, M., Song, D., Kragic, D.: From object categories to grasp transfer using probabilistic reasoning. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1716–1723 (2012)

  • Marton ZC, Pangercic D, Blodow N, Beetz M (2011) Combined 2d–3d categorization and classification for multimodal perception systems. Int J Robotics Res 30(11):1378–1402

    Google Scholar 

  • Marton, Z.C., Pangercic, D., Blodow, N., Kleinehellefort, J., Beetz, M.: General 3d modelling of novel objects from a single view. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3700–3705 (2010)

  • Matei B, Shan Y, Sawhney HS, Tan Y, Kumar R, Huber D, Hebert M (2006) Rapid object indexing using locality sensitive hashing and joint 3d-signature space estimation. IEEE Tran Pattern Anal Mach Intell 28(7):1111–1126

    Google Scholar 

  • Mateo, C.M., Gil, P., Torres, F.: A performance evaluation of surface normals-based descriptors for recognition of objects using cad-models. In: Proceedings of the International Conference on Informatics in Control, Automation and Robotics (ICINCO), pp. 428–435 (2014)

  • Maturana, D., Scherer, S.: Voxnet: A 3d convolutional neural network for real-time object recognition. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928. IEEE (2015)

  • Muja, M., Rusu, R.B., Bradski, G., Lowe, D.G.: Rein - a fast, robust, scalable recognition infrastructure. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2939–2946 (2011)

  • Munoz, D., Bagnell, J.A., Hebert, M.: Co-inference for multi-modal scene analysis. In: Proceedings of the 12th European conference on Computer Vision (ECCV), pp. 668–681 (2012)

  • Munoz, D., Bagnell, J.A., Vandapel, N., Hebert, M.: Contextual classification with functional max-margin markov networks. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 975–982. IEEE (2009)

  • Phong BT (1975) Illumination for computer generated pictures. ACM Commun 18(6):311–317

    Google Scholar 

  • Prakhya SM, Lin J, Chandrasekhar V, Lin W, Liu B (2017) 3dhopd: a fast low-dimensional 3-d descriptor. IEEE Robotics Automation Lett 2(3):1472–1479

    Google Scholar 

  • Prakhya, S.M., Liu, B., Lin, W.: B-shot: A binary feature descriptor for fast and efficient keypoint matching on 3d point clouds. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1929–1934 (2015)

  • Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 652–660 (2017)

  • Qi, C.R., Su, H., Nießner¸ner, M., Dai, A., Yan, M., Guibas, L.J.: Volumetric and multi-view cnns for object classification on 3d data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5648–5656 (2016)

  • Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)

  • Radu, B.S., Rusu, B., Konolige, K., Burgard, W.: Narf: 3d range image features for object recognition. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop, vol. 44 (2010)

  • Rahmani, H., Mahmood, A., Du, Q.H., Mian, A.: Hopc: Histogram of oriented principal components of 3d pointclouds for action recognition. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 742–757 (2014)

  • Rakotosaona, M.J., La Barbera, V., Guerrero, P., Mitra, N.J., Ovsjanikov, M.: Pointcleannet: Learning to denoise and remove outliers from dense point clouds. In: Computer Graphics Forum, vol. 39, pp. 185–203. Wiley Online Library (2020)

  • Ran, H., Liu, J., Wang, C.: Surface representation for point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18942–18952 (2022)

  • Rao, Y., Lu, J., Zhou, J.: Spherical fractal convolutional neural networks for point cloud recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 452–460 (2019)

  • Ravanbakhsh, S., Schneider, J., Poczos, B.: Deep learning with sets and point clouds. arXiv preprint arXiv:1611.04500 (2017)

  • Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  • Riegler, G., Osman Ulusoy, A., Geiger, A.: Octnet: Learning deep 3d representations at high resolutions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3577–3586 (2017)

  • Rostami R, Bashiri FS, Rostami B, Yu Z (2019) A survey on data-driven 3d shape descriptors. Comput Graphics Forum 38(1):356–393

    Google Scholar 

  • Roynard, X., Deschaud, J.E., Goulette, F.: Paris-lille-3d: A point cloud dataset for urban scene segmentation and classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2027–2030 (2018)

  • Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (fpfh) for 3d registration. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1848–1853 (2009)

  • Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M.: Aligning point cloud views using persistent feature histograms. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3384–3391 (2008)

  • Rusu, R.B., Bradski, G., Thibaux, R., Hsu, J.: Fast 3d recognition and pose using the viewpoint feature histogram. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2155–2162 (2014)

  • Rusu, R.B., Cousins, S.: 3d is here: Point cloud library (pcl). In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1–4 (2011)

  • Rusu, R.B., Holzbach, A., Beetz, M., Bradski, G.: Detecting and segmenting objects for mobile manipulation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops, pp. 47–54 (2009)

  • Rusu, R.B., Marton, Z.C., Blodow, N., Beetz, M.: Persistent point feature histograms for 3d point clouds. In: Proc 10th Int Conf Intel Autonomous Syst (IAS-10), Baden-Baden, Germany, pp. 119–128 (2008)

  • Salti S Petrelli A, T.F.: On the affinity between 3d detectors and descriptors. In: Proceedings of the International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 424–431 (2012)

  • Sánchez J, Perronnin F, Mensink T, Verbeek J (2013) Image classification with the fisher vector: theory and practice. Int J Comput Vision 105(3):222–245

    MathSciNet  MATH  Google Scholar 

  • Shan Y, Sawhney HS, Matei B, Kumar R (2006) Shapeme histogram projection and matching for partial object recognition. IEEE Tran Pattern Anal Mach Intell 28(4):568–577

    Google Scholar 

  • Shelhamer E, Long J, Darrell T (2014) Fully convolutional networks for semantic segmentation. IEEE Tran Pattern Anal Mach Intell 39(4):640–651

    Google Scholar 

  • Shen, Y., Feng, C., Yang, Y., Tian, D.: Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4548–4557 (2018)

  • Simonovsky, M., Komodakis, N.: Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3693–3702 (2017)

  • Singh, A., Sha, J., Narayan, K.S., Achim, T., Abbeel, P.: Bigbird: A large-scale 3d database of object instances. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 509–516. IEEE (2014)

  • Su, H., Jampani, V., Sun, D., Maji, S., Kalogerakis, E., Yang, M.H., Kautz, J.: Splatnet: Sparse lattice networks for point cloud processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2530–2539 (2018)

  • Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 945–953 (2015)

  • Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826 (2016)

  • Tang, K., Song, P., Chen, X.: Signature of geometric centroids for 3d local shape description and partial shape matching. In: Proceedings of the Asian Conference on Computer Vision (ACCV), pp. 311–326 (2016)

  • Tchapmi, L., Choy, C., Armeni, I., Gwak, J., Savarese, S.: Segcloud: Semantic segmentation of 3d point clouds. In: Proceedings of the International Conference on 3D Vision, pp. 537–547. IEEE (2017)

  • Te, G., Hu, W., Zheng, A., Guo, Z.: Rgcnn: Regularized graph cnn for point cloud segmentation. In: Proceedings of the ACM Multimedia Conference on Multimedia, pp. 746–754. ACM (2018)

  • Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: Flexible and deformable convolution for point clouds. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 6411–6420 (2019)

  • Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 356–369. Springer (2010)

  • Tombari, F., Salti, S., Stefano, L.D.: Unique shape context for 3d data description. In: Proceedings of the ACM Workshop on 3D Object Retrieval, pp. 57–62 (2010)

  • Tombari, F., Salti, S., Stefano, L.D.: Unique signatures of histograms for local surface description. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 356–369 (2010)

  • Tombari, F., Salti, S., Stefano, L.D.: A combined texture-shape descriptor for enhanced 3d feature matching. In: Proceedings of the IEEE International Conference on Image Processing (ICIP), vol. 263, pp. 809–812 (2011)

  • Triebel, R., Kersting, K., Burgard, W.: Robust 3d scan point classification using associative markov networks. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2603–2608 (2006)

  • Uy, M.A., Pham, Q.H., Hua, B.S., Nguyen, T., Yeung, S.K.: Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 1588–1597 (2019)

  • Vandapel, N., Huber, D.F., Kapuria, A., Hebert, M.: Natural terrain classification using 3-d ladar data. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), vol. 5, pp. 5117–5122 (2004)

  • Wahl, E., Hillenbrand, U., Hirzinger, G.: Surflet-pair-relation histograms: A statistical 3d-shape representation for rapid classification. In: Proceedings of the International Conference on 3-D Digital Imaging and Modeling, pp. 474–481 (2003)

  • Wang, C., Samari, B., Siddiqi, K.: Local spectral graph convolution for point set feature learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 52–66 (2018)

  • Wang DZ, Posner I (2015) Voting for voting in online point cloud object detection. Robotics: Sci Syst 1:3

    Google Scholar 

  • Wang, L., Huang, Y., Hou, Y., Zhang, S., Shan, J.: Graph attention convolution for point cloud semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10296–10305 (2019)

  • Wang PS, Liu Y, Guo YX, Sun CY, Tong X (2017) O-cnn: Octree-based convolutional neural networks for 3d shape analysis. ACM Trans Graphics (TOG) 36(4):72

    Google Scholar 

  • Wang, S., Suo, S., Ma, W.C., Pokrovsky, A., Urtasun, R.: Deep parametric continuous convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2589–2597 (2018)

  • Wang, W., Yu, R., Huang, Q., Neumann, U.: Sgpn: Similarity group proposal network for 3d point cloud instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2569–2578 (2018)

  • Wang, X., Liu, S., Shen, X., Shen, C., Jia, J.: Associatively segmenting instances and semantics in point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4096–4105 (2019)

  • Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM (2019) Dynamic graph cnn for learning on point clouds. Acm Trans Graphics 38(5):1–12

    Google Scholar 

  • Wang Z, Zhang L, Fang T, Mathiopoulos PT, Tong X, Qu H, Xiao Z, Li F, Chen D (2015) A multiscale and hierarchical feature extraction method for terrestrial laser scanning point cloud classification. IEEE Trans Geosci Remote Sens 53(5):2409–2425

    Google Scholar 

  • Wohlkinger, W., Vincze, M.: Ensemble of shape functions for 3d object classification. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics, pp. 2987–2992 (2012)

  • Wu, K., Li, X., Ranasinghe, R., Dissanayake, G., Liu, Y.: Risas: A novel rotation, illumination, scale invariant appearance and shape feature. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 4008–4015 (2017)

  • Wu, W., Qi, Z., Fuxin, L.: Pointconv: Deep convolutional networks on 3d point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9621–9630 (2019)

  • Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3d shapenets: A deep representation for volumetric shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1912–1920 (2015)

  • Xiang, T., Zhang, C., Song, Y., Yu, J., Cai, W.: Walk in the cloud: Learning curves for point clouds shape analysis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 915–924 (2021)

  • Xie, S., Liu, S., Chen, Z., Tu, Z.: Attentional shapecontextnet for point cloud recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4606–4615 (2018)

  • Xie Y, Tian J, Zhu XX (2020) Linking points with labels in 3d: a review of point cloud semantic segmentation. IEEE Geosci Remote Sens Mag 8(4):38–59

    Google Scholar 

  • Xu, M., Ding, R., Zhao, H., Qi, X.: Paconv: Position adaptive convolution with dynamic kernel assembling on point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3173–3182 (2021)

  • Xu, X., Lee, G.H.: Weakly supervised semantic point cloud segmentation: Towards 10x fewer labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13706–13715 (2020)

  • Xu, Y., Fan, T., Xu, M., Zeng, L., Qiao, Y.: Spidercnn: Deep learning on point sets with parameterized convolutional filters. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 87–102 (2018)

  • Yan, X., Zheng, C., Li, Z., Wang, S., Cui, S.: Pointasnl: Robust point clouds processing using nonlocal neural networks with adaptive sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5589–5598 (2020)

  • Yang J, Cao Z, Zhang Q (2016) A fast and robust local descriptor for 3d point cloud registration. Information Sci 346:163–179

    Google Scholar 

  • Yang, Y., Feng, C., Shen, Y., Tian, D.: Foldingnet: Point cloud auto-encoder via deep grid deformation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 206–215 (2018)

  • Yi L, Kim VG, Ceylan D, Shen I, Yan M, Su H, Lu C, Huang Q, Sheffer A, Guibas L et al (2016) A scalable active framework for region annotation in 3d shape collections. ACM Tran Graphics 35(6):1–12

    Google Scholar 

  • Yi, L., Su, H., Guo, X., Guibas, L.J.: Syncspeccnn: Synchronized spectral cnn for 3d shape segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2282–2290 (2017)

  • Zaidi S, Zela A, Elsken T, Holmes CC, Hutter F, Teh Y (2021) Neural ensemble search for uncertainty estimation and dataset shift. Adv Neural Information Proc Syst 34:7898–7911

    Google Scholar 

  • Zelener, A., Mordohai, P., Stamos, I.: Classification of vehicle parts in unstructured 3d point clouds. In: Proceedings of the International Conference on 3d Vision, pp. 147–154 (2015)

  • Zeng, A., Song, S., Nießner¸ner, M., Fisher, M., Xiao, J., Funkhouser, T.: 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  • Zeng, W., Gevers, T.: 3dcontextnet: Kd tree guided hierarchical learning of point clouds using local and global contextual cues. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 314–330. Springer (2018)

  • Zhang, C., Wan, H., Shen, X., Wu, Z.: Patchformer: An efficient point transformer with patch attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11799–11808 (2022)

  • Zhang, D., Lu, X., Qin, H., He, Y.: Pointfilter: Point cloud filtering via encoder-decoder modeling. arXiv preprint arXiv:2002.05968 (2020)

  • Zhang J, Zhao X, Chen Z, Lu Z (2019) A review of deep learning-based semantic segmentation for point cloud. IEEE Access 7:179118–179133

    Google Scholar 

  • Zhang, R., Guo, Z., Zhang, W., Li, K., Miao, X., Cui, B., Qiao, Y., Gao, P., Li, H.: Pointclip: Point cloud understanding by clip. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8552–8562 (2022)

  • Zhao, G., Yuan, J., Dang, K.: Height gradient histogram (high) for 3d scene labeling. In: Proceedings of the International Conference on 3d Vision, pp. 569–576 (2014)

  • Zhao, H., Jiang, L., Fu, C.W., Jia, J.: Pointweb: Enhancing local neighborhood features for point cloud processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5565–5573 (2019)

  • Zhao, H., Jiang, L., Jia, J., Torr, P.H., Koltun, V.: Point transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16259–16268 (2021)

  • Zhong, Y.: Intrinsic shape signatures: A shape descriptor for 3d object recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 689–696 (2010)

  • Zhou, L., Zhu, S., Luo, Z., Shen, T., Zhang, R., Zhen, M., Fang, T., Quan, L.: Learning and matching multi-view descriptors for registration of point clouds. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 505–522 (2018)

  • Zhou, Y., Tuzel, O.: Voxelnet: End-to-end learning for point cloud based 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4490–4499 (2018)

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (No. 62002299), and the Natural Science Foundation of Chongqing of China (No. cstc2020jcyj-msxmX0126), and the Fundamental Research Funds for the Central Universities (No. SWU120005).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xian-Feng Han.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, XF., Feng, ZA., Sun, SJ. et al. 3D point cloud descriptors: state-of-the-art. Artif Intell Rev 56, 12033–12083 (2023). https://doi.org/10.1007/s10462-023-10486-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-023-10486-4

Keywords

Navigation