Skip to main content
SpringerLink
Log in

A Comprehensive Performance Evaluation of 3D Local Feature Descriptors

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

A number of 3D local feature descriptors have been proposed in the literature. It is however, unclear which descriptors are more appropriate for a particular application. A good descriptor should be descriptive, compact, and robust to a set of nuisances. This paper compares ten popular local feature descriptors in the contexts of 3D object recognition, 3D shape retrieval, and 3D modeling. We first evaluate the descriptiveness of these descriptors on eight popular datasets which were acquired using different techniques. We then analyze their compactness using the recall of feature matching per each float value in the descriptor. We also test the robustness of the selected descriptors with respect to support radius variations, Gaussian noise, shot noise, varying mesh resolution, distance to the mesh boundary, keypoint localization error, occlusion, clutter, and dataset size. Moreover, we present the performance results of these descriptors when combined with different 3D keypoint detection methods. We finally analyze the computational efficiency for generating each descriptor.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Aldoma, A., Marton, Z., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., et al. (2012a). Tutorial: Point cloud library: Three-dimensional object recognition and 6 DOF pose estimation. IEEE Robotics & Automation Magazine, 19(3), 80–91.

    Article  Google Scholar 

  • Aldoma, A., Tombari, F., Di Stefano, L., & Vincze, M. (2012b). A global hypotheses verification method for 3D object recognition. In European Conference on Computer Vision, (pp 511–524).

  • Alexandre, L.A. (2012). 3D descriptors for object and category recognition: A comparative evaluation. In Workshop on Color-Depth Camera Fusion in Robotics at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

  • Assfalg, J., Bertini, M., Bimbo, A., & Pala, P. (2007). Content-based retrieval of 3-D objects using spin image signatures. IEEE Transactions on Multimedia, 9(3), 589–599.

    Article  Google Scholar 

  • Bariya, P., Novatnack, J., Schwartz, G., & Nishino, K. (2012). 3D geometric scale variability in range images: Features and descriptors. International Journal of Computer Vision, 99(2), 232–255.

    Article  MathSciNet  Google Scholar 

  • Bayramoglu, N., & Alatan, A. (2010). Shape index SIFT: Range image recognition using local features. In 20th International Conference on Pattern Recognition, (pp. 352–355).

  • Bennamoun, M., Guo, Y., & Sohel, F. (2015). Feature selection for 2D and 3D face recognition, In Encyclopedia of electrical and electronics engineering. Book Chapter (pp. 1–54).

  • Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

    Article  Google Scholar 

  • Boyer, E., Bronstein, A., & Bronstein, M., et al. (2011). SHREC 2011: Robust feature detection and description benchmark. In Eurographics Workshop on Shape Retrieval, (pp. 79–86).

  • Bronstein, A., Bronstein, M., & Bustos, B., et al. (2010). SHREC 2010: Robust feature detection and description benchmark. In Eurographics Workshop on 3D Object Retrieval, vol 2, p 6.

  • Bronstein, A., Bronstein, M., Guibas, L., & Ovsjanikov, M. (2011). Shape google: Geometric words and expressions for invariant shape retrieval. ACM Transactions on Graphics, 30(1), 1–20.

    Article  Google Scholar 

  • Burghouts, G. J., & Geusebroek, J. M. (2009). Performance evaluation of local colour invariants. Computer Vision and Image Understanding, 113(1), 48–62.

    Article  Google Scholar 

  • Chen, H., & Bhanu, B. (2007a). 3D free-form object recognition in range images using local surface patches. Pattern Recognition Letters, 28(10), 1252–1262.

    Article  Google Scholar 

  • Chen, H., & Bhanu, B. (2007b). Human ear recognition in 3D. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4), 718–737.

    Article  Google Scholar 

  • Chen, X., & Schmitt, F. (1992). Intrinsic surface properties from surface triangulation. In European Conference on Computer Vision, (pp. 739–743).

  • Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In 23rd Annual Conference on Computer Graphics and Interactive Techniques, (pp. 303–312).

  • Darom, T., & Keller, Y. (2012). Scale invariant features for 3D mesh models. IEEE Transactions on Image Processing, 21(5), 2758–2769.

    Article  MathSciNet  Google Scholar 

  • Davis, J., & Goadrich, M. (2006). The relationship between precision-recall and roc curves. In 23rd International Conference on Machine learning, (pp. 233–240).

  • Dinh, H., & Kropac, S. (2006). Multi-resolution spin-images. IEEE International Conference on Computer Vision and Pattern Recognition, 1, 863–870.

    Google Scholar 

  • Filipe, S., & Alexandre, L.A. (2014). A comparative evaluation of 3D keypoint detectors in a RGB-D object dataset. In 9th International Conference on Computer Vision Theory and Applications, (pp. 1–8).

  • Flint, A., Dick, A., & Hengel, A. (2007). THRIFT: Local 3D structure recognition. In 9th Conference on Digital Image Computing Techniques and Applications, (pp. 182–188).

  • Flint, A., Dick, A., & Van den Hengel, A. (2008). Local 3D structure recognition in range images. IET Computer Vision, 2(4), 208–217.

    Article  Google Scholar 

  • Frome, A., Huber, D., Kolluri, R., Bülow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. In 8th European Conference on Computer Vision, (pp. 224–237).

  • Gao, Y., & Dai, Q. (2014). View-based 3-D object retrieval: Challenges and approaches. IEEE Multimedia, 21(3), 52–57.

    Article  Google Scholar 

  • Guo, Y., Bennamoun, M., Sohel, F., Wan, J., & Lu, M. (2013a). 3D free form object recognition using rotational projection statistics. In IEEE 14th Workshop on the Applications of Computer Vision, (pp. 1–8).

  • Guo, Y., Sohel, F., Bennamoun, M., Lu, M., & Wan, J. (2013b). Rotational projection statistics for 3D local surface description and object recognition. International Journal of Computer Vision, 105(1), 63–86.

  • Guo, Y., Sohel, F., Bennamoun, M., Lu, M., Wan, J. (2013c). TriSI: A distinctive local surface descriptor for 3D modeling and object recognition. In 8th International Conference on Computer Graphics Theory and Applications, (pp. 86–93).

  • Guo, Y., Bennamoun, M., Sohel, F., Lu, M., & Wan, J. (2014a). 3D object recognition in cluttered scenes with local surface features: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(11), 2270–2287.

    Article  Google Scholar 

  • Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J., & Zhang, J. (2014b). Performance evaluation of 3D local feature descriptors. In 12th Asian Conference on Computer Vision, (pp. 1–17).

  • Guo, Y., Sohel, F., Bennamoun, M., Wan, J., & Lu, M. (2014c). An accurate and robust range image registration algorithm for 3D object modeling. IEEE Transactions on Multimedia, 16(5), 1377–1390.

  • Guo, Y., Zhang, J., Lu, M., Wan, J., & Ma, Y. (2014d). Benchmark datasets for 3D computer vision. In The 9th IEEE Conference on Industrial Electronics and Applications.

  • Guo, Y., Sohel, F., Bennamoun, M., Wan, J., & Lu, M. (2015). A novel local surface feature for 3D object recognition under clutter and occlusion. Information Sciences, 293(2), 196–213.

    Article  Google Scholar 

  • Johnson, A. E., & Hebert, M. (1998). Surface matching for object recognition in complex three-dimensional scenes. Image and Vision Computing, 16(9–10), 635–651.

    Article  Google Scholar 

  • Johnson, A. E., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449.

    Article  Google Scholar 

  • Ke, Y., & Sukthankar, R. (2004). PCA-SIFT: A more distinctive representation for local image descriptors. IEEE Conference on Computer Vision and Pattern Recognition, 2, 498–506.

    Google Scholar 

  • Kim, H., & Hilton, A. (2013). Evaluation of 3D feature descriptors for multi-modal data registration. In International Conference on 3D Vision, (pp. 119–126).

  • Koenderink, J., & van Doorn, A. (1992). Surface shape and curvature scales. Image and Vision Computing, 10(8), 557–564.

    Article  Google Scholar 

  • Lai, K., Bo, L., Ren, X., & Fox, D. (2011). A scalable tree-based approach for joint object and pose recognition. In 25th Conference on Artificial Intelligence.

  • Lei, Y., Bennamoun, M., Hayat, M., & Guo, Y. (2014). An efficient 3D face recognition approach using local geometrical signatures. Pattern Recognition, 47(2), 509–524.

    Article  Google Scholar 

  • Lo, T., & Siebert, J. (2009). Local feature extraction and matching on range images: 2.5D SIFT. Computer Vision and Image Understanding, 113(12), 1235–1250.

    Article  Google Scholar 

  • Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Matei, B., Shan, Y., Sawhney, H., Tan, Y., Kumar, R., Huber, D., et al. (2006). Rapid object indexing using locality sensitive hashing and joint 3D-signature space estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7), 1111–1126.

    Article  Google Scholar 

  • Meek, D. S., & Walton, D. J. (2000). On surface normal and gaussian curvature approximations given data sampled from a smooth surface. Computer Aided Geometric Design, 17(6), 521–543.

    Article  MathSciNet  MATH  Google Scholar 

  • Mian, A., Bennamoun, M., & Owens, R. (2006a). Three-dimensional model-based object recognition and segmentation in cluttered scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1584–1601.

    Article  Google Scholar 

  • Mian, A., Bennamoun, M., & Owens, R. A. (2006b). A novel representation and feature matching algorithm for automatic pairwise registration of range images. International Journal of Computer Vision, 66(1), 19–40.

    Article  Google Scholar 

  • Mian, A., Bennamoun, M., & Owens, R. (2010). On the repeatability and quality of keypoints for local feature-based 3D object retrieval from cluttered scenes. International Journal of Computer Vision, 89(2), 348–361.

    Article  Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2003). A performance evaluation of local descriptors. In IEEE Conference on Computer Vision and Pattern Recognition, vol 2, (pp. II-257).

  • Mikolajczyk, K., & Schmid, C. (2004). Scale & affine invariant interest point detectors. International Journal of Computer Vision, 60(1), 63–86.

    Article  Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615–1630.

    Article  Google Scholar 

  • Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., et al. (2005). A comparison of affine region detectors. International Journal of Computer Vision, 65(1), 43–72.

    Article  Google Scholar 

  • Moreels, P., & Perona, P. (2005). Evaluation of features detectors and descriptors based on 3D objects. In 10th IEEE International Conference on Computer Vision, vol 1, (pp. 800–807).

  • Moreels, P., & Perona, P. (2007). Evaluation of features detectors and descriptors based on 3D objects. International Journal of Computer Vision, 73(3), 263–284.

    Article  Google Scholar 

  • Restrepo, M.I., & Mundy, J.L. (2012). An evaluation of local shape descriptors in probabilistic volumetric scenes. In British Machine Vision Conference, (pp. 1–11).

  • Rodolà, E., Albarelli, A., Bergamasco, F., & Torsello, A. (2013). A scale independent selection process for 3D object recognition in cluttered scenes. In International Journal of Computer Vision pp 1–17.

  • Ruiz-Correa, S., Shapiro, L., & Melia, M. (2001). A new signature-based method for efficient 3-D object recognition. In IEEE Conference on Computer Vision and Pattern Recognition, vol 1, (pp. I-769).

  • Rusu, R.B., & Cousins, S. (2011). 3D is here: Point cloud library (PCL). In IEEE International Conference on Robotics and Automation, pp 1–4.

  • Rusu, R.B., Blodow, N., Marton, Z.C., & Beetz, M. (2008). Aligning point cloud views using persistent feature histograms. In IEEE/RSJ International Conference on Intelligent Robots and Systems, (pp. 3384–3391).

  • Rusu, R.B., Blodow, N., & Beetz, M. (2009). Fast point feature histograms (FPFH) for 3D registration. In IEEE International Conference on Robotics and Automation, (pp. 3212–3217).

  • Salti, S., Tombari, F., & Stefano, L. (2011). A performance evaluation of 3D keypoint detectors. In International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (pp. 236–243).

  • Salti, S., Petrelli, A., Tombari, F., & Di Stefano, L. (2012). On the affinity between 3D detectors and descriptors. In 2nd International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), (pp. 424–431).

  • Salti, S., Tombari, F., & Stefano, L. D. (2014). SHOT: Unique signatures of histograms for surface and texture description. Computer Vision and Image Understanding, 125(8), 251–264.

    Article  Google Scholar 

  • Schmid, C., Mohr, R., & Bauckhage, C. (2000). Evaluation of interest point detectors. International Journal of Computer Vision, 37(2), 151–172.

    Article  MATH  Google Scholar 

  • Shang, L., & Greenspan, M. (2010). Real-time object recognition in sparse range images using error surface embedding. International Journal of Computer Vision, 89(2), 211–228.

    Article  Google Scholar 

  • Sipiran, I., & Bustos, B. (2011). Harris 3D: a robust extension of the harris operator for interest point detection on 3D meshes. The Visual Computer pp. 1–14.

  • Sukno, F.M., Waddington, J.L., & Whelan, P.F. (2013). Rotationally invariant 3D shape contexts using asymmetry patterns. In 8th International Conference on Computer Graphics Theory and Applications.

  • Sun, J., Ovsjanikov, M., & Guibas, L. (2009). A concise and provably informative multi-scale signature based on heat diffusion. Computer Graphics Forum, 28, 1383–1392.

  • Taati, B., & Greenspan, M. (2011). Local shape descriptor selection for object recognition in range data. Computer Vision and Image Understanding, 115(5), 681–694.

  • Tangelder, J., Veltkamp, R. (2004). A survey of content based 3D shape retrieval methods. In IEEE International Conference on Shape Modeling and Applications, (pp. 145–156).

  • Tombari, F., Salti, S., & Di Stefano, L. (2010a), Unique shape context for 3D data description. In ACM Workshop on 3D Object Retrieval, (pp. 57–62).

  • Tombari, F., Salti, S., & Di Stefano, L. (2010b). Unique signatures of histograms for local surface description. In European Conference on Computer Vision, Springer, New York, (pp. 356–369).

  • Tombari, F., Salti, S., & Di Stefano, L. (2013). Performance evaluation of 3D keypoint detectors. International Journal of Computer Vision, 102(1), 198–220.

    Article  Google Scholar 

  • Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In IEEE Conference on Computer Vision and Pattern Recognition, (pp. 373–380).

  • Zaharescu, A., Boyer, E., & Horaud, R. (2012). Keypoints and local descriptors of scalar functions on 2D manifolds. International Journal of Computer Vision, 100, 78–98.

    Article  MATH  Google Scholar 

  • Zhong, Y. (2009). Intrinsic shape signatures: A shape descriptor for 3D object recognition. In IEEE International Conference on Computer Vision Workshops, (pp. 689–696).

Download references

Acknowledgments

This research is supported by a National Natural Science Foundation of China (NSFC) fund (No. 61471371), a China Scholarship Council (CSC) scholarship and Australian Research Council Grants (DE120102960, DP110102166, DP150100294).

Author information

Authors and Affiliations

  1. College of Electronic Science and Engineering, National University of Defense Technology, Changsha, 410073, Hunan, People’s Republic of China

    Yulan Guo, Min Lu & Jianwei Wan

  2. School of Computer Science and Software Engineering, The University of Western Australia, Perth, WA, 6009, Australia

    Yulan Guo, Mohammed Bennamoun & Ferdous Sohel

  3. School of Mechanical and Manufacturing Engineering, The University of New South Wales, Sydney, NSW, 2052, Australia

    Ngai Ming Kwok

Authors
  1. Yulan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed Bennamoun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ferdous Sohel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianwei Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ngai Ming Kwok
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yulan Guo.

Additional information

Communicated by M. Hebert.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, Y., Bennamoun, M., Sohel, F. et al. A Comprehensive Performance Evaluation of 3D Local Feature Descriptors. Int J Comput Vis 116, 66–89 (2016). https://doi.org/10.1007/s11263-015-0824-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-015-0824-y

Keywords

Search

Navigation