Advertisement

International Journal of Computer Vision

, Volume 116, Issue 1, pp 66–89 | Cite as

A Comprehensive Performance Evaluation of 3D Local Feature Descriptors

  • Yulan Guo
  • Mohammed Bennamoun
  • Ferdous Sohel
  • Min Lu
  • Jianwei Wan
  • Ngai Ming Kwok
Article

Abstract

A number of 3D local feature descriptors have been proposed in the literature. It is however, unclear which descriptors are more appropriate for a particular application. A good descriptor should be descriptive, compact, and robust to a set of nuisances. This paper compares ten popular local feature descriptors in the contexts of 3D object recognition, 3D shape retrieval, and 3D modeling. We first evaluate the descriptiveness of these descriptors on eight popular datasets which were acquired using different techniques. We then analyze their compactness using the recall of feature matching per each float value in the descriptor. We also test the robustness of the selected descriptors with respect to support radius variations, Gaussian noise, shot noise, varying mesh resolution, distance to the mesh boundary, keypoint localization error, occlusion, clutter, and dataset size. Moreover, we present the performance results of these descriptors when combined with different 3D keypoint detection methods. We finally analyze the computational efficiency for generating each descriptor.

Keywords

Performance evaluation Local feature descriptor Keypoint detector 3D object recognition  3D shape retrieval  3D modeling 3D surface 

Notes

Acknowledgments

This research is supported by a National Natural Science Foundation of China (NSFC) fund (No. 61471371), a China Scholarship Council (CSC) scholarship and Australian Research Council Grants (DE120102960, DP110102166, DP150100294).

References

  1. Aldoma, A., Marton, Z., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., et al. (2012a). Tutorial: Point cloud library: Three-dimensional object recognition and 6 DOF pose estimation. IEEE Robotics & Automation Magazine, 19(3), 80–91.CrossRefGoogle Scholar
  2. Aldoma, A., Tombari, F., Di Stefano, L., & Vincze, M. (2012b). A global hypotheses verification method for 3D object recognition. In European Conference on Computer Vision, (pp 511–524).Google Scholar
  3. Alexandre, L.A. (2012). 3D descriptors for object and category recognition: A comparative evaluation. In Workshop on Color-Depth Camera Fusion in Robotics at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Google Scholar
  4. Assfalg, J., Bertini, M., Bimbo, A., & Pala, P. (2007). Content-based retrieval of 3-D objects using spin image signatures. IEEE Transactions on Multimedia, 9(3), 589–599.CrossRefGoogle Scholar
  5. Bariya, P., Novatnack, J., Schwartz, G., & Nishino, K. (2012). 3D geometric scale variability in range images: Features and descriptors. International Journal of Computer Vision, 99(2), 232–255.MathSciNetCrossRefGoogle Scholar
  6. Bayramoglu, N., & Alatan, A. (2010). Shape index SIFT: Range image recognition using local features. In 20th International Conference on Pattern Recognition, (pp. 352–355).Google Scholar
  7. Bennamoun, M., Guo, Y., & Sohel, F. (2015). Feature selection for 2D and 3D face recognition, In Encyclopedia of electrical and electronics engineering. Book Chapter (pp. 1–54).Google Scholar
  8. Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.CrossRefGoogle Scholar
  9. Boyer, E., Bronstein, A., & Bronstein, M., et al. (2011). SHREC 2011: Robust feature detection and description benchmark. In Eurographics Workshop on Shape Retrieval, (pp. 79–86).Google Scholar
  10. Bronstein, A., Bronstein, M., & Bustos, B., et al. (2010). SHREC 2010: Robust feature detection and description benchmark. In Eurographics Workshop on 3D Object Retrieval, vol 2, p 6.Google Scholar
  11. Bronstein, A., Bronstein, M., Guibas, L., & Ovsjanikov, M. (2011). Shape google: Geometric words and expressions for invariant shape retrieval. ACM Transactions on Graphics, 30(1), 1–20.CrossRefGoogle Scholar
  12. Burghouts, G. J., & Geusebroek, J. M. (2009). Performance evaluation of local colour invariants. Computer Vision and Image Understanding, 113(1), 48–62.CrossRefGoogle Scholar
  13. Chen, H., & Bhanu, B. (2007a). 3D free-form object recognition in range images using local surface patches. Pattern Recognition Letters, 28(10), 1252–1262.CrossRefGoogle Scholar
  14. Chen, H., & Bhanu, B. (2007b). Human ear recognition in 3D. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4), 718–737.CrossRefGoogle Scholar
  15. Chen, X., & Schmitt, F. (1992). Intrinsic surface properties from surface triangulation. In European Conference on Computer Vision, (pp. 739–743).Google Scholar
  16. Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In 23rd Annual Conference on Computer Graphics and Interactive Techniques, (pp. 303–312).Google Scholar
  17. Darom, T., & Keller, Y. (2012). Scale invariant features for 3D mesh models. IEEE Transactions on Image Processing, 21(5), 2758–2769.MathSciNetCrossRefGoogle Scholar
  18. Davis, J., & Goadrich, M. (2006). The relationship between precision-recall and roc curves. In 23rd International Conference on Machine learning, (pp. 233–240).Google Scholar
  19. Dinh, H., & Kropac, S. (2006). Multi-resolution spin-images. IEEE International Conference on Computer Vision and Pattern Recognition, 1, 863–870.Google Scholar
  20. Filipe, S., & Alexandre, L.A. (2014). A comparative evaluation of 3D keypoint detectors in a RGB-D object dataset. In 9th International Conference on Computer Vision Theory and Applications, (pp. 1–8).Google Scholar
  21. Flint, A., Dick, A., & Hengel, A. (2007). THRIFT: Local 3D structure recognition. In 9th Conference on Digital Image Computing Techniques and Applications, (pp. 182–188).Google Scholar
  22. Flint, A., Dick, A., & Van den Hengel, A. (2008). Local 3D structure recognition in range images. IET Computer Vision, 2(4), 208–217.CrossRefGoogle Scholar
  23. Frome, A., Huber, D., Kolluri, R., Bülow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. In 8th European Conference on Computer Vision, (pp. 224–237).Google Scholar
  24. Gao, Y., & Dai, Q. (2014). View-based 3-D object retrieval: Challenges and approaches. IEEE Multimedia, 21(3), 52–57.CrossRefGoogle Scholar
  25. Guo, Y., Bennamoun, M., Sohel, F., Wan, J., & Lu, M. (2013a). 3D free form object recognition using rotational projection statistics. In IEEE 14th Workshop on the Applications of Computer Vision, (pp. 1–8).Google Scholar
  26. Guo, Y., Sohel, F., Bennamoun, M., Lu, M., & Wan, J. (2013b). Rotational projection statistics for 3D local surface description and object recognition. International Journal of Computer Vision, 105(1), 63–86.Google Scholar
  27. Guo, Y., Sohel, F., Bennamoun, M., Lu, M., Wan, J. (2013c). TriSI: A distinctive local surface descriptor for 3D modeling and object recognition. In 8th International Conference on Computer Graphics Theory and Applications, (pp. 86–93).Google Scholar
  28. Guo, Y., Bennamoun, M., Sohel, F., Lu, M., & Wan, J. (2014a). 3D object recognition in cluttered scenes with local surface features: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(11), 2270–2287.CrossRefGoogle Scholar
  29. Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J., & Zhang, J. (2014b). Performance evaluation of 3D local feature descriptors. In 12th Asian Conference on Computer Vision, (pp. 1–17).Google Scholar
  30. Guo, Y., Sohel, F., Bennamoun, M., Wan, J., & Lu, M. (2014c). An accurate and robust range image registration algorithm for 3D object modeling. IEEE Transactions on Multimedia, 16(5), 1377–1390.Google Scholar
  31. Guo, Y., Zhang, J., Lu, M., Wan, J., & Ma, Y. (2014d). Benchmark datasets for 3D computer vision. In The 9th IEEE Conference on Industrial Electronics and Applications.Google Scholar
  32. Guo, Y., Sohel, F., Bennamoun, M., Wan, J., & Lu, M. (2015). A novel local surface feature for 3D object recognition under clutter and occlusion. Information Sciences, 293(2), 196–213.CrossRefGoogle Scholar
  33. Johnson, A. E., & Hebert, M. (1998). Surface matching for object recognition in complex three-dimensional scenes. Image and Vision Computing, 16(9–10), 635–651.CrossRefGoogle Scholar
  34. Johnson, A. E., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449.CrossRefGoogle Scholar
  35. Ke, Y., & Sukthankar, R. (2004). PCA-SIFT: A more distinctive representation for local image descriptors. IEEE Conference on Computer Vision and Pattern Recognition, 2, 498–506.Google Scholar
  36. Kim, H., & Hilton, A. (2013). Evaluation of 3D feature descriptors for multi-modal data registration. In International Conference on 3D Vision, (pp. 119–126).Google Scholar
  37. Koenderink, J., & van Doorn, A. (1992). Surface shape and curvature scales. Image and Vision Computing, 10(8), 557–564.CrossRefGoogle Scholar
  38. Lai, K., Bo, L., Ren, X., & Fox, D. (2011). A scalable tree-based approach for joint object and pose recognition. In 25th Conference on Artificial Intelligence.Google Scholar
  39. Lei, Y., Bennamoun, M., Hayat, M., & Guo, Y. (2014). An efficient 3D face recognition approach using local geometrical signatures. Pattern Recognition, 47(2), 509–524.CrossRefGoogle Scholar
  40. Lo, T., & Siebert, J. (2009). Local feature extraction and matching on range images: 2.5D SIFT. Computer Vision and Image Understanding, 113(12), 1235–1250.CrossRefGoogle Scholar
  41. Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.CrossRefGoogle Scholar
  42. Matei, B., Shan, Y., Sawhney, H., Tan, Y., Kumar, R., Huber, D., et al. (2006). Rapid object indexing using locality sensitive hashing and joint 3D-signature space estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7), 1111–1126.CrossRefGoogle Scholar
  43. Meek, D. S., & Walton, D. J. (2000). On surface normal and gaussian curvature approximations given data sampled from a smooth surface. Computer Aided Geometric Design, 17(6), 521–543.MathSciNetCrossRefMATHGoogle Scholar
  44. Mian, A., Bennamoun, M., & Owens, R. (2006a). Three-dimensional model-based object recognition and segmentation in cluttered scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1584–1601.CrossRefGoogle Scholar
  45. Mian, A., Bennamoun, M., & Owens, R. A. (2006b). A novel representation and feature matching algorithm for automatic pairwise registration of range images. International Journal of Computer Vision, 66(1), 19–40.CrossRefGoogle Scholar
  46. Mian, A., Bennamoun, M., & Owens, R. (2010). On the repeatability and quality of keypoints for local feature-based 3D object retrieval from cluttered scenes. International Journal of Computer Vision, 89(2), 348–361.CrossRefGoogle Scholar
  47. Mikolajczyk, K., & Schmid, C. (2003). A performance evaluation of local descriptors. In IEEE Conference on Computer Vision and Pattern Recognition, vol 2, (pp. II-257).Google Scholar
  48. Mikolajczyk, K., & Schmid, C. (2004). Scale & affine invariant interest point detectors. International Journal of Computer Vision, 60(1), 63–86.CrossRefGoogle Scholar
  49. Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615–1630.CrossRefGoogle Scholar
  50. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., et al. (2005). A comparison of affine region detectors. International Journal of Computer Vision, 65(1), 43–72.CrossRefGoogle Scholar
  51. Moreels, P., & Perona, P. (2005). Evaluation of features detectors and descriptors based on 3D objects. In 10th IEEE International Conference on Computer Vision, vol 1, (pp. 800–807).Google Scholar
  52. Moreels, P., & Perona, P. (2007). Evaluation of features detectors and descriptors based on 3D objects. International Journal of Computer Vision, 73(3), 263–284.CrossRefGoogle Scholar
  53. Restrepo, M.I., & Mundy, J.L. (2012). An evaluation of local shape descriptors in probabilistic volumetric scenes. In British Machine Vision Conference, (pp. 1–11).Google Scholar
  54. Rodolà, E., Albarelli, A., Bergamasco, F., & Torsello, A. (2013). A scale independent selection process for 3D object recognition in cluttered scenes. In International Journal of Computer Vision pp 1–17.Google Scholar
  55. Ruiz-Correa, S., Shapiro, L., & Melia, M. (2001). A new signature-based method for efficient 3-D object recognition. In IEEE Conference on Computer Vision and Pattern Recognition, vol 1, (pp. I-769).Google Scholar
  56. Rusu, R.B., & Cousins, S. (2011). 3D is here: Point cloud library (PCL). In IEEE International Conference on Robotics and Automation, pp 1–4.Google Scholar
  57. Rusu, R.B., Blodow, N., Marton, Z.C., & Beetz, M. (2008). Aligning point cloud views using persistent feature histograms. In IEEE/RSJ International Conference on Intelligent Robots and Systems, (pp. 3384–3391).Google Scholar
  58. Rusu, R.B., Blodow, N., & Beetz, M. (2009). Fast point feature histograms (FPFH) for 3D registration. In IEEE International Conference on Robotics and Automation, (pp. 3212–3217).Google Scholar
  59. Salti, S., Tombari, F., & Stefano, L. (2011). A performance evaluation of 3D keypoint detectors. In International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (pp. 236–243).Google Scholar
  60. Salti, S., Petrelli, A., Tombari, F., & Di Stefano, L. (2012). On the affinity between 3D detectors and descriptors. In 2nd International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), (pp. 424–431).Google Scholar
  61. Salti, S., Tombari, F., & Stefano, L. D. (2014). SHOT: Unique signatures of histograms for surface and texture description. Computer Vision and Image Understanding, 125(8), 251–264.CrossRefGoogle Scholar
  62. Schmid, C., Mohr, R., & Bauckhage, C. (2000). Evaluation of interest point detectors. International Journal of Computer Vision, 37(2), 151–172.CrossRefMATHGoogle Scholar
  63. Shang, L., & Greenspan, M. (2010). Real-time object recognition in sparse range images using error surface embedding. International Journal of Computer Vision, 89(2), 211–228.CrossRefGoogle Scholar
  64. Sipiran, I., & Bustos, B. (2011). Harris 3D: a robust extension of the harris operator for interest point detection on 3D meshes. The Visual Computer pp. 1–14.Google Scholar
  65. Sukno, F.M., Waddington, J.L., & Whelan, P.F. (2013). Rotationally invariant 3D shape contexts using asymmetry patterns. In 8th International Conference on Computer Graphics Theory and Applications.Google Scholar
  66. Sun, J., Ovsjanikov, M., & Guibas, L. (2009). A concise and provably informative multi-scale signature based on heat diffusion. Computer Graphics Forum, 28, 1383–1392.Google Scholar
  67. Taati, B., & Greenspan, M. (2011). Local shape descriptor selection for object recognition in range data. Computer Vision and Image Understanding, 115(5), 681–694.Google Scholar
  68. Tangelder, J., Veltkamp, R. (2004). A survey of content based 3D shape retrieval methods. In IEEE International Conference on Shape Modeling and Applications, (pp. 145–156).Google Scholar
  69. Tombari, F., Salti, S., & Di Stefano, L. (2010a), Unique shape context for 3D data description. In ACM Workshop on 3D Object Retrieval, (pp. 57–62).Google Scholar
  70. Tombari, F., Salti, S., & Di Stefano, L. (2010b). Unique signatures of histograms for local surface description. In European Conference on Computer Vision, Springer, New York, (pp. 356–369).Google Scholar
  71. Tombari, F., Salti, S., & Di Stefano, L. (2013). Performance evaluation of 3D keypoint detectors. International Journal of Computer Vision, 102(1), 198–220.CrossRefGoogle Scholar
  72. Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In IEEE Conference on Computer Vision and Pattern Recognition, (pp. 373–380).Google Scholar
  73. Zaharescu, A., Boyer, E., & Horaud, R. (2012). Keypoints and local descriptors of scalar functions on 2D manifolds. International Journal of Computer Vision, 100, 78–98.CrossRefMATHGoogle Scholar
  74. Zhong, Y. (2009). Intrinsic shape signatures: A shape descriptor for 3D object recognition. In IEEE International Conference on Computer Vision Workshops, (pp. 689–696).Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Yulan Guo
    • 1
    • 2
  • Mohammed Bennamoun
    • 2
  • Ferdous Sohel
    • 2
  • Min Lu
    • 1
  • Jianwei Wan
    • 1
  • Ngai Ming Kwok
    • 3
  1. 1.College of Electronic Science and EngineeringNational University of Defense TechnologyChangshaPeople’s Republic of China
  2. 2.School of Computer Science and Software EngineeringThe University of Western AustraliaPerthAustralia
  3. 3.School of Mechanical and Manufacturing EngineeringThe University of New South WalesSydneyAustralia

Personalised recommendations