Multimedia Systems

, Volume 20, Issue 3, pp 253–281 | Cite as

Spatially aggregating spectral descriptors for nonrigid 3D shape retrieval: a comparative survey

Regular Paper

Abstract

This paper presents a comprehensive review and analysis of recent spectral shape descriptors for nonrigid 3D shape retrieval. More specifically, we compare the latest spectral descriptors based on the Laplace–Beltrami (LB) operator, including ShapeDNA, heat kernel signature, scale invariant heat kernel signature, heat mean signature, wave kernel signature, and global point signature. We also include the eigenvalue descriptor (EVD), which is a geodesic distance-based shape signature. The global descriptors ShapeDNA and EVD are compared via the chi-squared distance, while all local descriptors are compared using the codebook model. Moreover, we investigate the ambiguity modeling of codebook for the densely distributed low-level shape descriptors. Inspired by the ability of spatial cues to improve discrimination between shapes, we also propose to adopt the isocontours of the second eigenfunction of the LB operator to perform surface partition, which can significantly ameliorate the retrieval performance of the time-scaled local descriptors. In addition, we introduce an intrinsic spatial pyramid matching approach in a bid to further enhance the retrieval accuracy. Extensive experiments are carried out on two 3D shape benchmarks to assess the performance of the spectral descriptors. Our proposed approach is shown to provide the best performance.

Keywords

Shape retrieval Spectral geometry Intrinsic partition Aggregate local descriptors 

References

  1. 1.
    Yang, Y., Lin, H., Zhang, Y.: Content-based 3-D model retrieval: a survey. IEEE Trans. Syst. Man Cybern. Part C 37(6), 1081–1098 (2007)CrossRefGoogle Scholar
  2. 2.
    Del Bimbo, A., Pala, P.: Content-based retrieval of 3D models. ACM Trans. Multimedia Comput. Commun. Appl. 2(1), 20–43 (2006)CrossRefGoogle Scholar
  3. 3.
    Tangelder, J.W.H., Veltkamp, R.C.: A survey of content based 3D shape retrieval methods. Multimedia Tools Appl. 39(3), 441–471 (2008)CrossRefGoogle Scholar
  4. 4.
    Bustos, B., Keim, D.A., Saupe, D., Schreck, T., Vranic, D.V.: Feature-based similarity search in 3D object databases. ACM Comput. Surv. 37(4), 345–387 (2005)CrossRefGoogle Scholar
  5. 5.
    Jain, V., Zhang, H.: A spectral approach to shape-based retrieval of articulated 3D models. Comput. Aided Design 39(5), 398–407 (2007)CrossRefGoogle Scholar
  6. 6.
    Macrini, D., Siddiqi, K., Dickinson, S.J.: From skeletons to bone graphs: medial abstraction for object recognition. In: Proc. CVPR, pp. 1–8 (2008)Google Scholar
  7. 7.
    Siddiqi, K., Zhang, J., Macrini, D., Shokoufandeh, A., Bouix, S., Dickinson, S.J.: Retrieving articulated 3-D models using medial surfaces. Mach. Vis. Appl. 19(4), 261−275 (2008)CrossRefGoogle Scholar
  8. 8.
    Siddiqi, K., Pizer, S. (eds.): Medial representations: mathematics, algorithms and applications. Springer, Berlin (2008)Google Scholar
  9. 9.
    Li, C., Ben Hamza, A.: Skeleton path based approach for nonrigid 3D shape analysis and retrieval. In: Proc. IWCIA, LNCS, pp. 84–95 (2011)Google Scholar
  10. 10.
    Mohamed, W., Ben Hamza, A.: Reeb graph path dissimilarity for 3D object matching and retrieval. V. Comput. 28(3), 305–318 (2012)CrossRefGoogle Scholar
  11. 11.
    Sun, J., Ovsjanikov, M., Guibas, L.J.: A concise and provably informative multi-scale signature based on heat diffusion. Comput. Graph. Forum 28(5), 1383–1392 (2009)CrossRefGoogle Scholar
  12. 12.
    Kokkinos, I., Bronstein, M.M., Yuille, A.: Dense scale-invariant descriptors for images and surfaces. Research Report, INRIA RR-7914 (2012)Google Scholar
  13. 13.
    Fang, Y., Sun, M., Kim, M., Ramani, K.: Heat-mapping: a robust approach toward perceptually consistent mesh segmentation. In: Proc. CVPR, pp. 2145–2152 (2011)Google Scholar
  14. 14.
    Aubry, M., Schlickewei, U., Cremers, D.: The wave kernel signature: a quantum mechanical approach to shape analysis. In: Proceedings of computational methods for the innovative design of electrical devices, pp. 1626–1633 (2011)Google Scholar
  15. 15.
    Rustamov, R.M.: Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In: Proceedings of symposium on geometry processing, pp. 225–233 (2007)Google Scholar
  16. 16.
    Reuter, M., Wolter, F., Peinecke, N.: Laplace-Beltrami spectra as ’Shape-DNA’ of surfaces and solids. Comput. Aided Design 38(4), 342–366 (2006)CrossRefGoogle Scholar
  17. 17.
    Fang, Y., Sun, M., Ramani, K.: Temperature distribution descriptor for robust 3D shape retrieval. In: Proceedings of workshop on non-rigid shape analysis and deformable image alignment, CVPR (2011)Google Scholar
  18. 18.
    EL Khoury, R., Vandeborre, J.-P., Daoudi, M.: Indexed heat curves for 3D-model retrieval. In: Proceedings of ICPR (2012)Google Scholar
  19. 19.
    Rosenberg, S.: The Laplacian on a Riemannian manifold. Cambridge University Press, Cambridge (1997)CrossRefMATHGoogle Scholar
  20. 20.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Numerical geometry of non-rigid shapes. Springer Berlin (2008)MATHGoogle Scholar
  21. 21.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  22. 22.
    Abdel-Hakim, A.E., Farag, A.: CSIFT: a SIFT descriptor with color invariant characteristics. In: Proceedings of CVPR, pp. 1978–1983 (2006)Google Scholar
  23. 23.
    Bronstein, A.M., Bronstein, M.M., Guibas, L.J., Ovsjanikov, M.: Shape google: seometric words and expressions for invariant shape retrieval. ACM Trans. Graph. 30(1) (2011)Google Scholar
  24. 24.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of CVPR, pp. 2169–2178 (2006)Google Scholar
  25. 25.
    Shi, Y., Lai, R., Krishna, S., Dinov, I., Toga, A.W.: Anisotropic Laplace-Beltrami eigenmaps: bridging reeb graphs and skeletons. In: Proceedings of CVPR Workshops, pp. 1–7 (2008)Google Scholar
  26. 26.
    Funkhouser, T.A., Min, P., Kazhdan, M.M., Chen, J., Halderman, J.A., Dobkin, D.P., Jacobs, D.P.: A search engine for 3D models. ACM Trans. Graph. 22(1), 83–105 (2003)CrossRefGoogle Scholar
  27. 27.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)CrossRefGoogle Scholar
  28. 28.
    Kortgen, M., Park, G.-J., Novotni, M., Klein, R.: 3D shape matching with 3D shape contexts. In: The 7th central European seminar on computer graphics (2003)Google Scholar
  29. 29.
    Osada, R., Funkhouser, T.A., Chazelle, B., Dobkin, D.P.: Shape distributions. ACM Trans. Graph. 21(4), 807–832 (2002)CrossRefGoogle Scholar
  30. 30.
    Kazhdan, M.M., Funkhouser, T.A., Rusinkiewicz, S.: Rotation invariant spherical harmonic representation of 3D shape descriptors. In: Proceedings of symposium on geometry processing, pp. 156–165 (2003)Google Scholar
  31. 31.
    Elad, A., Kimmel, R.: On bending invariant signatures for surfaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1285–1295 (2003)CrossRefGoogle Scholar
  32. 32.
    Ben Hamza, A., Krim, H.: Geodesic matching of triangulated surfaces. IEEE Trans. Image Process. 15(8), 2249–2258 (2006)CrossRefGoogle Scholar
  33. 33.
    Jain, V., Zhang, H., Kaick, O.V.: Non-rigid spectral correspondence of triangle meshes. Int. J. Shape Model. 13(1), 101–124 (2007)CrossRefMATHMathSciNetGoogle Scholar
  34. 34.
    Coifman, R., Lafon, S.: Diffusion maps. Appl. Comput. Harmonic Anal. 21(1), 5–30 (2006)CrossRefMATHMathSciNetGoogle Scholar
  35. 35.
    Levy, B.: Laplace-Beltrami eigenfunctions: towards an algorithm that understands geometry. In: Proceedings of IEEE international conference on shape modeling and applications, pp. 13–20 (2006)Google Scholar
  36. 36.
    Bronstein, M.M., Bronstein, A.M.: Shape recognition with spectral distances. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 1065–1071 (2011)CrossRefGoogle Scholar
  37. 37.
    Fouss, F., Pirotte, A., Renders, J., Saerens, M.: Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Trans. Knowl. Data Eng. 19(3), 355–369 (2007)CrossRefGoogle Scholar
  38. 38.
    Lipman, Y., Rustamov, RM., Funkhouser, T.A.: Biharmonic distance. ACM Trans. Graph. 29(3) (2010)Google Scholar
  39. 39.
    Zhang, H., Kaick, O.V., Dyer, R.: Spectral mesh processing. Comput. Graph. Forum 29(6), 1865–1894 (2010)CrossRefGoogle Scholar
  40. 40.
    Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: Proceedings of ICCV, pp. 1470–1477 (2003)Google Scholar
  41. 41.
    Gemert, J.V., Snoek, C.G.M., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.: Comparing compact codebooks for visual categorization. Comput. Vis. Image Underst. 114(4), 450–462 (2010)CrossRefGoogle Scholar
  42. 42.
    Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification. In: Proceedings of CVPR, pp. 1–8 (2008)Google Scholar
  43. 43.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: improving particular object retrieval in large scale image databases. In: Proceedings of CVPR, pp. 1–8 (2008)Google Scholar
  44. 44.
    Gemert, J.V., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.: Visual word ambiguity. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1271–1283 (2010)CrossRefGoogle Scholar
  45. 45.
    Yang, J., Yu, K., Gong, Y., Huang, T.S.: Linear spatial pyramid matching using sparse coding for image classification. In: Proceedings of CVPR, pp. 1794–1801 (2009)Google Scholar
  46. 46.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T.S., Gong, Y.: Locality-constrained linear coding for image classification. In: Proceedings of CVPR, pp. 3360–3367 (2010)Google Scholar
  47. 47.
    Perronnin, F., Dance, C.R.: Fisher kernels on visual vocabularies for image categorization. In: Proceedings of CVPR, pp. 1–8 (2007)Google Scholar
  48. 48.
    Jégou, H., Douze, M., Schmid, C., Perez, P.: Aggregating local descriptors into a compact image representation. In: Proceedings of CVPR, pp. 3304–3311 (2010)Google Scholar
  49. 49.
    Jégou, H., Perronnin, F., Douze, M., Sánchez, J., Pérez, P., Schmid, C.: Aggregating local images descriptors into compact codes. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1704–1716 (2012)CrossRefGoogle Scholar
  50. 50.
    Picard, D., Gosselin, P.: Improving image similarity with vectors of locally aggregated tensors. In: Proceedings of ICIP, pp. 669–672 (2011)Google Scholar
  51. 51.
    Bronstein, A.M., Bronstein, M.M.: Spatially-sensitive affine-invariant image descriptors. In: Proceedings of ECCV, pp. 197–208 (2010)Google Scholar
  52. 52.
    Savarese, S., Winn, J.M., Criminisi, A.: Discriminative object class models of appearance and shape by correlatons. In: Proceedings of CVPR, pp. 2033–2040 (2006)Google Scholar
  53. 53.
    Ling, H., Soatto, S.: Proximity distribution kernels for geometric context in category recognition. In: Proceedings of ICCV, pp. 1–8 (2007)Google Scholar
  54. 54.
    Behmo, R., Paragios, N., Prinet, V.: Graph commute times for image representation. In: Proceedings of CVPR, pp. 1–8 (2008)Google Scholar
  55. 55.
    Liu, D., Hua, G., Viola, P.A., Chen, T.: Integrated feature selection and higher-order spatial feature extraction for object categorization. In: Proceedings of CVPR, pp. 1–8 (2008)Google Scholar
  56. 56.
    Grauman, K., Darrell, T.: The pyramid match kernel: Discriminative classification with sets of image features. In: Proceedings of ICCV, pp. 1458–1465 (2005)Google Scholar
  57. 57.
    Cao, Y., Wang, C., Li, Z., Zhang, L., Zhang, L.: Spatial-bag-of-features. In: Proceedings of CVPR, pp. 3352–3359 (2010)Google Scholar
  58. 58.
    Yang, Y., Newsam, S.: Spatial pyramid co-occurrence for image classification. In: Proceedings of ICCV, pp. 1465–1472 (2011)Google Scholar
  59. 59.
    Zhang, Y., Jia, Z., Chen, T.: Image retrieval with geometry-preserving visual phrases. In: Proceedings of CVPR, pp. 809–816 (2011)Google Scholar
  60. 60.
    Jia, Y., Huang, C., Darrell, T.: Beyond spatial pyramids: receptive field learning for pooled image features. In: Proceedings of CVPR, pp. 3370–3377 (2012)Google Scholar
  61. 61.
    Krapac, J., Verbeek, J.J., Jurie, F.: Modeling spatial layout with fisher vectors for image categorization. In: Proceedings of ICCV, pp. 1487–1494 (2011)Google Scholar
  62. 62.
    Meyer, M., Desbrun, M., Schröder, P., Barr, A.: Discrete differential-geometry operators for triangulated 2-manifolds. In: Visualization and Mathematics III, Springer, Berlin, pp. 35–57 (2003)Google Scholar
  63. 63.
    Wardetzky, M., Mathur, S., Kälberer, F., Grinspun, E.: Discrete Laplace operators: no free lunch. In: Proceedings of Eurographics symposium on geometry processing, pp. 33–37 (2008)Google Scholar
  64. 64.
    Belkin, M., Sun, J., Wang, Y.: Discrete laplace operator on meshed surfaces. In: Proceedings of SCG, pp. 278–287 (2008)Google Scholar
  65. 65.
    Hildebrandt, K., Polthier, K.: On approximation of the Laplace–Beltrami operator and the Willmore energy of surfaces. Comput. Gr. Forum 30(5), 1513–1520 (2011)CrossRefMathSciNetGoogle Scholar
  66. 66.
    Davies, E.B., Safarov, Y. (eds.): Spectral theory and geometry. Cambridge University Press, Cambridge (1999)MATHGoogle Scholar
  67. 67.
    Vaxman, A., Ben-Chen, M.,, Gotsman, C.: A multi-resolution approach to heat kernels on discrete surfaces. ACM Trans. Graph. 29(4) (2010)Google Scholar
  68. 68.
    Uhlenbeck, K.: Generic properties of eigenfunctions. Am. J. Math. 98(4), 1059–1078 (1976)CrossRefMATHMathSciNetGoogle Scholar
  69. 69.
    Jarvelin, K., Kekalainen, J.: IR evaluation methods for retrieving highly relevant documents. In: Proceedings of SIGIR, pp. 41–48 (2000)Google Scholar
  70. 70.
    Lian, Z., Godil, A., Fabry, T., Furuya, T., Hermans, J., Ohbuchi, R., Shu, C., Smeets, D., Suetens, P., Vandermeulen, D., Wuhrer, S.: SHREC’10 track: non-rigid 3D shape retrieval. In: Proceedings of Eurographics/ACM SIGGRAPH Sympo. 3D Object Retrieval, pp. 101–108 (2010)Google Scholar
  71. 71.
    Lian, Z., Godil, A., Bustos, B., Daoudi, M., Hermans, J., Kawamura, S., Kurita, Y., Lavoue, G., Nguyen, H.V., Ohbuchi, R., Ohkita, Y., Ohishi, Y., Reuter, F.P.M., Sipiran, I., Smeets, D., Suetens, P., Tabia, H., Vandermeulen, D.: SHREC ’11 track: Shape retrieval on non-rigid 3D watertight meshes. In: Proceedings of Eurographics/ACM SIGGRAPH symposium on 3D object retrieval, pp. 79–88 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Concordia Institute for Information Systems EngineeringConcordia UniversityMontréalCanada

Personalised recommendations